Python 如何使用 Selenium 和 Headless Chrome 从网页中抓取表格

我试图用Python编写一个脚本，从下面的URL中将Product Attributes表导出为Excel文件(或CSV).

我编写了一个脚本并try 了一个不同的类名，但我遇到了一个错误！

URL: https://www.digikey.com/en/products/detail/texas-instruments/uln2003aidre4/1912622个

我不知道这条消息是什么原因，因为我可以从不同的网站导出表，但我的代码在这个网站上崩溃了.(还有Mouser.com)

我有一个理论，我认为这两个网站屏蔽了我的脚本，以避免输出他们的数据，但我不确定.

The table I want to export and its inspection

以下是我的代码:

import time
import requests
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import pandas as pd

def get_specifications_table(url):
    options = Options()
    options.add_argument('--headless')  # Run the browser in headless mode (no visible window)

    driver = webdriver.Chrome(options=options)

    driver.get(url)
    time.sleep(5)  # Add a delay to allow the webpage to load (adjust the time as needed)

    try:
        # Find the element with the specified class name "MuiTable-root css-u6unfi" and extract the table
        class_name = "MuiTable-root.css-u6unfi"
        table_element = driver.find_element("css selector", f".{class_name}")
        table_html = table_element.get_attribute('outerHTML')
        df = pd.read_html(table_html)[0]
        return df
    except Exception as e:
        print("Error:", e)
    finally:
        driver.quit()

    return None

def export_to_excel(df, output_file):
    writer = pd.ExcelWriter(output_file, engine='xlsxwriter')
    df.to_excel(writer, index=False)
    writer.save()
    writer.close()

if __name__ == '__main__':
    url = "https://www.digikey.com/en/products/detail/texas-instruments/uln2003aidre4/1912622"
    output_excel_file = "Specifications_Table_Digikey.xlsx"

    print("Fetching the webpage and extracting the table...")
    specifications_df = get_specifications_table(url)
    
    if specifications_df is not None:
        print("Exporting the table to Excel...")
        export_to_excel(specifications_df, output_excel_file)
        print(f"Table 'Specifications' exported to '{output_excel_file}' successfully.")
    else:
        print("Table extraction or export failed.")

但我面对的是这样一个错误:

Fetching the webpage and extracting the table...
Error: Message: no such element: Unable to locate element: {"method":"css selector","selector":".MuiTable-root.css-u6unfi"}
  (Session info: headless chrome=115.0.5790.110); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
Backtrace:
    GetHandleVerifier [0x004BA813+48355]
    (No symbol) [0x0044C4B1]
    (No symbol) [0x00355358]
    (No symbol) [0x003809A5]
    (No symbol) [0x00380B3B]
    (No symbol) [0x003AE232]
    (No symbol) [0x0039A784]
    (No symbol) [0x003AC922]
    (No symbol) [0x0039A536]
    (No symbol) [0x003782DC]
    (No symbol) [0x003793DD]
    GetHandleVerifier [0x0071AABD+2539405]
    GetHandleVerifier [0x0075A78F+2800735]
    GetHandleVerifier [0x0075456C+2775612]
    GetHandleVerifier [0x005451E0+616112]
    (No symbol) [0x00455F8C]
    (No symbol) [0x00452328]
    (No symbol) [0x0045240B]
    (No symbol) [0x00444FF7]
    BaseThreadInitThunk [0x772500C9+25]
    RtlGetAppContainerNamedObjectPath [0x77BC7B4E+286]
    RtlGetAppContainerNamedObjectPath [0x77BC7B1E+238]

Table extraction or export failed.