我使用的是本教程中的代码:https://towardsdatascience.com/how-to-use-selenium-to-web-scrape-with-example-80f9b23a843a

在运行Python脚本时,Web驱动程序确实会选取所需的玩家,但在第一个"有效的"玩家名称之前,列表中会返回多个空值.

options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_experimental_option("detach", True)

driver = webdriver.Chrome(options=options)
driver.get('https://hoopshype.com/salaries/players/')

players = driver.find_elements(By.XPATH,'//td[@class="name"]/a')
players_list = []
for p in players:
    print(p.text)
    players_list.append(p.text)

print(players_list)

控制台返回PLAYS_LIST的下列值:

玩家列表(_L):

['', '', '', '', '', '',... 'Stephen Curry', 'Kevin Durant', ...]

虽然删除空白值很容易,但我想知道是什么导致了空白值,因为判断Chromium中的XPath不会产生任何空白值.

推荐答案

要从网站HOOPSHYPE提取并打印player names,理想情况下,您需要将WebDriverWait归结为visibility_of_all_elements_located(),并且您可以使用以下Locator Strategies之一:

  • 使用CSS_SELECTOR:

    driver.get("https://hoopshype.com/salaries/players/")
    print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "td.name > a")))])
    
  • 使用XPATH:

    driver.get("https://hoopshype.com/salaries/players/")
    print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//td[@class='name']/a[text()]")))])
    
  • 控制台输出:

    ['Stephen Curry', 'Kevin Durant', 'Nikola Jokic', 'LeBron James', 'Joel Embiid', 'Bradley Beal', 'Paul George', 'Kawhi Leonard', 'Giannis Antetokounmpo', 'Damian Lillard', 'Jimmy Butler', 'Klay Thompson', 'Rudy Gobert', 'Fred VanVleet', 'Anthony Davis', 'Trae Young', 'Zach LaVine', 'Luka Doncic', 'Tobias Harris', 'Ben Simmons', 'Pascal Siakam', 'Kyrie Irving', 'Jrue Holiday', 'Karl-Anthony Towns', 'Devin Booker', 'Kristaps Porzingis', 'CJ McCollum', 'James Harden', 'Darius Garland', 'Ja Morant', 'Zion Williamson', 'Jamal Murray', 'Brandon Ingram', 'Michael Porter', 'Shai Gilgeous-Alexander', 'Donovan Mitchell', 'Jayson Tatum', 'Bam Adebayo', "De'Aaron Fox", 'Deandre Ayton', 'Jaylen Brown', 'Gordon Hayward', 'Chris Paul', 'Domantas Sabonis', 'Kyle Lowry', 'Khris Middleton', 'DeMar DeRozan', 'Julius Randle', 'Jordan Poole', 'Jerami Grant', 'Jaren Jackson Jr', 'Tyler Herro', 'Jalen Brunson', 'Cameron Johnson', 'Kyle Kuzma', 'John Collins', 'Brook Lopez', 'Mike Conley', 'Andrew Wiggins', 'Anfernee Simons', 'RJ Barrett', 'Jordan Clarkson', 'Terry Rozier', 'Dillon Brooks', 'Malcolm Brogdon', 'Draymond Green', 'Aaron Gordon', 'Bruce Brown', 'Mikal Bridges', 'Myles Turner', 'Clint Capela', 'Lonzo Ball', 'Spencer Dinwiddie', "De'Andre Hunter", 'Jarrett Allen', 'Jakob Poeltl', 'Bojan Bogdanovic', 'Keldon Johnson', 'Joe Harris', 'Buddy Hield', 'Evan Fournier', 'Bogdan Bogdanovic', 'OG Anunoby', 'Marcus Smart', 'Gary Trent Jr', 'Nikola Vucevic', 'Derrick White', 'Dejounte Murray', 'Duncan Robinson', 'Norman Powell', 'Tim Hardaway Jr', 'Jonathan Isaac', 'Collin Sexton', "D'Angelo Russell", 'Lauri Markkanen', 'Marcus Morris', 'Harrison Barnes', 'Davis Bertans', 'Markelle Fultz', 'Jusuf Nurkic', 'Kevin Porter', 'Rui Hachimura', 'Mitchell Robinson', 'Kevin Huerter', 'Jonas Valanciunas', 'Caris LeVert', 'Luguentz Dort', 'Luke Kennard', 'Kentavious Caldwell-Pope', 'Max Strus', 'Tyus Jones', 'Dorian Finney-Smith', 'Doug McDermott', 'Anthony Edwards', 'Wendell Carter', 'Gary Harris', 'Josh Hart', 'Naz Reid', 'Steven Adams', 'Marvin Bagley', 'Brandon Clarke', 'Grant Williams', 'Dennis Schroeder', 'Daniel Gafford', 'Kelly Olynyk', 'Victor Wembanyama', 'James Wiseman', 'Devonte Graham', 'Richaun Holmes', 'Austin Reaves', 'Herb Jones', 'Chris Boucher', 'Bobby Portis', 'Nicolas Batum', 'Robert Covington', 'Paolo Banchero', 'Robert Williams', 'Coby White', 'Cade Cunningham', 'Talen Horton-Tucker', 'PJ Tucker', 'Maxi Kleber', 'Joe Ingles', 'Ivica Zubac', 'LaMelo Ball', 'Donte DiVincenzo', 'Brandon Miller', 'Terance Mann', 'Matisse Thybulle', 'Gabe Vincent', 'Alec Burks', 'Reggie Bullock', 'Chet Holmgren', 'Larry Nance Jr', 'Landry Shamet', 'Al Horford', 'Malik Monk', 'Tre Jones', 'Jalen Green', 'Patrick Williams', 'Monte Morris', 'Scoot Henderson', 'Nicolas Claxton', 'Jeff Green', "Royce O'Neale", 'Alex Caruso', 'Victor Oladipo', 'Pat Connaughton', 'Jabari Smith', 'Isaiah Hartenstein', 'Kyle Anderson', 'Daniel Theis', 'Grayson Allen', 'Isaac Okoro', 'Evan Mobley', 'Amen Thompson', 'Georges Niang', 'Gary Payton II', 'TJ McConnell', 'Keegan Murray', 'Delon Wright', 'Onyeka Okongwu', 'Scottie Barnes', 'Jock Landale', 'Trey Lyles', 'Moritz Wagner', "De'Anthony Melton", 'Thaddeus Young', 'Ausar Thompson', 'Miles Bridges', 'Vasilije Micic', 'Paul Reed', 'Zach Collins', 'Jaden Ivey', 'Cody Martin', 'Kevon Looney', 'Killian Hayes', 'Jalen Suggs', 'Anthony Black', 'Khem Birch', 'Bennedict Mathurin', 'Obi Toppin', 'Caleb Martin', 'Patrick Mills', 'Danilo Gallinari', 'Cedi Osman', 'Bilal Coulibaly', 'Josh Giddey', "Jae'Sean Tate", 'Cameron Payne', 'Rudy Gay', 'Sasha Vezenkov', 'Shaedon Sharpe', 'Otto Porter', 'Deni Avdija', 'Nassir Little', 'Jevon Carter', 'Kenrich Williams', 'Ricky Rubio', 'Jarace Walker', 'Jonathan Kuminga', 'Devin Vassell', 'Tyrese Haliburton', 'Dyson Daniels', 'JaVale McGee', 'Kira Lewis', 'Dean Wade', 'Aaron Nesmith', 'Taylor Hendricks', 'Cole Anthony', 'Franz Wagner', 'Furkan Korkmaz', 'Jeremy Sochan', 'Cason Wallace', 'Isaiah Stewart', 'Chuma Okeke', 'Davion Mitchell', 'Johnny Davis', 'Jalen Smith', 'Jett Howard', 'Aleksej Pokusevski', 'Reggie Jackson', 'Shake Milton', 'Mason Plumlee', 'Ziaire Williams', 'Ousmane Dieng', 'Dereck Lively', 'Josh Green', 'Jarred Vanderbilt', 'Nickeil Alexander-Walker', 'James Bouknight', 'Jalen Williams', 'Saddiq Bey', 'Gradey Dick', 'Jalen McDaniels', 'Taurean Prince', 'Precious Achiuwa', 'Tyrese Maxey', 'Josh Primo', 'Jalen Duren', 'Danuel House', 'Jordan Hawkins', 'Zeke Nnaji', 'Immanuel Quickley', 'Chris Duarte', 'Ochai Agbaji', 'Kobe Bufkin', 'Payton Pritchard', 'Dwight Powell', 'Troy Brown', 'Seth Curry', 'Moses Moody', 'Mark Williams', 'Jaden McDaniels', 'Keyonte George', 'Malachi Flynn', 'Desmond Bane', 'Russell Westbrook', 'Kevin Love', 'Corey Kispert', 'AJ Griffin', 'Jalen Hood-Schifino', 'Amir Coffey', 'Alperen Sengun', 'Tari Eason', 'Jaime Jaquez', 'Mike Muscala', 'Andre Drummond', 'Trey Murphy', 'Brandin Podziemski', 'Dalen Terry', 'Cameron Whitmore', 'Jake LaRavia', 'Derrick Rose', 'Eric Gordon', 'Justin Holiday', 'Cory Joseph', 'Jae Crowder', 'Robin Lopez', 'Cody Zeller', 'Patrick Beverley', 'Alex Len', 'Tre Mann', 'Noah Clowney', 'Malaki Branham', 'Kai Jones', 'Simone Fontecchio', 'Dante Exum', 'Julian Champagnie', 'Jordan Nwora', 'Dariq Whitehead', 'Christian Braun', 'Jalen Johnson', 'Josh Richardson', 'Montrezl Harrell', 'Kris Murray', 'Andrew Nicholson', 'Walker Kessler', 'Josh Okogie', 'Keon Johnson', 'Omer Yurtseven', 'Dewayne Dedmon', 'Olivier-Maxence Prosper', 'David Roddy', 'Malik Beasley', 'Isaiah Jackson', 'Marcus Sasser', 'MarJon Beauchamp', 'Charles Bassey', 'Usman Garuba', 'Damian Jones', 'Kris Dunn', 'Bruno Fernando', 'Ben Sheppard', 'Torrey Craig', 'Thomas Bryant', 'Damion Lee', 'Dennis Smith', 'Blake Wesley', 'Josh Christopher', 'Nick Smith', 'Brice Sensabaugh', 'Ty Jerome', 'Julian Strawther', 'Wendell Moore', 'Kobe Brown', 'Luke Kornet', 'John Konchar', 'Quentin Grimes', 'Nikola Jovic', 'Keita Bates-Diop', 'Drew Eubanks', 'Yuta Watanabe', 'Lonnie Walker', 'Aaron Holiday', 'Mo Bamba', 'Chimezie Metu', 'Patrick Baldwin', 'TyTy Washington', 'Jordan McLaughlin', 'Bones Hyland', 'Peyton Watson', 'Cam Thomas', 'Vlatko Cancar', 'Jaden Springer', "Day'Ron Sharpe", 'Santiago Aldama', 'Jaxson Hayes', 'Cam Reddish', 'Darius Bazley', 'Bol Bol', 'Andrew Nembhard', 'Kobi Simmons', 'DaQuan Jeffries', 'Isaiah Roby', 'Goga Bitadze', 'Luka Samanic', 'Colby Jones', 'Dalano Banton', 'Garrison Mathews', 'Jaylin Williams', 'Caleb Houstan', 'Sam Merrill', 'Isaiah Joe', 'Anthony Gill', 'KJ Martin', 'Xavier Tillman', 'Naji Marshall', 'Carlik Jones', 'Justin Champagnie', 'Sam Hauser', 'Jericho Sims', 'Jordan Goodwin', 'Ishmail Wainright', 'Kessler Edwards', 'Haywood Highsmith', 'AJ Green', 'Jeremiah Robinson-Earl', 'Vit Krejci', 'JT Thor', 'Isaiah Livers', 'BJ Boston', 'Jason Preston', 'Isaiah Todd', 'Miles McBride', 'Aaron Wiggins', 'EJ Liddell', 'Orlando Robinson', 'Leonard Miller', 'Bryce McGowens', 'Jaden Hardy', 'Kennedy Chandler', 'Kenneth Lofton', 'Josh Minott', 'Jabari Walker', 'Christian Koloko', 'Xavier Cooks', 'Ryan Rollins', 'Max Christie', 'Jeenathan Williams', 'Jalen Pickett', 'Julian Phillips', 'Eric Bledsoe', 'Jordan Walsh', 'Hunter Tyson', 'Trayce Jackson-Davis', 'Maxwell Lewis', 'Andre Jackson Jr', 'Chris Livingston', 'Filip Petrusev', 'Mouhamadou Gueye', 'Armaan Franklin', 'Andrew Funk', 'Jazian Gortman', 'Jacob Toppin', 'Toumani Camara', 'Rayan Rupert', 'Nathan Knight', 'Nik Stauskas', 'Juwan Morgan', 'Craig Porter', 'Jaylen Clark', 'Jaylen Martin', 'Seth Lundy', 'Miles Norris', 'JD Davison', 'Jay Scrubb', 'Armoni Brooks', 'Jalen Wilson', 'Amari Bailey', 'Leaky Black', 'Xavier Sneed', 'Adama Sanogo', 'Emoni Bates', 'Isaiah Mobley', 'AJ Lawson', 'Mike Miles', 'Jay Huff', 'Braxton Key', 'Malcolm Cazalon', 'Jared Rhoden', 'Darius Days', 'Trevor Hudgins', 'Collin Gillespie', 'Oscar Tshiebwe', 'Isaiah Wong', 'Moussa Diabate', 'Colin Castleton', 'Cole Swider', "D'Moi Hodge", 'Jacob Gilyard', 'Vince Williams Jr', 'Jamaree Bouyea', 'Dru Smith', 'Omari Moore', 'Lindell Wigginton', 'Luka Garza', 'Dereon Seabron', 'Trevor Keels', 'Keyontae Johnson', 'Kevon Harris', 'Ricky Council', 'Terquavion Smith', 'Azuolas Tubelis', 'Saben Lee', 'Ibou Badji', 'John Butler', 'Keon Ellis', 'Jalen Slawson', "Sir'Jabari Rice", 'Ron Harper Jr', 'Markquis Nowell', 'Joey Hauser', 'Johnny Juzang', 'Micah Potter', 'Quenton Jackson', 'Eugene Omoruyi', 'Malik Fitts', 'Didi Louzada']
    
  • Note:您必须添加以下导入:

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    

Python相关问答推荐

如何在Deliveryter笔记本中从同步上下文正确地安排和等待Delivercio代码中的结果?

Python在tuple上操作不会通过整个单词匹配

Polars比较了两个预设-有没有方法在第一次不匹配时立即失败

为什么tkinter框架没有被隐藏?

如果值不存在,列表理解返回列表

将pandas Dataframe转换为3D numpy矩阵

我对我应该做什么以及我如何做感到困惑'

梯度下降:简化要素集的运行时间比原始要素集长

Python+线程\TrocessPoolExecutor

isinstance()在使用dill.dump和dill.load后,对列表中包含的对象失败

启动带有参数的Python NTFS会导致文件路径混乱

在方法中设置属性值时,如何处理语句不可达[Unreacable]";的问题?

基于另一列的GROUP-BY聚合将列添加到Polars LazyFrame

当单元测试失败时,是否有一个惯例会抛出许多类似的错误消息?

如何在Gekko中处理跨矢量优化

PYTHON中的pd.wide_to_long比较慢

使用xlsxWriter在EXCEL中为数据帧的各行上色

如何将验证器应用于PYDANC2中的EACHY_ITEM?

ValueError:必须在Pandas 中生成聚合值

如何计算Pandas 中具有特定条件的行之间的天差