Python 无法访问网站上的某些标签以进行网络抓取

这里的主要问题是，如前所述，数据是动态加载的，所以您可以从它的来源获取它.

https://s3.amazonaws.com/tripdata将提供一个包含您的数据的XML文件，只需迭代它的项并 Select 您的URL.

Example

import requests
from bs4 import BeautifulSoup

base_url = 'https://s3.amazonaws.com/tripdata/'
links = [
         base_url+key.text
         for key in BeautifulSoup(requests.get(base_url).text).select('Key')
        ]
links

Output

['https://s3.amazonaws.com/tripdata/201306-citibike-tripdata.zip',
 'https://s3.amazonaws.com/tripdata/201307-201402-citibike-tripdata.zip',
 'https://s3.amazonaws.com/tripdata/201307-citibike-tripdata.zip',
 'https://s3.amazonaws.com/tripdata/201308-citibike-tripdata.zip',
 'https://s3.amazonaws.com/tripdata/201309-citibike-tripdata.zip',
 'https://s3.amazonaws.com/tripdata/201310-citibike-tripdata.zip',...]

根据你的 comments ，你可以使用dict或list(满分dicts):

import requests
from bs4 import BeautifulSoup

base_url = 'https://s3.amazonaws.com/tripdata/'

data = []
         
for e in BeautifulSoup(requests.get(base_url).text).select('Contents'):
        data.append({
                'url':base_url+e.key.text,
                'date': e.lastmodified.text
        })
data