这里的工作是将一个从https://xxx.xxx.xxx/xxx/1.json
到https://xxx.xxx.xxx/xxx/1417749.json
的API从一个站点中删除,并将其准确地写入mongodb.为此,我有以下代码:
client = pymongo.MongoClient("mongodb://127.0.0.1:27017")
db = client["thread1"]
com = db["threadcol"]
start_time = time.time()
write_log = open("logging.log", "a")
min = 1
max = 1417749
for n in range(min, max):
response = requests.get("https:/xx.xxx.xxx/{}.json".format(str(n)))
if response.status_code == 200:
parsed = json.loads(response.text)
inserted = com.insert_one(parsed)
write_log.write(str(n) + "\t" + str(inserted) + "\n")
print(str(n) + "\t" + str(inserted) + "\n")
write_log.close()
但完成这项任务需要很多时间.这里的问题是我如何加快这个过程.