Python 为什么在浏览器开发工具与 BeautifulSoap Postman 中获取不同的数据

发布于03月23日

我想从这个网页上删除数据

我想得到所有的博客...在结果标签(<div class="results">)下

In browser tools there it is showing under result tag there are 10 snippets...

但使用Beautifulsoap我得到了

<div class="results">
</div>

postman 也得到了同样的东西..

我就是这么做的..

topicuri = "https://search.donanimhaber.com/portal?q=vodafone&p=3&devicetype=browsermobile&order=date_desc&in=all&contenttype=all&wordtype=both&range=all"
r = s.get(topicuri)
soup = BeautifulSoup(r.text, 'html.parser')
pages = soup.find('div', {'class': 'results'})
print(pages)

推荐答案

您还可以从api调用json响应中获取数据

import requests
import json 
body= "vodafone"
headers= {
    'content-type': 'application/json'
   }

api_url = "https://search.donanimhaber.com/api/search/portal/?q=vodafone&p=3&devicetype=browsermobile&order=date_desc&in=all&contenttype=all&wordtype=both&daterange=all"

jsonData = requests.post(api_url, data=json.dumps(body), headers=headers).json()

for item in jsonData['contents']:
    categoryName=item['categoryName']
    print(categoryName)

输出:

Operatörler - Kurumsal Haberler
Operatörler - Kurumsal Haberler
Operatörler - Kurumsal Haberler
Mobil Aksesuarlar
Operatörler - Kurumsal Haberler
Kripto Para
Sinema ve Dizi
Mobil Oyunlar
Operatörler - Kurumsal Haberler
Operatörler - Kurumsal Haberler