这些信息是通过Java脚本XHR调用从API反馈到页面的[这些XHR调用是由Java脚本在加载初始页面的HTML之后进行的，因此请求无法看到数据，因为它不能执行Java脚本].您需要删除该API端点(您可以在开发工具-网络选项卡中找到它).以下是一种方法:
import requests
import pandas as pd
from tqdm import tqdm ## if using Jupyter notebook, import as: from tqdm.notebook import tqdm
s = requests.Session()
big_df = pd.DataFrame()
pd.set_option('display.max_columns', None)
pd.set_option('display.max_colwidth', None)
for x in tqdm(range(2, 5)):
r = s.get(f'https://uk.trustpilot.com/_next/data/businessunitprofile-consumersite-5670/review/bupa.co.uk.json?page={x}&businessUnit=bupa.co.uk')
df = pd.json_normalize(r.json()['pageProps']['reviews'])
big_df = pd.concat([big_df, df], axis=0, ignore_index=True)
print(big_df)
结果为终端:
100%
1/1 [00:01<00:00, 1.21s/it]
id filtered pending text rating title likes report hasUnhandledReports consumersReviewCountOnSameDomain consumersReviewCountOnSameLocation productReviews language location labels.merged labels.verification.isVerified labels.verification.createdDateTime labels.verification.reviewSourceName labels.verification.verificationSource labels.verification.verificationLevel dates.experiencedDate dates.publishedDate dates.updatedDate consumer.id consumer.displayName consumer.imageUrl consumer.numberOfReviews consumer.countryCode consumer.hasImage consumer.isVerified reply.message reply.publishedDate reply.updatedDate
0 633340161133a76521b9a54a False False Having just received a breast cancer diagnosis from a routine mammogram. I was unfortunately let down by the NHS.dropped me out of the system.\nFortunately, I have had medical insurance with BUPA.for the past 48 yrs.So I contacted them & they were very instrumental in getting me the treatment I needed very quickly. 5 Let down by NHS. 1 None False 1 None [] en None None True 2022-09-27T20:25:26.000Z BusinessGeneratedLink invitation invited 2021-10-07T00:00:00.000Z 2022-09-27T20:25:26.000Z None 573c3c860000ff000a212594 Hilary Davies 7 GB False False Hi Hilary, thank you for sharing your experience. Wishing you all the very best. Amy 2022-09-28T08:50:31.797Z None
1 6333f9061c24523f3ca038d2 False False I used the dental care plan and was so pleased with how quick and easy it was to put a claim through. After sending my receipt which included breakdown of dental work I received I received the funds straight into my nominated bank account within 10 days. No fuss or multiple emails/calls required. Super service! 5 Dental Care plan- super easy claim process 0 None False 1 None [] en None None True 2022-09-28T09:34:30.000Z BusinessGeneratedLink invitation invited 2022-09-14T00:00:00.000Z 2022-09-28T09:34:30.000Z None 5e9b08555efa59374f58b372 Jessica 4 GB False False Great, thanks for sharing your experience Jessica. Amy 2022-09-28T09:42:50.036Z None
2 633322a61133a76521b983b9 False False Bupa left a text on my phone that they needed to speak to me about a medical issue. They had given me an authority number. \n\nI have tried six times to get through. No name was left. I couldn't find a human being to talk to. This is weeks ago. No one has tried again. The message was left on date below 1 Bupa deals with insurance and takes money every month. It shouldn't be so difficult dealing with them when it comes to paying for a medical issue. 0 None False 1 None [] en None None True 2022-09-27T18:19:50.000Z BusinessGeneratedLink invitation invited 2022-09-12T00:00:00.000Z 2022-09-27T18:19:50.000Z None 58e7abeb0000ff000a8ac3aa a levin 2 GB False False Hi there, I'm really sorry about this, I've asked the team to give you a call to discuss the reason for the Text, you'll hear from them shortly. Thanks Amy 2022-09-28T08:37:11.614Z None
3 633696ee3d107cfdfccef1fa False False BUPA processed my request quickly and allowed me the care I needed however they raised my fee from £135 to £206 a month while I was in the middle of treatment so was unable to look elsewhere. 3 BUPA processed my request quickly 1 None False 1 None [] en None None True 2022-09-30T09:12:46.000Z BusinessGeneratedLink invitation invited 2022-07-28T00:00:00.000Z 2022-09-30T09:12:46.000Z None 5f1b27d613b1ca629d80359d Judy March 9 GB False False Hi Judy. Thanks for your feedback. You may wish to give your Loyalty team a call on 0800 010 383 to discuss the cost of your policy in more detail, and to see if there any options open to you. Many thanks, Brian 2022-09-30T10:05:44.158Z None
4 6332c3f5484ccb2c525e811c False False All very straight forward when making a claim just make sure you talk to BUPA first to check your cover then get a private referral from a GP, go to the website and choose your consultant / Hospital and give them a call to arrange an appointment. I added my wife to my policy a couple of years ago and glad I did as she recently needed some quick diagnoses and treatment for heart condition. All sorted within 6 weeks of seeing the consultant. 5 Quick and simple from diagnoses to treatment. 0 None False 1 None [] en None None True 2022-09-27T11:35:49.000Z BusinessGeneratedLink invitation invited 2022-08-15T00:00:00.000Z 2022-09-27T11:35:49.000Z None 5ea9d7c9bbf1afc1c1f68ef6 Philip 12 GB False False Thanks for taking the time to review us Phillip, we really appreciate it. Amy 2022-09-27T11:50:14.607Z None
5 6332da211c24523f3c9f71e5 False False Cover was not enough to complete my treatment. I appreciate that I pay a certain amount of cover however, there was no way I could complete treatment and consultations with the benefit received. Surely enough cover should be provided to complete treatment. 3 Insufficient cover 1 None False 1 None [] en None None True 2022-09-27T13:10:25.000Z BusinessGeneratedLink invitation invited 2022-08-28T00:00:00.000Z 2022-09-27T13:10:25.000Z None 600ff7dd771237001951b6fd Diane 3 GB False False Hi Diane, thanks for your comments. Sorry you feel that your cover is insufficient. We would happily discuss your policy options with you at your renewal. Amy 2022-09-27T13:22:00.300Z None
6 6332cd761133a76521b92128 False False As over the years, during my last contact with Bupa staff member to get authorisation for consultation, I did not experience any issue what so ever. The staff member was very helpful, knowledgeable and understanding. Mine is a complex which has not been professionally diagnosed but only I know what my issue is. Throughout I have had immense moral support on top of medical support which has been very valuable. \nUnderstanding of most staff members has been above and beyond expectations. 5 Bupa has been my life saver. 0 None False 1 None [] en None None True 2022-09-27T12:16:22.000Z BusinessGeneratedLink invitation invited 2022-09-27T00:00:00.000Z 2022-09-27T12:16:22.000Z None 5992b48b0000ff000abbf4db Manju 9 GB False False Thanks for sharing your experience Manju, we really appreciate it. All the best, Amy. 2022-09-27T12:22:09.148Z None
7 6332d0f31133a76521b924dd False False Poor at best:\n\n1) Admin, confusion when mistakes are made by them, I have to sort their errors out.\n2) Changing terms and what I can claim for, meaning that I'm going through the NHS for me treatment.\n\nI can go on but pointless as you get my drift. 1 Poorest provider yet 0 None False 1 None [] en None None True 2022-09-27T12:31:15.000Z BusinessGeneratedLink invitation invited 2022-09-23T00:00:00.000Z 2022-09-27T12:31:15.000Z None 5cb772a47fe10841711b6bea Guido 4 GB False False Hi Guido, I'm really sorry that this is how you feel after your recent experience. It looks like we have discussed your concerns, but if you would like to discuss this any further, or have any questions we'd be happy to help, call our Customer Relations team on 0345 606 6739. Amy 2022-09-27T12:57:42.082Z None
8 63342e5c1133a76521ba2ec0 False False Am in Bupa with my employer.\nHaving various tests done and need \nBupa’s approval to go ahead.\nAt this stressful time for me they have been brilliant. Call handlers polite ever so friendly and extremely helpful. 5 Am in Bupa with my employer. 0 None False 1 None [] en None None True 2022-09-28T13:22:04.000Z BusinessGeneratedLink invitation invited 2022-09-28T00:00:00.000Z 2022-09-28T13:22:04.000Z None 5c61a0faf7edcbadbf39934d Peter Francis. Somerset 4 GB False False Thank you for taking the time to review us Peter. Wishing you all the best with any further treatment. Amy 2022-09-29T09:01:26.925Z None
9 6332ce3d1c24523f3c9f654f False False Not great to be honest, it isn’t this stressful on the NHS. Very disappointing, constantly having to call them, I was under the impression paying private would make life easier, I couldn’t have been more wrong. Dreadful service. Paid more out in excess than what the treatment actually cost? 1 Not great to be honest 0 None False 1 None [] en None None True 2022-09-27T12:19:41.000Z BusinessGeneratedLink invitation invited 2022-09-27T00:00:00.000Z 2022-09-27T12:19:41.000Z None 6332ce3c56accb0012f8db20 MJH 1 GB False False Hi there, thank you for taking the time to share your experience. I can see that we've looked into your concerns and responded to you. If you feel that you would like to discuss this further or if you're unhappy with our response, your next steps will be noted on our correspondence. Amy 2022-09-27T13:01:57.551Z None
10 6332ca3a1c24523f3c9f610e False False As I get older I get more and more grateful that I have BUPA to contact if I need serious medical attention not readily available on the NHS. Their systems for processing inquiries and authorisations is very efficient. And I am very impressed by the BUPA hospital near me, the Cromwell. 5 As I get older I get more and more… 0 None False 1 None [] en None None True 2022-09-27T12:02:34.000Z BusinessGeneratedLink invitation invited 2022-09-15T00:00:00.000Z 2022-09-27T12:02:34.000Z None 4fd6243600006400011afc2a Ian Dunlop 12 GB False False Hi Ian, thanks for taking the time to review us. We're really pleased this is how you feel after your recent experience. Thank you, Amy. 2022-09-27T12:19:55.899Z None
11 6332e3161133a76521b938a7 False False Had a tooth ???? broken and went to Malmesbury Dental Facility, they fixed it on the day and submitted my claim through the Bupa app. Received confirmation immediately and my claim was paid within 3 days. Great and efficient service ! ???? 5 Had a tooth ???? broken and went to … 0 None False 1 None [] en None None True 2022-09-27T13:48:39.000Z BusinessGeneratedLink invitation invited 2022-09-09T00:00:00.000Z 2022-09-27T13:48:39.000Z None 61939eeed5fd380012bff341 Mr Biggs 3 GB False False Hi Mr Biggs. It's really pleasing to hear of the excellent service you received at a difficult time. Best wishes, Brian 2022-09-27T14:55:51.644Z None
12 6332a70b5871ff1ef90b50a6 False False I broke my hand. First Bupa did not approve saying need to check if it’s a pre-existing condition!? A broken hand? Then they called to say the cast is not covered!! It was of course not a pre-existing condition and the cast was covered but I guess common sense does not prevail. 2 I broke my hand 0 None False 1 None [] en None None True 2022-09-27T09:32:27.000Z BusinessGeneratedLink invitation invited 2022-08-29T00:00:00.000Z 2022-09-27T09:32:27.000Z 2022-09-27T11:41:26.000Z 54bdda760000ff0001a547f5 Suzie Pilkington 7 AE False False Hi Suzie, sorry to se you're unhappy with the terms of your scheme. If there's anything you'd like to discuss further, please let us know. Amy 2022-09-27T09:45:46.722Z None
13 6336060d4a5fff53cad25028 False False A very positive experience. \nI was seen by a consultant within days of my enquiry . It was a very positive experience and my surgery was scheduled for the following week. 5 A very positive experience 0 None False 1 None [] en None None True 2022-09-29T22:54:37.000Z BusinessGeneratedLink invitation invited 2022-09-29T00:00:00.000Z 2022-09-29T22:54:37.000Z None 57a614090000ff000a386665 Sonia 7 GB False False Hi Sonia. Thanks for the wonderful 5 star feedback. I wish you all the best with your surgery and recovery. Regards, Brian 2022-09-30T08:06:25.153Z None
14 63333cb11133a76521b9a198 False False Contacted BUPA as having private Healthcare through work.\nExcellent communication and had a call from the Healthcare provider the following afternoon. All now sorted and appointment with a consultant in a couple of weeks. 5 Contacted BUPA as having private… 1 None False 1 None [] en None None True 2022-09-27T20:10:57.000Z BusinessGeneratedLink invitation invited 2022-09-19T00:00:00.000Z 2022-09-27T20:10:57.000Z None 5d47efe56bc3c146a509621e Marie Stott 11 GB False False Thank you Marie, Amy. 2022-09-28T08:51:01.849Z None
15 6332e07a1133a76521b935b3 False False Bupa like most insurers take an adversarial position in settling claims. They owe me 1500 pounds for 8 months and after arguing with them and spending 6 hours on the phone I finally gave up and forgot the money owed to me. 2 Bupa like most insurers take an… 0 None False 1 None [] en None None True 2022-09-27T13:37:30.000Z BusinessGeneratedLink invitation invited 2022-01-15T00:00:00.000Z 2022-09-27T13:37:30.000Z None 62336118e7dd2a0013cfbac2 customergg 3 GB False False Hi there. Thanks for getting in touch. I see this has already been looked into for you, but please give our Customer Relations team a call on 0345 606 6739 if you'd like to discuss this in more detail. Many thanks, Brian 2022-09-27T14:44:04.578Z None
16 6332b5715871ff1ef90b5e03 False False Well, if it hadn't been for one particular individual, I would have given you five stars. Please feel free to contact me for further details.\n\nFurther to your response Amy, if you want further details, you contact me. You have my contact details in n file !! 1 Insensitive Personel 0 None False 1 None [] en None None True 2022-09-27T10:33:53.000Z BusinessGeneratedLink invitation invited 2022-09-02T00:00:00.000Z 2022-09-27T10:33:53.000Z 2022-09-27T11:33:48.000Z 57fcd0370000ff000a4c74ec Mr Simon Belson 10 GB False False Hi Simon, we really sorry this is the case, we'd like to look into this for you. Please contact our Customer Relations team on 0345 606 3739. Amy 2022-09-27T11:20:56.132Z None
17 6335474e272def1ea67aafcd False False To date Bupa have been slow and ineffective resulting in delays to a treatment I need. \nMy business moved to them recently and I am unimpressed and will review this when renewal is due. 1 To date Bupa have been slow and… 0 None False 1 None [] en None None True 2022-09-29T09:20:46.000Z BusinessGeneratedLink invitation invited 2022-09-08T00:00:00.000Z 2022-09-29T09:20:46.000Z None 62156e4b648564001211ea2a Mr Merton 2 GB False False Hi David, we're really sorry to hear this. I can see this is something we've discussed with you. If you still have concerns please to contact us to discuss further. Amy 2022-09-29T09:42:48.269Z None
18 63349ef3272def1ea67a6a15 False False When I first contacted BUPA to get authorisation for a procedure I could “talk” only to a chat bot on my iPhone; very difficult to get “her” to understand what I wanted. Difficult to see the conversation and unable to save & print it as I don’t have a printer. This resulted in the incorrect code sent to me. It caused me some embarrassment with the clinic. I then phoned again and was able to speak to a person to whom I was able to converse and who eventually issued the correct procedure code. “Speaking” to a chat bot was ineffective. When one is anxious because of a health problem one does not welcome not being able to easily and simply talk to someone. I was relieved that this happened eventually but wonder why BUPA thinks that using chatbot is efficient and progressive. It is not suitable for elderly folk such as me, who is not tech savvy. 1 Contacting BUPA for help 0 None False 1 None [] en None None True 2022-09-28T21:22:27.000Z BusinessGeneratedLink invitation invited 2022-08-08T00:00:00.000Z 2022-09-28T21:22:27.000Z None 52efe2d0000064000161f688 Angela Pitfield 4 GB False False Hi Angela, we're really sorry this was your recent experience. We're working hard to improve our channels of communication but we appreciate we still have some work to do. We really appreciate your feedback so thank you. Amy 2022-09-29T08:26:43.053Z None
19 6332d6741c24523f3c9f6dd8 False False Bupa were great from start to finish! \nEverything got sorted so quickly and Bupa were great at authorising treatment and any questions that I had, would definitely recommend Bupa to anyone! 5 Bupa were great from start to finish 0 None False 1 None [] en None None True 2022-09-27T12:54:44.000Z BusinessGeneratedLink invitation invited 2022-09-27T00:00:00.000Z 2022-09-27T12:54:44.000Z None 6332d673e7e9140012411715 Sophie 1 GB False False Thanks for taking the time to review us Sophie, we really appreciate it. Amy 2022-09-27T13:03:36.057Z None
该数据帧中包含大量信息，您可能希望按您认为合适的方式对其进行分割.此外，还有许多带有 comments 的页面，因此您可能想要相应地调整您的范围(由于某种原因，我在try 从第1页开始时遇到错误，请调查).
有关TQDM，请访问https://pypi.org/project/tqdm/
有关请求文档，请参阅https://requests.readthedocs.io/en/latest/
同样对于大Pandas :https://pandas.pydata.org/pandas-docs/stable/index.html只