相关套餐:

import pandas as pd
import json

我正在努力将Twitter API的一些JSON输出正常化.所有的事情都是用Python来完成的.我正在使用pd.json_normalize函数来try 标准化下面的JSON输出,并且我try 了许多方法.我似乎遇到的问题是,响应的引用Tweet部分并不是在每个响应中都出现(可能是因为该响应没有引用的Twets).我没有在任何论坛或在线文章中看到过这种情况,所以希望我们能在这里得到一个很好的解决方案,供我自己和其他遇到这个问题的人参考.

我一直在用来try 正常化的代码如下所示

df_test=pd.json_normalize(json_response, 
                          record_path='data',
                          errors='ignore'
                          )

我已经对参数进行了大量的修改,但无法使其发挥作用.希望一个更聪明的程序员能帮助我解决这个问题.

以下是JSON:

{
    "data": [
        {
            "author_id": "2244994945",
            "created_at": "2022-08-12T17:00:25.000Z",
            "id": "1558136333867970560",
            "public_metrics": {
                "like_count": 11,
                "quote_count": 0,
                "reply_count": 0,
                "retweet_count": 7
            },
            "referenced_tweets": [
                {
                    "id": "1558084759812710401",
                    "type": "quoted"
                }
            ],
            "text": "\ud83d\udce3There\u2019s still time to register for #Chirp!\u23f"
        },
        {
            "author_id": "2244994945",
            "created_at": "2022-08-11T22:04:00.000Z",
            "id": "1557850345392377856",
            "public_metrics": {
                "like_count": 18,
                "quote_count": 0,
                "reply_count": 2,
                "retweet_count": 5
            },
            "text": "Discover how @ListenFirst, a social analytics platform and Twitter @OfficialPartner, improved their efficiency and accuracy by leveraging the v2 API. \ud83d\udca1\ud83d\udcbb\n\nRead it here \u2b07\ufe0f \"
        },
        {
            "author_id": "2244994945",
            "created_at": "2022-08-11T21:35:15.000Z",
            "id": "1557843109681123328",
            "public_metrics": {
                "like_count": 14,
                "quote_count": 0,
                "reply_count": 1,
                "retweet_count": 3
            },
            "referenced_tweets": [
                {
                    "id": "1553987008355254272",
                    "type": "quoted"
                }
            ],
            "text": "\ud83d\ude80\ud83e\udd29\u2728"
        },
        {
            "author_id": "2244994945",
            "created_at": "2022-08-11T21:25:01.000Z",
            "id": "1557840534252662784",
            "public_metrics": {
                "like_count": 1,
                "quote_count": 0,
                "reply_count": 0,
                "retweet_count": 0
            },
            "referenced_tweets": [
                {
                    "id": "1557737011041038337",
                    "type": "replied_to"
                }
            ],
            "text": "@talkshrey @talkshrey this is awesome, we can\u2019t wait to see what you build! \ud83d\ude80"
        },
        {
            "author_id": "2244994945",
            "created_at": "2022-08-11T18:00:09.000Z",
            "id": "1557788977842765827",
            "public_metrics": {
                "like_count": 18,
                "quote_count": 0,
                "reply_count": 2,
                "retweet_count": 8
            },
            "text": "\ud83d\udce3 There is only ONE DAY LEFT to register to attend the #Chirp Developer Conference in person! \n\nGet your applications in by 11:59pm PT on Friday, August 12, so you don\u2019t miss out on your chance to attend. \ud83d\udcc5\ud83d\udc40\n\nRegister here \u2935\ufe0f "
        },
        {
            "author_id": "1555310752176668675",
            "created_at": "2022-08-11T16:30:53.000Z",
            "id": "1557766512915742722",
            "public_metrics": {
                "like_count": 0,
                "quote_count": 0,
                "reply_count": 0,
                "retweet_count": 0
            },
            "text": "yo! what's wrong with the bird app today??\ni can't follow, neither be followed!!?\n#twitter #TwitterDown #twitterdev"
        },
        {
            "author_id": "1550706584204021762",
            "created_at": "2022-08-11T05:59:09.000Z",
            "id": "1557607534596431873",
            "public_metrics": {
                "like_count": 1,
                "quote_count": 0,
                "reply_count": 0,
                "retweet_count": 0
            },
            "referenced_tweets": [
                {
                    "id": "1539383247355793408",
                    "type": "replied_to"
                }
            ],
            "text": "@fake_Awareness @Eminem I was thinking about enter #TwitterDev #Chirp and writing a application for #HackAThon that would find every scammers account by AI algorithms."
        },
        {
            "author_id": "1512404349598957573",
            "created_at": "2022-08-10T18:12:36.000Z",
            "id": "1557429723357401088",
            "public_metrics": {
                "like_count": 0,
                "quote_count": 0,
                "reply_count": 0,
                "retweet_count": 0
            },
            "text": "#TwitterDev"
        },
        {
            "author_id": "1389763639159230464",
            "created_at": "2022-08-10T15:22:48.000Z",
            "id": "1557386991607627776",
            "public_metrics": {
                "like_count": 0,
                "quote_count": 0,
                "reply_count": 0,
                "retweet_count": 0
            },
            "text": "#twitterdeveloper #twitterdev #twitterapi\nwhy can i only make sense of it doesn't drast\u2026."
        },
        {
            "author_id": "2244994945",
            "created_at": "2022-08-09T17:58:58.000Z",
            "id": "1557063904655654912",
            "public_metrics": {
                "like_count": 11,
                "quote_count": 0,
                "reply_count": 1,
                "retweet_count": 4
            },
            "referenced_tweets": [
                {
                    "id": "1557063868551114752",
                    "type": "replied_to"
                }
            ],
            "text": "\u23f3 Time is running out to enter the #ChirpDevChallenge!\n\nSubmit your app by August 19th with an app in one of our three categories and help people unlock brand new experiences on Twitter. \ud83d\udcbb\u2699\ufe0f\n\nJoin the challenge \u2b07"
        }
    ],
    "meta": {
        "newest_id": "1558136333867970560",
        "oldest_id": "1557063904655654912",
        "result_count": 10
    }
}

推荐答案

如果dct是你的词典中的问题,你可以做:

df = pd.DataFrame(dct["data"]).explode("referenced_tweets")
df["referenced_id"] = np.where(
    df["referenced_tweets"].notna(), df["referenced_tweets"].str["id"], ""
)

df["referenced_type"] = np.where(
    df["referenced_tweets"].notna(), df["referenced_tweets"].str["type"], ""
)

df.pop("referenced_tweets")

df = pd.concat([df, df.pop("public_metrics").apply(pd.Series)], axis=1)

print(df)

打印:

             author_id                created_at                   id                                                                                                                                                                                                                                    text        referenced_id referenced_type  like_count  quote_count  reply_count  retweet_count
0           2244994945  2022-08-12T17:00:25.000Z  1558136333867970560                                                                                                                                                                                            ????There’s still time to register for #Chirp!ȿ  1558084759812710401          quoted          11            0            0              7
1           2244994945  2022-08-11T22:04:00.000Z  1557850345392377856                                                             Discover how @ListenFirst, a social analytics platform and Twitter @OfficialPartner, improved their efficiency and accuracy by leveraging the v2 API. ????????\n\nRead it here ⬇️                                               18            0            2              5
2           2244994945  2022-08-11T21:35:15.000Z  1557843109681123328                                                                                                                                                                                                                                     ????????✨  1553987008355254272          quoted          14            0            1              3
3           2244994945  2022-08-11T21:25:01.000Z  1557840534252662784                                                                                                                                                           @talkshrey @talkshrey this is awesome, we can’t wait to see what you build! ????  1557737011041038337      replied_to           1            0            0              0
4           2244994945  2022-08-11T18:00:09.000Z  1557788977842765827  ???? There is only ONE DAY LEFT to register to attend the #Chirp Developer Conference in person! \n\nGet your applications in by 11:59pm PT on Friday, August 12, so you don’t miss out on your chance to attend. ????????\n\nRegister here ⤵️                                                18            0            2              8
5  1555310752176668675  2022-08-11T16:30:53.000Z  1557766512915742722                                                                                                                   yo! what's wrong with the bird app today??\ni can't follow, neither be followed!!?\n#twitter #TwitterDown #twitterdev                                                0            0            0              0
6  1550706584204021762  2022-08-11T05:59:09.000Z  1557607534596431873                                                                 @fake_Awareness @Eminem I was thinking about enter #TwitterDev #Chirp and writing a application for #HackAThon that would find every scammers account by AI algorithms.  1539383247355793408      replied_to           1            0            0              0
7  1512404349598957573  2022-08-10T18:12:36.000Z  1557429723357401088                                                                                                                                                                                                                             #TwitterDev                                                0            0            0              0
8  1389763639159230464  2022-08-10T15:22:48.000Z  1557386991607627776                                                                                                                                              #twitterdeveloper #twitterdev #twitterapi\nwhy can i only make sense of it doesn't drast….                                                0            0            0              0
9           2244994945  2022-08-09T17:58:58.000Z  1557063904655654912                ⏳ Time is running out to enter the #ChirpDevChallenge!\n\nSubmit your app by August 19th with an app in one of our three categories and help people unlock brand new experiences on Twitter. ????⚙️\n\nJoin the challenge ⬇  1557063868551114752      replied_to          11            0            1              4

编辑:最新答案为public_metrics

Python相关问答推荐

使用Python异步地持久跟踪用户输入

Python将一个列值分割成多个列,并保持其余列相同

Js的查询结果可以在PC Chrome上显示,但不能在Android Chrome、OPERA和EDGE上显示,而两者都可以在Firefox上运行

在第一次调用时使用不同行为的re. sub的最佳方式

如何为需要初始化的具体类实现依赖反转和接口分离?

Python OPCUA,modbus通信代码运行3小时后出现RuntimeError

Pandas:将值从一列移动到适当的列

python3中np. divide(x,y)和x/y有什么区别?'

是否将列表分割为2?

运行从Airflow包导入的python文件,需要airflow实例?

Pandas查找给定时间戳之前的最后一个值

使用Numpy进行重写For循环矢量化

在单个图形上使用Pandas DataFrame.lot()绘制比例大不相同且x轴相同的条形图和y线条

.yml不会在专用环境中安装来自.requirements.txt的软件包

获取Python中的层次 struct 数据

基于反向特征 Select 的SCRICKIT-LINE和PCA特征 Select

使用boto3检索AWS机密

PYTHON中的regex与PostgreSQL等其他系统中的regex有区别吗?

POLARS:从GROUP_BY列表中 Select 值,并从另一列中 Select 值

Pandas Groupby在try 聚合列时引发ValueError:LEN(索引)!=len(标签)