Python3.x 将行附加到 DataFrame 的最快和最有效的方法是什么

正如莫希特·莫特瓦尼所建议的，最快的方法是将数据收集到字典中，然后将所有数据加载到数据帧中.以下是一些速度测量示例:

import pandas as pd
import numpy as np
import time
import random

end_value = 10000

用于创建字典列表并在最后将所有字典加载到数据帧中的度量

start_time = time.time()
dictinary_list = []
for i in range(0, end_value, 1):
    dictionary_data = {k: random.random() for k in range(30)}
    dictionary_list.append(dictionary_data)

df_final = pd.DataFrame.from_dict(dictionary_list)

end_time = time.time()
print('Execution time = %.6f seconds' % (end_time-start_time))

执行时间=0.090153秒

将数据添加到列表和concat到数据框中的测量:

start_time = time.time()
appended_data = []
for i in range(0, end_value, 1):
    data = pd.DataFrame(np.random.randint(0, 100, size=(1, 30)), columns=list('A'*30))
    appended_data.append(data)

appended_data = pd.concat(appended_data, axis=0)

end_time = time.time()
print('Execution time = %.6f seconds' % (end_time-start_time))

执行时间=4.183921秒

附加数据帧的测量:

start_time = time.time()
df_final = pd.DataFrame()
for i in range(0, end_value, 1):
    df = pd.DataFrame(np.random.randint(0, 100, size=(1, 30)), columns=list('A'*30))
    df_final = df_final.append(df)

end_time = time.time()
print('Execution time = %.6f seconds' % (end_time-start_time))

执行时间=11.085888秒

按loc使用插入数据的测量:

start_time = time.time()
df = pd.DataFrame(columns=list('A'*30))
for i in range(0, end_value, 1):
    df.loc[i] = list(np.random.randint(0, 100, size=30))


end_time = time.time()
print('Execution time = %.6f seconds' % (end_time-start_time))

执行时间=21.029176秒