在 Python 中对长列表进行高效的过滤、映射和归约操作

回答

Numpy在计算中速度很快，但在将字符串列表转换为int列表时效率较低

小贴士:

1. 仅将Numpay数组与NumPy函数一起使用，将Numpay数据 struct 与Pythonic函数一起使用效率很低，反之亦然
2. 计数器在某些情况下(在计数器内执行转换时)是Pythonic方法中最快的
3. 要获得Numpy效率的好处，请使用Python进行列表转换.

为了更好地比较结果，该问题的运行时间可以分为三类:

1. 读取文件的时间(所有try 都相同)
2. 将字符串列表转换为整型列表的时间(对于pythonlist和numpy数组不同)
3. 计算时间(因方法不同而异)

为了清楚地显示python list和NumPy数组的效率，这种分离是必需的. 请看一下代码:

import time
import numpy as np
from functools import reduce
from collections import Counter


def print_result(total_price, duration, method):
    print('{method:<35s} {0:>20.3f} (sec), total_price={1:<10.3f}'
          .format(duration, total_price, method=method))


start = time.time()
with open('price_list.txt') as f:
    price_list = f.read().split('\n')
print('Duration for reading from file: {0:<50.3f}'.format(time.time() - start))

start = time.time()
np_price_array = np.array(price_list).astype(int)  # uing numpy array
print('Duration for converting to numpy int array: {0:<50.3f}'.format(
    time.time() - start))

start = time.time()
price_list = list(map(int, price_list))
print('Duration for converting to python int list: {0:<50.3f}'.format(
    time.time() - start))

start = time.time()    
total_price = np.sum(np_price_array[np_price_array < 30] * 1.05)
print_result(total_price, time.time() - start, "NumPy function (numpy array)")

start = time.time()
total_price = sum([x for x in np_price_array if x < 30])*1.05
print_result(total_price, time.time() - start,
             "List comprehension (numpy array)")


start = time.time()
total_price = sum((x for x in price_list if x < 30))*1.05
print_result(total_price, time.time() - start, "List generator (Python list)")

start = time.time()
total_price = sum([x for x in price_list if x < 30])*1.05
print_result(total_price, time.time() - start,
             "List comprehension (Python list)")

start = time.time()
total_price = sum([float(x[0]) * float(x[1])
                  for x in Counter(price_list).items() if x[0] < 30]) * 1.05
print_result(total_price, time.time() - start, "Counter (Python list)")

start = time.time()
total_price = reduce(
    lambda x, y: x+y, filter(lambda x: x < 30, price_list)) * 1.05
print_result(total_price, time.time() - start, "functions (Python list)")

以下图表是代码输出的结果: