我有一个dict的list，它遵循一个一致的 struct ，每个dict都有一个由list组成的整数.但是，我需要确保每个dict的ByteSize(转换为JSON字符串时)小于指定的阈值.
如果dict超过字节大小阈值，我需要分块该DICT的整数list.
try :
import json
payload: list[dict] = [
{"data1": [1,2,3,4]},
{"data2": [8,9,10]},
{"data3": [1,2,3,4,5,6,7]}
]
# Max size in bytes we can allow. This is static and a hard limit that is not variable.
MAX_SIZE: int = 25
def check_and_chunk(arr: list):
def check_size_bytes(item):
return True if len(json.dumps(item).encode("utf-8")) > MAX_SIZE else False
def chunk(item, num_chunks: int=2):
for i in range(0, len(item), num_chunks):
yield item[i:i+num_chunks]
# First check if the entire payload is smaller than the MAX_SIZE
if not check_size_bytes(arr):
return arr
# Lets find the items that are small and items that are too big, respectively
small, big = [], []
# Find the indices in the payload that are too big
big_idx: list = [i for i, j in enumerate(list(map(check_size_bytes, arr))) if j]
# Append these items respectively to their proper lists
item_append = (small.append, big.append)
for i, item in enumerate(arr):
item_append[i in set(big_idx)](item)
# Modify the big items until they are small enough to be moved to the small_items list
for i in big:
print(i)
# This is where I am unsure of how best to proceed. I'd like to essentially split the big dictionaries in the 'big' list such that it is small enough where each element is in the 'small' result.
可能的预期结果示例:
payload: list[dict] = [
{"data1": [1,2,3,4]},
{"data2": [8,9,10]},
{"data3": [1,2,3,4]},
{"data3": [5,6,7]}
]