我一直在试验python中的多处理模块,我想知道如何通过派生的进程来处理不同并行化方法的参数.这里是我使用的代码:
import os
import time
import multiprocessing
class StateClass:
def __init__(self):
self.state = 0
def __call__(self):
return f"I am {id(self)}: {self.state}"
CONTEXT = multiprocessing.get_context("fork")
nb_workers = 2
stato = StateClass()
def wrapped_work_function(a1, a2, sss, qqq):
time.sleep(a1 + 1)
if a1 == 0:
sss.state = 0
else:
sss.state = 123
for eee in a2:
time.sleep(a1 + 1)
sss.state += eee
print(
f"Worker {a1} in process {os.getpid()} (parent process {os.getppid()}): {eee}, {sss()}"
)
return sss
print("main", id(stato), stato)
manager = CONTEXT.Manager()
master_workers_queue = manager.Queue()
work_args_list = [
(
worker_index,
[iii for iii in range(4)],
stato,
master_workers_queue,
)
for worker_index in range(nb_workers)
]
pool = CONTEXT.Pool(nb_workers)
result = pool.starmap_async(wrapped_work_function, work_args_list)
pool.close()
pool.join()
print("Finish")
bullo = result.get(timeout=100)
bullo.append(stato)
for sss in bullo:
print(sss, id(sss), sss.state)
例如,我从中获得以下输出:
main 140349939506416 <__main__.StateClass object at 0x7fa5c449dcf0>
Worker 0 in process 9075 (parent process 9047): 0, I am 140350069832528: 0
Worker 0 in process 9075 (parent process 9047): 1, I am 140350069832528: 1
Worker 1 in process 9077 (parent process 9047): 0, I am 140350069832528: 123
Worker 0 in process 9075 (parent process 9047): 2, I am 140350069832528: 3
Worker 0 in process 9075 (parent process 9047): 3, I am 140350069832528: 6
Worker 1 in process 9077 (parent process 9047): 1, I am 140350069832528: 124
Worker 1 in process 9077 (parent process 9047): 2, I am 140350069832528: 126
Worker 1 in process 9077 (parent process 9047): 3, I am 140350069832528: 129
Finish
<__main__.StateClass object at 0x7fa5c43ac190> 140349938516368 6
<__main__.StateClass object at 0x7fa5c43ac4c0> 140349938517184 129
<__main__.StateClass object at 0x7fa5c449dcf0> 140349939506416 0
初始类实例stato
的id为140349939506416
,并像我所期望的那样在其整个生命周期中保持它.在starmap_async
方法中,我确实得到了同一类的两个不同实例(每个工作者/进程一个实例),我可以修改它们,并且在脚本结束之前保留它们的state
属性.无论如何,这些实例的id最初是相同的(140350069832528
),并且在脚本末尾,它们都有另一个id,这也不同于原始实例的id.