Python 新进程不会在运行FastApi的Docker中启动

我在一个停靠容器中运行FastApi，该容器有8个可用的CPU，我想通过python多处理来利用它们.

在docker容器外运行我的程序，一切都像我预期的那样工作(见下文)，但是当我构建和运行容器时，子进程不会启动.

下面是我的应用程序中的相关代码

import multiprocessing
from multiprocessing import Process, Manager, Queue
from contextlib import asynccontextmanager
from fastapi import FastAPI

CLIENT_CORES = 5
connection = "mongo_connection_string"

@asynccontextmanager
async def lifespan(app: FastAPI):
    
    manager = multiprocessing.Manager()

    global dbWorkers
    global clientWorkers

    global clientQueue
    global clientResponse

    

    #resources
    dbQueue = Queue()
    clientQueue = Queue()
    clientResponse = manager.dict()
    psetObjects = manager.dict()
    dbWorkers = []
    clientWorkers = []

    #start client and db workers
    config_processes(dbQueue,clientQueue,clientResponse,psetObjects)

    yield
    
    graceful_shutdown()

def config_processes(dbQueue,clientQueue,clientResponse,psetObjects):

    #db
    start_db_workers(dbQueue,psetObjects)

    #client
    start_client_workers(clientQueue,psetObjects,dbQueue,clientResponse)


def start_db_workers(dbQueue,psetObjects):
    print("Building db Workers")

    for p in range(DB_PROCS):
        dbworker = Process(target=db_worker,args=(connection,dbQueue,psetObjects,p))
        dbWorkers.append(dbworker)
        dbworker.start()

    print("db Pool of {} db Worker(s) Built -OK".format(DB_PROCS))


def start_client_workers(clientQueue,psetObjects,dbQueue,clientResponse):
    print("Building client Workers")
    print("CLIENT CORES: {}".format(CLIENT_CORES))
    
    for p in range(CLIENT_CORES):
        clientWorker = Process(target=client_worker,args=(clientQueue,psetObjects,dbQueue,clientResponse,p),name="client-worker-{}".format(p))
        clientWorkers.append(clientWorker)
        clientWorker.start()
    
    print("Client Pool of {} Worker(s) Built -OK".format(len(clientWorkers)))

    
def db_worker(connection,dbQueue,pset_objects,n):
    dbworker = DbWorker(connection,dbQueue,pset_objects,n)
    print("built dbworker {}".format(n))
    dbworker.listen()

def client_worker(clientQueue,psetObjects,dbQueue,clientResponse,p):
    clientworker = ClientWorker(clientQueue,psetObjects,dbQueue,clientResponse,p)
    print("built client worker {}".format(p))
    clientworker.listen()

在Docker中运行时，日志(log)显示:

INFO:     Started server process [7]
INFO:     Waiting for application startup.
CPUS: 8
Building db Workers
db Pool of 1 db Worker(s) Built -OK
Building client Workers
CLIENT CORES: 5
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)

也就是说，我们永远不会看到client_worker()和db_worker()目标永远不会被调用，start_client_workers()永远不会完成.直接使用uvicorn运行我得到:

CPUS: 8
INFO:     Started server process [40230]
INFO:     Waiting for application startup.
Building db Workers
db Pool of 1 db Worker(s) Built -OK
Building client Workers
CLIENT CORES: 5
Client Pool of 5 Worker(s) Built -OK
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
built client worker 4
built client worker 2
built client worker 1
built client worker 0
built client worker 3
built dbworker 0

我注意到后一个例子中服务器进程的数量非常荒谬，我想知道我是否在try 使用FastApi的多处理时做错了什么-尽管我理解，例如，从here开始，这是(或应该是)好的，并且在任何情况下，我的问题是Docker与外部行为之间的差异.

有没有办法用这种方法获得我想要的集装箱化行为，或者我应该寻求重构以使用Gunicorn、 docker 群或类似的方法？如果是后者，你能告诉我怎么做才好吗？谢谢.

-根据@datawookie的答复编辑2024年2月12日

我已经(准确地)在一个新的环境中实现了下面给出的答案，非常奇怪的是，我看到了以下内容:

INFO:     Started server process [1]
INFO:     Waiting for application startup.
Building db Workers
db Pool of 2 db Worker(s) Built -OK
Building client Workers
CLIENT CORES: 5
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
built client worker 0
built dbworker 1
built client worker 1
built dbworker 0
built client worker 3
built client worker 4
built client worker 2

值得注意的是，尽管我看到了从client_worker()开始的所有打印语句，但我没有看到对print(f"Client Pool of {len(clientWorkers)} Worker(s) Built -OK")的最终调用，这在@datawookie的S的回答中很明显.这与我在自己的应用程序环境中看到的行为相呼应--即最后一个Process.start()调用似乎没有完成--更奇怪的是，无论我为CLIENT_CORES Select 什么值，都会发生这种情况--对Process.start()的最后一个调用总是不会返回.

当我添加应用程序的完整逻辑(如最初的问题所示，构建一个ClientWorker对象并调用.listen()方法)时，事情变得更加奇怪.在本例中，输出为:

INFO:     Started server process [7]
INFO:     Waiting for application startup.
CPUS: 8
Building db Workers
db Pool of 1 db Worker(s) Built -OK
Building client Workers
CLIENT CORES: 5
built dbworker 0
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)

但如果我像这样注释掉.listen()电话:

def client_worker(clientQueue,psetObjects,dbQueue,clientResponse,p):
    clientworker = ClientWorker(clientQueue,psetObjects,dbQueue,clientResponse,p)
    print("built client worker {}".format(p))
    # clientworker.listen()

我明白了:

INFO:     Started server process [6]
INFO:     Waiting for application startup.
CPUS: 8
Building db Workers
db Pool of 1 db Worker(s) Built -OK
Building client Workers
CLIENT CORES: 5
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)
built client worker 0
built dbworker 0
built client worker 2
built client worker 1
built client worker 3
built client worker 4

为清楚起见，ClientWorker.listen()的形式如下:

def listen(self):
    print("started")
    while self.stay_alive == True:
        try:
            args = self.clientQueue.get(block=True)
            #do_work(args)
        except:
           #report_error()

再说一次，在uvicorn中一切都很好，请谁来解释一下 docker 环境中发生了什么？！