我正在try 从Python运行MATLAB模型,以便使用OpenMDAO在遗传算法中优化模型设计参数. 我想将MATLAB模型运行的次数除以可用的PC内核数. 因此,给定90个MATLAB模型运行,每个核心将运行一个MATLAB引擎实例,占总模型运行的15个. 我希望所有的MATLAB实例都并行运行.

我的代码的问题是它逐个测试矩阵.Windows任务管理器显示一个MatLab引擎实例运行前15个模型,然后第二个MatLab引擎运行下一个15个模型,以此类推.因此,我的模型调用方法似乎没有节省时间.你如何才能迫使matlab引擎并行运行呢?我正在try 一些类似于顶部答案帖子here的东西,除了并行操作是在如下所示的函数中实现的.

# Open MATLAB engines
n_cores = 6 # Number of available PC cores
eng = []
for i in range(n_cores):
    eng.append(matlab.engine.start_matlab())
    eng[i].cd(r'C:\path_to_model', nargout=0)


# OpenMDAO genetic algorithm wraps around Python function that calls function that calls MATLAB engines
f = (omf.wrap(model_run_func))

#Set up design and objective variables for optimization within OpenMDAO
prob.model.add_design_var(des_vars)
prob.model.add_objective(obj_vars)

# Run the optimization 
prob.run_driver

# MATLAB model caller
def model_run_func(des_vars):

    '''
    Set up test cases and specify model inputs
    '''
    # Call function that divides model calls by number of PC cores
    output_list =  parallel(inputs_list)

    '''
    Unpack output_list and return model performance scores to OpenMDAO
    '''
# Divide-by-core function
def parallel(*args): 
    n_cases = len(args[0][0])
    n_cases_per_batch = math.ceil(n_cases/n_cores)
    test_index = np.array(range(n_cases))
    input_list = args[0]        
    for i_core in range(n_cores):
        test_index_this_core = 0
        inputs_list_this_core = [] 
        
        # This goes from 0:14, then 15:29, then 30 to...                
        test_index_this_batch = test_index_this_core + n_cases_per_batch * i_core
        
        # break if we exceed 15 cases for this core, or total number of cases
        if (test_index_this_core > n_cases_per_batch) or (test_index_this_batch > n_cases):
            break
        
        # inputs_list_this_core = inputs_list_this_core.append(input_list[test_index_this_batch])
        for inner_index in range(n_cases_per_batch):
            inner_test_index_this_batch = inner_index + test_index_this_batch
            if (inner_test_index_this_batch > (n_cases - 1)):
                break
            inputs_list_this_core_appendage = [x[inner_test_index_this_batch] for x in input_list[:]]
            inputs_list_this_core.append(inputs_list_this_core_appendage)
            
            # Add core index so you're not opening new MATLAB instances and crashing the program
            for index in range(len(inputs_list_this_core)):
                inputs_list_this_core[index].append(i_core)

        future = model(*inputs_list_this_core)
        
        test_index_this_core = test_index_this_core + 1
    for list in range(len(future_list)):
        for number in range(len(future_list[list])):
            if future_list[list][number].done():
                output_list.append(future_list[list][number].result())
            elif not future_list[list][number].done():
                while True:
                    if not future_list[list][number].done():
                        time.sleep(0.005)
                        continue
                    else:
                        output_list.append(future_list[list][number].result())
                        break
                
    return  output_list


def model(*args)

    output = []
    
    # Call # of parallel instances of the MATLAB engine to run the model based on # of available PC cores (with inputs specified above)
    number_cases_this_core = range(len(args))

    for i in number_cases_this_core:
        test_index = args[i][0] 
        '''
        Unpack model inputs to be given to MATLAB engine
        '''
        core_index = args[i][13]
        
        items = eng[int(core_index)].matlab_model(model_inputs,nargout=26,background=True)
        output.append(items)
            
    return output

推荐答案

以下是@MegaIng提供的解决方案的代码.

# Open MATLAB engines
n_cores = 6 # Number of available PC cores
eng = []
for i in range(n_cores):
    eng.append(matlab.engine.start_matlab())
    eng[i].cd(r'C:\path_to_model', nargout=0)


# OpenMDAO genetic algorithm wraps around Python function that calls function that calls MATLAB engines
f = (omf.wrap(model_run_func))

#Set up design and objective variables for optimization within OpenMDAO
prob.model.add_design_var(des_vars)
prob.model.add_objective(obj_vars)

# Run the optimization 
prob.run_driver

# MATLAB model caller
def model_run_func(des_vars):

    '''
    Set up test cases and specify model inputs
    '''
    # Call function that divides model calls by number of PC cores
    output_list =  parallel(inputs_list)

    '''
    Unpack output_list and return model performance scores to OpenMDAO
    '''
# Divide-by-core function
def parallel(*args): 
    n_cases = len(args[0][0])
    n_cases_per_batch = math.ceil(n_cases/n_cores)
    test_index = np.array(range(n_cases))
    input_list = args[0]        
    for i_core in range(n_cores):
        test_index_this_core = 0
        inputs_list_this_core = [] 
        
        # This goes from 0:14, then 15:29, then 30 to...                
        test_index_this_batch = test_index_this_core + n_cases_per_batch * i_core
        
        # break if we exceed 15 cases for this core, or total number of cases
        if (test_index_this_core > n_cases_per_batch) or (test_index_this_batch > n_cases):
            break
        
        # inputs_list_this_core = inputs_list_this_core.append(input_list[test_index_this_batch])
        for inner_index in range(n_cases_per_batch):
            inner_test_index_this_batch = inner_index + test_index_this_batch
            if (inner_test_index_this_batch > (n_cases - 1)):
                break
            inputs_list_this_core_appendage = [x[inner_test_index_this_batch] for x in input_list[:]]
            inputs_list_this_core.append(inputs_list_this_core_appendage)
            
            # Add core index so you're not opening new MATLAB instances and crashing the program
            for index in range(len(inputs_list_this_core)):
                inputs_list_this_core[index].append(i_core)

        future = model(*inputs_list_this_core)
        
        test_index_this_core = test_index_this_core + 1
    for list in range(len(future_list)):
        for number in range(len(future_list[list])):
            if future_list[list][number].done():
                output_list.append(future_list[list][number].result())
            elif not future_list[list][number].done():
                while True:
                    if not future_list[list][number].done():
                        time.sleep(0.005)
                        continue
                    else:
                        output_list.append(future_list[list][number].result())
                        break
                
    return  output_list


def model(*args)

    output = []
    
    # Call # of parallel instances of the MATLAB engine to run the model based on # of available PC cores (with inputs specified above)
    number_cases_this_core = range(len(args))

    for i in number_cases_this_core:
        test_index = args[i][0] 
        '''
        Unpack model inputs to be given to MATLAB engine
        '''
        core_index = args[i][13]
        
        items = eng[int(core_index)].matlab_model(model_inputs,nargout=26,background=True)
        output.append(items)
            
    return output

Python相关问答推荐

如何检测背景有噪的图像中的正方形

如何将双框框列中的成对变成两个新列

对于一个给定的数字,找出一个整数的最小和最大可能的和

输出中带有南的亚麻神经网络

将两只Pandas rame乘以指数

如何使用LangChain和AzureOpenAI在Python中解决AttribeHelp和BadPressMessage错误?

从groupby执行计算后创建新的子框架

Python+线程\TrocessPoolExecutor

driver. find_element无法通过class_name找到元素'""

处理具有多个独立头的CSV文件

pandas:对多级列框架的列进行排序/重新排序

在方法中设置属性值时,如何处理语句不可达[Unreacable]";的问题?

循环浏览每个客户记录,以获取他们来自的第一个/最后一个渠道

浏览超过10k页获取数据,解析:欧洲搜索服务:从欧盟站点收集机会的微小刮刀&

Seaborn散点图使用多个不同的标记而不是点

如何将一个文件的多列导入到Python中的同一数组中?

Pandas 删除只有一种类型的值的行,重复或不重复

Groupby并在组内比较单独行上的两个时间戳

删除另一个div中的特定div容器

基于2级列表的Pandas 切片3级多索引