Hi Barry, I used the branch barry/2020-11-10/add-matlab-engine-host. Now it appears that the matlab engine starts can start on any node but it is not able to execute any matlab script. If I revert back to PETSc 3.13.5 the head compute node is able to execute the MATLAB script. See the following error (load_mpc_parameters.m is a MATLAB script that the code is trying to execute):
Starting Matlab engine... Matlab engine started Storing variables in Matlab workspace... Undefined function or variable 'load_mpc_parameters'. Error using save Variable 'a' not found. [0]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- [0]PETSC ERROR: Error in external library [0]PETSC ERROR: Unable to get array a from matlab [0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Development GIT revision: unknown GIT Date: unknown [0]PETSC ERROR: ./main3d on a linux-opt named node35 by kkhedkar Thu Nov 12 02:39:47 2020 [0]PETSC ERROR: Configure options --CC=/home/kkhedkar/sfw/mpich3.3_mellanox/3.3/bin/mpicc --CXX=/home/kkhedkar/sfw/mpich3.3_mellanox/3.3/bin/mpicxx --FC=/home/kkhedkar/sfw/mpich3.3_mellanox/3.3/bin/mpif90 --COPTFLAGS=-O3 --CXXOPTFLAGS=-O3 --FOPTFLAGS=-O3 --PETSC_ARCH=linux-opt --with-debugging=0 --download-hypre=1 --with-x=0 --download-fblaslapack=1 --with-matlab-dir=/nas/app/MATLAB/matlab2019a/ --with-matlab-engine=1 --with-matlab-engine-dir=/nas/app/MATLAB/matlab2019a/extern/engines/ [0]PETSC ERROR: #1 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c Error using save Variable 'h' not found. [0]PETSC ERROR: #2 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c Error using save Variable 'mpc_start_time' not found. [0]PETSC ERROR: #3 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c Error using save Variable 'sample_size' not found. sample_size = 0 [0]PETSC ERROR: #4 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c Error using save Variable 'dt_sampling' not found. Storing up past excitation force samples every 0 time-steps. [0]PETSC ERROR: #5 PetscMatlabEngineGetArray() line 407 in /home/kkhedkar/sfw/petsc_matlab_host_fix/petsc/src/sys/classes/matlabengine/matlab.c IBRedundantInitializer: Deallocating initialization data. My code is using Petsc matlab engine in this way (“master” is the hostname of the head node where MATLAB is licensed): if(SAMRAI_MPI::getRank() == 0) { std::cout << "\nStarting Matlab engine...\n" << std::endl; PetscMatlabEngineCreate(PETSC_COMM_SELF, "master", &(mpc->mengine)); std::cout << "Matlab engine started" << std::endl; PetscMatlabEngineEvaluate(mpc->mengine,"clc; clear all; close all; addpath([cd,'/MPC_matlab_code'])"); std::cout << "Storing variables in Matlab workspace...\n" << std::endl; PetscMatlabEngineEvaluate(mpc->mengine,"global hydro wave; wave.H_wave = %f ; wave.Tp = %f; m = %f;", input_db->getDouble("HEIGHT"), input_db->getDouble("TIME_PERIOD"), input_db->getDouble("MASS")); PetscMatlabEngineEvaluate(mpc->mengine,"load_mpc_parameters; calculate_mpc_matrices"); PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &(mpc->m_plus_Ainf), "a"); PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &(mpc->dt_controller), "h"); PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &(mpc->mpc_start_time), "mpc_start_time"); PetscScalar sample_size; PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &sample_size, "sample_size"); mpc->sample_size = int(sample_size); std::cout << "\nsample_size = " << mpc->sample_size << std::endl; mpc->F_past.resize(mpc->sample_size); mpc->t_past.resize(mpc->sample_size); double dt = input_db->getDouble("DT_MAX"); mpc->initial_position = input_db->getDouble("ZCOM"); mpc->current_position = mpc->initial_position; PetscScalar dt_sampling; PetscMatlabEngineGetArray(mpc->mengine, 1, 1, &dt_sampling, "dt_sampling"); mpc->sampling_interval = int(dt_sampling / dt); std::cout << "Storing up past excitation force samples every " << mpc->sampling_interval << " time-steps.\n" << std::endl; } From: Barry Smith <[email protected]> Date: Tuesday, November 10, 2020 at 5:22 PM To: Kaustubh Khedkar <[email protected]> Cc: "[email protected]" <[email protected]> Subject: Re: [petsc-users] Using Petsc-Matlab engine on a cluster In the git branch barry/2020-11-10/add-matlab-engine-host I have added the option -matlab_engine_host hostname Note this does require you have ssh access to hostname from where your PETSc program is running which may require setting up some ssh files Good luck Barry On Nov 9, 2020, at 5:35 PM, Kaustubh Khedkar <[email protected]<mailto:[email protected]>> wrote: Hello, I am using PETSc with Matlab engine. I have built an application that uses Matlab engine through PETSc interface. I am trying to run this application on a cluster which has Matlab installed on a network drive (nas drive). I have made sure that I am able to access Matlab using my university details on the cluster (by activating Matlab license through Mathworks.com<http://mathworks.com/> account). I am also able to run this application using mpirun on the head node (master) which starts the Matlab engine but if I try to run this application using slurm on compute node then the Matlab engine does not start. I have also confirmed that if I login into one of the compute nodes and do a mpirun the Matlab engine does not start there. If I try to launch Matlab on a compute node it asks for Matlab license which the compute node is unable to activate possibly because it is not connected to the internet directly. Is there a way start matlab engine on the head node through Petsc interface even though the application is running on compute nodes or is there any other work around this issue? Thanks, Kaustubh Khedkar
