When you launch a job with "srun", you have to include the option that specifies which MPI "glue" to use. You can find the available options with "--mpi=list" - you should see a "pmix_v3" option. Your cmd line should then be "srun --mpi=pmix_v3 ..."
You can also set that to be the default in your Slurm config file. > On Aug 5, 2020, at 8:40 AM, Luis Cebamanos via devel > <devel@lists.open-mpi.org> wrote: > > Hi, > > Just an update on this. I've managed to build OpenMPI 4.0.1 using > > --with-pmix=/lustre/z04/pmix --width-slurm > -with-libevent=/lustre/z04/pmix/libevent/ > > Unfortunately I keep getting the same warning when using srun (i.e. srun > my_ompi_app.x): > > The application appears to have been direct launched using "srun", > but OMPI was not built with SLURM's PMI support and therefore cannot > execute. There are several options for building PMI support under > SLURM, depending upon the SLURM version you are using: > > version 16.05 or later: you can use SLURM's PMIx support. This > requires that you configure and build SLURM --with-pmix. > > Versions earlier than 16.05: you must use either SLURM's PMI-1 or > PMI-2 support. SLURM builds PMI-1 by default, or you can manually > install PMI-2. You must then build Open MPI using --with-pmi pointing > to the SLURM PMI library location. > > Please configure as appropriate and try again. > > We use slurm 20.02.3 so configuring --with-pmix it is the correct way > according to this message. My worry is if we need to use the same PMIx > library that was used to build slurm, because that one has not been made > available. > > Regards, > Luis > > > > On 05/08/2020 16:10, Luis Cebamanos via devel wrote: >> Hi Ralph, >> >> Thanks for pointing me to this. I've done that and although configure >> does not report any errors, make won't build with following errors >> >> Making all in mca/pmix/ext3x >> make[2]: Entering directory '/lustre/z04/openmpi-4.0.1/opal/mca/pmix/ext3x' >> CC mca_pmix_ext3x_la-ext3x_local.lo >> CC mca_pmix_ext3x_la-ext3x.lo >> CC mca_pmix_ext3x_la-ext3x_client.lo >> CC mca_pmix_ext3x_la-ext3x_component.lo >> In file included from ext3x_local.c:21: >> ext3x.h:35:10: fatal error: pmix_server.h: No such file or directory >> #include "pmix_server.h" >> ^~~~~~~~~~~~~~~ >> >> My configure options are as follow >> >> ./configure --prefix=/lustre/z04/ompi_slurm >> --with-pmix=/lustre/z04/pmix --with-pmi=/lustre/z04/pmix --with-slurm >> --with-cuda=/lustre/sw/nvidia/hpcsdk/Linux_x86_64/cuda/10.1/include/ >> --with-libevent=/lustre/z04/pmix/libevent/ >> >> and I can see the that header file in the pmix built: >> $ls /lustre/z04/pmix/include/ >> pmi2.h pmix_common.h pmix.h pmix_server.h pmix_version.h >> pmi.h pmix_extend.h pmix_rename.h pmix_tool.h >> >> What am I missing? >> >> Cheers, >> Luis >> On 05/08/2020 15:21, Ralph Castain via devel wrote: >>> For OMPI, I would recommend installing PMIx: >>> https://github.com/openpmix/openpmix/releases/tag/v3.1.5 >>> >>> >>>> On Aug 5, 2020, at 12:40 AM, Luis Cebamanos via devel >>>> <devel@lists.open-mpi.org> wrote: >>>> >>>> Hi all, >>>> >>>> We are trying to install OpenMPI with Slurm support on a recently >>>> upgraded system. Unfortunately libpmi, libpmi2 or limpix don't seem to >>>> be available. Could I install these libraries myself to build OpenMPI >>>> and run it with Slurm? If so, which one would be needed? I guess these >>>> libraries should have been installed and made available whith the Slurm >>>> installation, but unfortunately they had not... >>>> >>>> checking for pmi.h... yes >>>> checking for libpmi in /usr/include/slurm/lib... checking for libpmi in >>>> /usr/include/slurm/lib64... not found >>>> checking for pmi2.h in /usr/include/slurm... found >>>> checking pmi2.h usability... yes >>>> checking pmi2.h presence... yes >>>> checking for pmi2.h... yes >>>> checking for libpmi2 in /usr/include/slurm/lib... checking for libpmi2 >>>> in /usr/include/slurm/lib64... not found >>>> checking for pmix.h in /usr/include/slurm... not found >>>> checking for pmix.h in /usr/include/slurm/include... not found >>>> checking can PMI support be built... no >>>> configure: WARNING: PMI support requested (via --with-pmi) but neither >>>> pmi.h, >>>> configure: WARNING: pmi2.h or pmix.h were found under locations: >>>> configure: WARNING: /usr/include/slurm >>>> configure: WARNING: /usr/include/slurm/slurm >>>> configure: WARNING: Specified path: /usr/include/slurm >>>> configure: WARNING: OR neither libpmi, libpmi2, or libpmix were found >>>> under: >>>> configure: WARNING: /lib >>>> configure: WARNING: /lib64 >>>> >>>> >>>> Regards, >>>> Luis >>>> >>>> The University of Edinburgh is a charitable body, registered in Scotland, >>>> with registration number SC005336. >>>> <pEpkey.asc> > > -- > ~~~~~~~~~~~~~~~~~~~~~~~~~ | EPCC | ~~~~~~~~~~~~~~~~~~~~~~~~~ > > Luis Cebamanos, HPC Applications Consultant > Email: l.cebama...@epcc.ed.ac.uk Phone: +44 (0) 131 651 3479 > > http://www.epcc.ed.ac.uk/ > The Bayes Centre, 47 Potterrow, Edinburgh UK > EH8 9BT > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > <pEpkey.asc>
