Hi Loris, If you want to enable PMIx in Slurm: * You have to build Slurm with an external PMIx library `--with-pmix`, then you get the pmix MPI plugin in Slurm. AFAIK Slurm does not bundle any PMIx, it only provides its own implementation of PMI2. * You can use OpenMPI with either its internal PMIx or with an external one. That does not change the configuration of Slurm. Just make sure that the major version of PMIx used by OpenMPI corresponds to the pmix plugin in Slurm.
The easyconfigs of OpenMPI v4 are already configured to use an external PMIx, so they should work with Slurm out of the box as long as you have that pmix plugin. The comment about "--with-slurm" is there only for backwards compatibility. I agree that it could be explained in more detail though. BTW this weekend I'll talk about all these configuration settings regarding PMIx-Slurm-OpenMPI at FOSDEM: https://fosdem.org/2022/schedule/event/exascale_pmi/ Kind regards, Alex Domingo HPC Team VUB - Informatie & Communicatietechnologie Pleinlaan 9 - 1050 Brussel https://hpc.vub.be ________________________________________ From: [email protected] <[email protected]> on behalf of Loris Bennett <[email protected]> Sent: Wednesday, 2 February 2022 15:21 To: [email protected] Subject: Re: [easybuild] OpenMPI: --with-slurm needed or not Hi Alex, Thanks for the clarification. Would it be worth expanding the comment in the OpenMPI EC to make this clear, at least for v4 and later? And what is the current thinking with regards to building PMIx support for Slurm? There seem to be 4 main options: - Slurm's internal PMIx - OpenMPI's internal PMIx - external PMIx from, say, operating system packages - external PMIx built by EasyBuild as a dependency of OpenMPI It seems to me that for simplicity's sake, the 4th option would be the best, since then Slurm and OpenMPI would be using the same versions I realise that this is not essential, but if anything goes wrong, then I only need to look at one version of PMIx and not two. WDYT? Cheers, Loris Alex DOMINGO TORO <[email protected]> writes: > Hi Loris, > > That depends on how Slurm is configured on your side. If you have Slurm > installed with the pmix plugin (`srun --mpi=list`), then "--with-slurm" and > "--with-pmi" are indeed not needed with modern OpenMPI. It will use PMIx to > communicate with Slurm. This is the default in Easybuild for OpenMPI v4. > > If your installation of Slurm only has the pmi2 plugin or you want to install > old versions of OpenMPI, then you need to enable "--with-slurm" and > "--with-pmi". > > Kind regards, > > Alex Domingo > > HPC Team > VUB - Informatie & Communicatietechnologie > Pleinlaan 9 - 1050 Brussel > https://hpc.vub.be > > ________________________________________ > From: [email protected] <[email protected]> on > behalf of Loris Bennett <[email protected]> > Sent: Wednesday, 2 February 2022 14:39 > To: easybuild > Subject: [easybuild] OpenMPI: --with-slurm needed or not > > Hi, > > While searching again for stuff to alleviate my recurring confusion > about OpenMPI, PMIx and Slurm, I can across the following posting from > myself: > > https://www.mail-archive.com/[email protected]/msg05468.html > > In it I asked with reference to building OpenMPI with --with-slurm: > > It seems to me that '--with-slurm' is not needed at all, since this has > been the default at least as far back as OpenMPI 1.0.1. So following > Åke's argument in the discussion in the link, wouldn't it be better to > change the comments in the OpenMPI ECs to just: > > # to enable PMI integration (site-specific) > # configopts = '--with-pmi' > > ? > > Back then no-one responded and the thread died. > > However, the following > > https://www.open-mpi.org/faq/?category=building#build-rte > > seems to support the idea that --with-slurm is not needed. > > So should the comment in the OpenMPI EC be changed? > > Cheers, > > Loris > > -- > Dr. Loris Bennett (Herr/Mr) > ZEDAT, Freie Universität Berlin Email [email protected] -- Dr. Loris Bennett (Herr/Mr) ZEDAT, Freie Universität Berlin Email [email protected]
