Hi Åke, Thanks for your comments.
On Fri, 6 Dec 2019 at 15:09, Åke Sandgren <[email protected]> wrote: > > On 12/6/19 3:59 PM, Will Furnass wrote: > > > > We're using EasyBuild 4.0.0 on a Centos 7 + Slurm 19.05 system. We're > > able to build and use 'foss-2019a' EasyBuild toolchain (which includes > > OpenMPI 3.1.3) using the libpmi2 provided by the Slurm folks (by > > uncommenting "configopts = '--with-slurm --with-pmi" in > > OpenMPI-3.1.3-GCC-8.2.0-2.31.1.eb). However, when trying to build R > > 3.6.0 and Rmpi with that toolchain we get: > > 1 - Did you build the pmi and/or pmi2 parts of Slurm? At least pmi is > now in contrib and not built by default. We build Slurm with just 'rpmbuild -ta', which results in headers for libpmi and libpmi2 being installed into /usr/include/slurm and the corresponding shared libs into /usr/lib64. > 2 - --with-pmi as config opt to openmpi means pmi1 not pmi2 if i don't > remember incorrectly. OpenMPI's ./configure help suggests --with-pmi can be used for either: ./configure --help | grep with-pmi --with-pmix(=DIR) Build PMIx support. DIR can take one of three --with-pmix-libdir=DIR Look for libpmix the given directory DIR, DIR/lib or --with-pmi(=DIR) Build PMI support, optionally adding DIR to the --with-pmi-libdir=DIR Look for libpmi or libpmi2 in the given directory > 3 - I'd strongly suggest using PMIx instead. What I've read about PMIx suggests it's only real advantage is faster startup times for large jobs, which isn't particularly important for this particular system (<100 nodes; 25GbE). When we set up the cluster a few months ago we went with pmi2 as we could build it directly from the Slurm source. > 4 - You can take a look at > https://github.com/easybuilders/easybuild-framework/pull/2777 to see how > we use hooks to add support for PMIx on the fly when building OpenMPI > through easybuild. So you install PMIx using EasyBuild? I think I'm right in saying that Slurm also depends on PMIx (and PMIx itself depends on hwloc). Out of curiosity, how do you manage Slurm depending on PMIx given you have multiple PMIx builds but only one Slurm install? Regards, Will -- Dr Will Furnass | Research Software Engineering team Computer Science | University of Sheffield +44 (0) 114 22 21903 | https://rse.shef.ac.uk @willfurnass | https://learningpatterns.me --- Book a code clinic session: https://rse.shef.ac.uk/support/code-clinic/
