Åke Sandgren <[email protected]> writes: > On 2/13/20 4:32 PM, Loris Bennett wrote: >> Hi Åke, >> >> I see you that in your parse_hook you have >> >> if ec.name == 'OpenMPI': >> ... >> if LooseVersion(ec.version) >= LooseVersion('2.1'): >> pmix_version = '1.2.5' >> ucx_version = '1.4.0' >> if LooseVersion(ec.version) >= LooseVersion('3'): >> pmix_version = '2.2.1' >> if LooseVersion(ec.version) >= LooseVersion('4'): >> pmix_version = '3.0.2' # OpenMPI 4.0.0 is not compatible >> with PMIx 3.1.x >> >> extra_deps.append(('PMIx', pmix_version)) >> >> So do you build Slurm against the corresponding PMIx module? > > No, we build pmix/ucx/libevent for slurm outside of EB. But using the > corresponding versions. > >> I'd sort of felt that, as we don't build Slurm as a module, it should >> just depend on the system libraries, but if there are dependencies >> between PMIx and OpenMPI to be taken into account, that simplistic >> approach isn't going to work. >> >> However, if I tie Slurm to a particular PMIx version, won't I then have >> problems when I start running programs built with a different toolchain >> with a different version of OpenMPI with a dependency on a potentially >> different version of PMIx? > > That why we build slurm with > --with-ucx=/lap/ucx/1.6.1 > --with-pmix=/lap/pmix/3.1.4:/lap/pmix/2.2.3:/lap/pmix/1.2.5 > so it can talk to all pmix versions. Which one gets used is set in the > module for each openmpi version depending on which pmix version it can use > >> Cheers, >> >> Loris >> >> PS: Slightly strangely, EB has a patch for PMIx 2.2.1 but no actual EC: >> >> [build@admin ~]$ eb -S pmix-2 >> >> CFGS1=/trinity/shared/easybuild/software/EasyBuild/4.1.1/easybuild/easyconfigs/p/PMIx >> * $CFGS1/PMIx-2.1.3-GCCcore-7.3.0.eb >> * $CFGS1/PMIx-2.2.1_fix_seg_hdr_problem.patch > > > Doh... did I forget something again... I'll check that out as soon as i > can...
OK, thanks for that. Now I think I get it. Just to be absolutely clear and apologies in advance for nitpicking, but the PMIx v2 you use is 2.2.1, as also referenced in your parse hook. However, in your Slurm config you have --with-pmix=/lap/pmix/3.1.4:/lap/pmix/2.2.3:/lap/pmix/1.2.5 Shouldn't that also be 2.2.1? Cheers, Loris -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email [email protected]
