Hi Åke,
I see you that in your parse_hook you have
if ec.name == 'OpenMPI':
...
if LooseVersion(ec.version) >= LooseVersion('2.1'):
pmix_version = '1.2.5'
ucx_version = '1.4.0'
if LooseVersion(ec.version) >= LooseVersion('3'):
pmix_version = '2.2.1'
if LooseVersion(ec.version) >= LooseVersion('4'):
pmix_version = '3.0.2' # OpenMPI 4.0.0 is not compatible with
PMIx 3.1.x
extra_deps.append(('PMIx', pmix_version))
So do you build Slurm against the corresponding PMIx module?
I'd sort of felt that, as we don't build Slurm as a module, it should
just depend on the system libraries, but if there are dependencies
between PMIx and OpenMPI to be taken into account, that simplistic
approach isn't going to work.
However, if I tie Slurm to a particular PMIx version, won't I then have
problems when I start running programs built with a different toolchain
with a different version of OpenMPI with a dependency on a potentially
different version of PMIx?
Cheers,
Loris
PS: Slightly strangely, EB has a patch for PMIx 2.2.1 but no actual EC:
[build@admin ~]$ eb -S pmix-2
CFGS1=/trinity/shared/easybuild/software/EasyBuild/4.1.1/easybuild/easyconfigs/p/PMIx
* $CFGS1/PMIx-2.1.3-GCCcore-7.3.0.eb
* $CFGS1/PMIx-2.2.1_fix_seg_hdr_problem.patch
Loris Bennett <[email protected]> writes:
> Hi Åke,
>
> Thanks for the link - very useful. I see there are various pitfalls
> regarding the compatibility of various OpenMPI and PMIx versions that I
> was only partially aware of.
>
> Cheers,
>
> Loris
>
> Åke Sandgren <[email protected]> writes:
>
>> Yeah, the comment in that easyconfig is due for a rewrite.
>>
>> You could also look at
>> https://github.com/easybuilders/easybuild-framework/pull/2777 for more
>> ways to handle this.
>>
>> On 2/13/20 1:37 PM, Loris Bennett wrote:
>>> Hi,
>>>
>>> In
>>>
>>> OpenMPI-4.0.2-GCC-9.2.0-2.32.eb
>>>
>>> it says
>>>
>>> # to enable SLURM integration (site-specific)
>>> # configopts += '--with-slurm --with-pmi=/usr/include/slurm
>>> --with-pmi-libdir=/usr'
>>>
>>> Regarding the Slurm option, here
>>>
>>> https://www.open-mpi.org/faq/?category=building#build-rte
>>>
>>> it says
>>>
>>> Slurm support is built automatically; there is nothing that you need
>>> to do.
>>>
>>> Admittedly here
>>>
>>> https://www.open-mpi.org/faq/?category=slurm
>>>
>>> it also says
>>>
>>> 1. How do I run jobs under Slurm?
>>> The short answer is yes, provided you configured OMPI --with-slurm.
>>>
>>> Apart from the fact that 'yes' is not a good answer to the question, I
>>> feel that the information regarding '--with-slurm' may be outdated.
>>>
>>> Regarding PMI, for OpenMPI 4.0.2
>>>
>>> ./config --help
>>>
>>> says
>>>
>>> --with-pmix(=DIR) Build PMIx support. DIR can take one of three
>>> values: "internal", "external", or a valid
>>> directory
>>> name. "internal" (or no DIR value) forces Open MPI
>>> to use its internal copy of PMIx. "external"
>>> forces
>>> Open MPI to use an external installation of PMIx.
>>> Supplying a valid directory name also forces Open
>>> MPI to use an external installation of PMIx, and
>>> adds DIR/include, DIR/lib, and DIR/lib64 to the
>>> search path for headers and libraries. Note that
>>> Open MPI does not support --without-pmix.
>>>
>>> In the past the problem was that both PMIX and Slurm packages tried to
>>> install PMI1 and PMI2 libraries into the same location. Slurm now
>>> creates a separate package with these libraries and it is possible to
>>> build PMIX in such a way that libpmi.so and libpmi2.so end up in a
>>> separate package.
>>>
>>> The separate pmix-libpmi package installs the headers pmi.h and pmi2.h
>>> to
>>>
>>> /usr/include
>>>
>>> whereas slurm-devel installs them to
>>>
>>> /usr/include/slurm
>>>
>>> Therefore I think the advice in the EC should be more like
>>>
>>> # If the PMI{1,2} libraries from Slurm, rather than those from PMIx
>>> # have been installed (site-specific), use the the following
>>> # configopts += '--with-pmi=/usr/include/slurm'
>>>
>>> Is this correct?
>>>
>>> Cheers,
>>>
>>> Loris
>>>
>>> --
>>> Dr. Loris Bennett (Mr.)
>>> ZEDAT, Freie Universität Berlin Email [email protected]
>>>
--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email [email protected]
