Hi all,I'm trying to find a generic solution for making different MPI libraries available for our users "out of the box" (i.e., without them having to specify additional srun parameters). Our default MPI type is set to "openmpi" in slurm.conf, because our main MPI library is based on OpenMPI, but we also wish to provide the Intel MPI library.
From what I can tell, there are two ways to make Intel MPI work together with srun under this configuration: a) setting the MPI type to "pmi2" either via srun parameter --mpi, or via the environment variable SLURM_MPI_TYPE b) setting the environment variable I_MPI_PMI_LIBRARY to libpmi.so from SLURM
We were running with variant b) for some time, however, this had the downside that mpirun/mpiexec will not work anymore at all, which also breaks Co-Array Fortran compatibility (which uses mpiexec internally).
So variant a) looked like the obvious solution (setting it in the Intel MPI modulefile), BUT, I noticed that sbatch seems to unset this variable for the job environment. This can easily be reproduced by putting an "echo $SLURM_MPI_TYPE" into a batch file and submitting it having the variable set.
My questions: - is this intended behaviour?- if yes, how are we supposed to make Intel MPI work together with SLURM/srun without burdening our users with having to provide additional settings (either srun parameters or exports in their batch scripts) for their jobs?
Thank you kindly in advance. -- Maik Schmidt Technische Universität Dresden Zentrum für Informationsdienste und Hochleistungsrechnen (ZIH) Willers-Bau A116 D-01062 Dresden Telefon: +49 351 463-32836
smime.p7s
Description: S/MIME Cryptographic Signature
