I solved this with the following (in lua syntax) for when I want to unload a
module in the "compiler" family that sits underneath the "compilermpi" family:
if (mode() == "unload") then
child=os.getenv("LMOD_FAMILY_COMPILERMPI") or ""
if ( isloaded(child) ) then
LmodMessage("Unloading child module ", child)
-- This is weird but because I'm in unload mode this has to be load (since
the command gets reversed)
load(child)
end
end
On 13 October 2015 at 11:50, Alan O'Cais
<[email protected]<mailto:[email protected]>> wrote:
Hi guys,
I'm looking for ideas in how to deal with the following scenario:
* I have two toolchain hierarchies that align with respect to
functionality: GNU -> gompi -> FOSS, iccifort -> iimpi -> intel
* I can give each level an lmod family so you can't load two from any level
concurrently: GNU/iccifort -> 'compiler', gompi/iimpi -> 'compilerMPI',
FOSS/intel -> 'FullToolchain'
* I have a toolchain-name based module hierarchy (i.e., 3 levels in the
hierarchy, one for each family and extended according to the toolchain name)
My question is how do I deal with the situation that a user has FOSS loaded and
then swaps gompi for iimpi? As far as I understand things that will mean FOSS
remains loaded but it's subtoolchain gets replaced. Can I trigger family-based
unloads in lmod? In qseudo-code, what I would like in the gompi module is:
if is-loaded family 'FullToolchain':
module unload family 'FullToolchain'
Is there another approach to this that I'm missing?
Alan
On 5 October 2015 at 10:53, Alan O'Cais
<[email protected]<mailto:[email protected]>> wrote:
Hi Kenneth, Bart,
Ok, getting more specific to our toolchain-based naming scheme, I was pretty
much planning to follow Barts lead (recursive unloading is really necessary for
toolchains but should be avoided anywhere else). My issue would be that it
looks like if I want to move away from the underlying dummy compiler I need to
build binutils with the binutils-bootstrapped GCC (because I can't manage to
build ld.gold with intel compilers) and have that available to all toolchains.
That essentially moves me to a `new` dummy that I control better. I like this
because I can stick tools in there that I want everywhere, binutils is the
first use case but maybe there will be more. For me, the reason I can't have
GNU as the underlying common denominator is because I want to build the same
MPI implementation with GNU and also iccifort leading to a potential problem if
someone happens to do a module reload on GNU (the path gets (re-)extended so
the GNU built MPI suddenly gets priority and replaces the iccifort version,
completely messing with any other loaded modules).
I agree that for a subtoolchain search I would probably go with the hierarchy
Kenneth described, but the way a toolchain based MNS works means the actual
extended paths are actually like I described.
I think I will hack the underlying intel/2015b to be composed of the same
underlying software but constructed to my specs, and include a recursive unload
for my toolchains like Bart. That way I should have compatability for all other
eb files but maintain the control I want.
Once we have the subtoolchain search working I can probably handle all this
directly in the naming scheme, this is a good opportunity to get the idea
straight.
Thanks,
Alan
On 2 October 2015 at 18:29, Bart Oldeman
<[email protected]<mailto:[email protected]>> wrote:
Hi Alan,
following your advice earlier I implemented such toolchains here (that
is for now opt-in for users, using a file named ~/.lmod in their
home), but since we have much more experience with OpenMPI than with
Intel MPI, instead of intel/2015b I used iomkl/2015b (+ foss/2015b),
where iomkl/2015b is defined as follows:
...
compver = '2015.3.187'
gccsuff = '-GNU-4.9.3-2.25'
dependencies = [
('icc', compver, gccsuff),
('ifort', compver, gccsuff),
('OpenMPI', '1.8.8', '', ('iccifort', '%s%s' % (compver, gccsuff))),
('iompi', version),
('imkl', '11.2.3.187', '', ('iompi', '%s' % (version))),
]
...
in combination with
module-naming-scheme=ToolchainMNS
recursive-module-unload = 1
in ~/.config/easybuild/config.cfg. It has thus similar dependencies as
intel/2015b.
With that it works fine, i.e. I can do
module load OpenFOAM
module switch iomkl foss
which reports:
Due to MODULEPATH changes the following have been reloaded:
1) OpenFOAM/2.3.1 2) OpenMPI/1.8.8 3) SCOTCH/6.0.4
what was very important though is to get the dependencies order in the
right place, ie. put iompi *after* its components.
Similarly I have iompi-2015b.eb:
version = '2015b'
iccifortsuff = '-GNU-4.9.3-2.25'
suff = '3.187'
compver = '2015.%s' % suff
dependencies = [
('icc', compver, iccifortsuff),
('ifort', compver, iccifortsuff),
('iccifort', compver, iccifortsuff),
('OpenMPI', '1.8.8', '', ('iccifort', '%s%s' % (compver, iccifortsuff))),
]
where iccifort needed to go after icc and ifort.
iimpi needed this change:
@@ -15,6 +15,7 @@
dependencies = [
('icc', compver, versionsuffix),
('ifort', compver, versionsuffix),
+ ('iccifort', compver, versionsuffix),
('impi', '5.0.3.048', '', ('iccifort', '%s%s' % (compver, versionsuffix))),
]
and intel this:
@@ -15,6 +15,7 @@
('icc', compver, gccsuff),
('ifort', compver, gccsuff),
('impi', '5.0.3.048', '', ('iccifort', '%s%s' % (compver, gccsuff))),
+ ('iimpi', '7.3.5', gccsuff),
('imkl', '11.2.3.187', '', ('iimpi', '7.3.5%s' % gccsuff)),
]
I did *not* let iccifort depend on GNU (i.e. took the unchanged
iccifort-2015.3.187-GNU-4.9.3-2.25.eb file), but just on icc and
ifort.
The (unchanged) icc-2015.3.187-GNU-4.9.3-2.25.eb and ifort then depend on GNU.
In practice I like having the GNU toolchain available as a
subtoolchain in all configurations, so that I only need for instance
one installation of git (git-2.5.3-GNU-4.9.3-2.25.eb).
Without the recursive module unload set indeed lmod was terribly
broken for the switch.
Regards,
Bart
On Fri, Oct 2, 2015 at 12:02 PM, Kenneth Hoste
<[email protected]<mailto:[email protected]>> wrote:
> Hi Alan,
>
> On 02/10/15 12:29, Alan O'Cais wrote:
>
> Hi all,
>
> I want to implement the new intel/2015b toolchains at our site because they
> fix problems (GMP I'm looking at you) and line up very well with the
> toolchains we'd already been using. I just want to make sure I understand
> the hierarchy of the toolchains and request a slight change so that things
> are more compatible with a hierarchical naming scheme.
>
> As I understand it, intel/2015b has the hierarchy:
> GCC -> GNU -> iccifort -> iimpi -> intel
> and the foss toolchain has the hierarchy:
> GCC -> GNU -> gompi -> foss
>
>
> Actually, the hierarchy is more like:
>
> iccifort -> iimpi -> intel
> GNU -> gompi -> foss
>
> which line up nicely.
>
> This also matches with the fact that they're both the last component in the
> toolchain hierarchy where 'dummy' is being used as a toolchain for
> installing.
>
> The fact that GNU is actually GCC+binutils, and that GNU is underneath
> iccifort, doesn't really matter for the hierarchy (correct me if I'm wrong).
>
> Basically, you should see 'iccifort' as the 'compiler' component in 'intel',
> and 'GNU' as the compiler component in 'foss'.
> The fact that they have dependencies underneath doesn't/shouldn't matter.
>
>
> This leads to (potential) problems in a HMNS because you may (like us) want
> to build an MPI implementation/version with iccifort for an intel-type
> toolchain and an identical MPI with GNU for a foss-type toolchain. In a HMNS
> you now have a weird situation where lmod will swap the MPI implementation
> when it unloads the iimpi-type module (since no recursive unload happens by
> default). That would trigger swapping a ton of other software in the users
> environment and probably a lot of confusion from the user-side. This is just
> an example, in a toolchain based naming scheme things are worse, if for some
> reason someone reloaded the GNU module then the MPI implementation gets
> swapped to the GNU version but no other modules change (eek!).
>
> Given that there is a sort-of equivalence between GNU and iccifort, wouldn't
> it be better to take the bintutils dep found in GNU and add it to the icc
> and ifort eb files directly instead. That would remove the need for GNU from
> the intel/2015b hierarchy but leave it essentially unchanged. The fork
> between the two would then occur at the level of GCC and I can safely build
> my MPI implementations with GNU and iccifort. Then I can then safely place
> GNU and iccifort in the same lmod family to avoid conflicts and then not
> worry about the order of module loads/unload/reload and what impact that
> will have on users.
>
> What do you think?
>
> Would the above fix the problems you're running into?
>
> I'm not sure what would need to be adjusted to make this work as expected.
>
> Probably avoiding the GCC extends $MODULEPATH? More?
>
>
> regards,
>
> Kenneth
--
Dr. Bart E. Oldeman | [email protected]<mailto:[email protected]> |
[email protected]<mailto:[email protected]>
Scientific Computing Analyst / Analyste en calcul scientifique
McGill HPC Centre / Centre de Calcul Haute Performance de McGill |
http://www.hpc.mcgill.ca
Calcul Québec | http://www.calculquebec.ca
Compute/Calcul Canada | http://www.computecanada.ca
Tel/Tél: 514-396-8926 | Fax/Télécopieur: 514-396-8934
--
Dr. Alan O'Cais
Application Support
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany
Phone: +49 2461 61 5213<tel:%2B49%202461%2061%205213>
Fax: +49 2461 61 6656<tel:%2B49%202461%2061%206656>
E-mail: [email protected]<mailto:[email protected]>
WWW: http://www.fz-juelich.de/ias/jsc/EN
--
Dr. Alan O'Cais
Application Support
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany
Phone: +49 2461 61 5213<tel:%2B49%202461%2061%205213>
Fax: +49 2461 61 6656<tel:%2B49%202461%2061%206656>
E-mail: [email protected]<mailto:[email protected]>
WWW: http://www.fz-juelich.de/ias/jsc/EN
--
Dr. Alan O'Cais
Application Support
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany
Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: [email protected]<mailto:[email protected]>
WWW: http://www.fz-juelich.de/ias/jsc/EN
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
