Re: [easybuild] Lmod integration - Families

[Kenneth Hoste]

On 24/03/2017 15:03, Vanzo, Davide wrote:

Yup, that is it!
Thanks Markus!

I think now I owe you guys a keg of beer...


Do we get to pick which beer? ;-)


K.



--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 24 2017, at 8:49 am, Markus Geimer  wrote:

Davide,

> However, it comes out is is an Lmod problem. I get the same
message when
> I try to load the hidden module intel/.2016b. But I cannot
figure out
> why. I will ask on the Lmod list.

Wait... It just crossed my mind that one has to add 'iccifort' as a
dependency to the toolchain. This is because the SwatHMNS disables
the $MODULEPATH extension for icc and ifort (which the "standard"
HMNS does).

Markus


> On Mar 24 2017, at 3:26 am, Markus Geimer
 wrote:
>
> Davide,
>
> > I am using your SwatHMNS module and everything is working fine
until I
> > try to build something that requires the imkl module. The error I
> see is
> > the following:
> >
> > Lmod has detected the following error: These module(s) exist but
> cannot be
> > loaded as requested: "imkl/11.3.3.210", "IntelMPI/5.1.3.181"
> >
> > It looks like EB is trying to load the IntelMPI and imkl in the
> opposite
> > order. If I manually try to load the (hidden) intel/2016b module
> > everything gets loaded correctly. Any idea?
>
> Which toolchain are you using? The order in which the
> dependencies are listed in the toolchain easyconfig should
> be the order in which modules are loaded. If not, we are
> in trouble (I think)...
>
> Markus
>
> --
> Dr. Markus Geimer
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
>
> Phone: +49-2461-61-1773
> Fax: +49-2461-61-6656
> E-mail: m.gei...@fz-juelich.de
> WWW: http://www.fz-juelich.de/jsc/
>
>


>


> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt
(Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>


>


>


-- 
Dr. Markus Geimer

Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW: http://www.fz-juelich.de/jsc/

Re: [easybuild] Lmod integration - Families

[Kenneth Hoste]

On 16/03/2017 15:24, Markus Geimer wrote:

Davide,


as a matter of fact I was thinking too about renaming iccifort to
something a little more familiar to the users. The problem is that if I
do so I need to modify all other easyconfig dependencies whenever they
depend on iccifort. Or do you have a better way?

You can do so via the 'modaltsoftname' easyconfig parameter and
a custom module naming scheme (needed since 'iccifort' is a
toolchain).  But then the easyconfigs can still refer to 'iccifort'.

@Kenneth: We really, really, really, need a way to install icc
and ifort as a bundle such that only a single compiler module
remains...  (Note: Not the full PSXE, just the compilers)


That should be possible with the changes in 
https://github.com/hpcugent/easybuild-framework/pull/2096.


It needs a little bit of work, and was mainly intended for the 
intel-psxe use case, but it's broader than that...



K.



So if I understand correctly your suggestion, I may want to consider
using the --recursive_module_unload flag only when building GCC and
iccifort, sonce those are my only two base compilers, correct?

Yes, you can put

 recursive_module_unload = True

in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'.  Then
the recursive unloading is done for the compilers only.

Markus



On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:

 Davide,

 yes, 'iccifort' needs to be visible. In fact, in my setup I am
 hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
 (well, it is actually also renamed to 'Intel', but that is a
 different story...). The 'recursive_module_unload' is not
 necessarily a global setting; you can also put it in an the
 corresponding easyconfig where you want this behavior, e.g.,
 in all compiler modules which are the lowest level of the
 hierarchy and the recursive unloading shouldn't do any harm.

 Markus




--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone:  +49-2461-61-1773
Fax:+49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW:http://www.fz-juelich.de/jsc/





Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

I am using intel-2016b.
However, it comes out is is an Lmod problem. I get the same message when I try 
to load the hidden module intel/.2016b. But I cannot figure out why. I will ask 
on the Lmod list.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 24 2017, at 3:26 am, Markus Geimer  wrote:

Davide,

> I am using your SwatHMNS module and everything is working fine until I
> try to build something that requires the imkl module. The error I see is
> the following:
>
> Lmod has detected the following error: These module(s) exist but cannot be
> loaded as requested: "imkl/11.3.3.210", "IntelMPI/5.1.3.181"
>
> It looks like EB is trying to load the IntelMPI and imkl in the opposite
> order. If I manually try to load the (hidden) intel/2016b module
> everything gets loaded correctly. Any idea?

Which toolchain are you using? The order in which the
dependencies are listed in the toolchain easyconfig should
be the order in which modules are loaded. If not, we are
in trouble (I think)...

Markus

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW: http://www.fz-juelich.de/jsc/



Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Maxime Boissonneault]

Oh, thanks for pointing this out.

Will correct.

Maxime

On 17-03-16 10:50, Markus Geimer wrote:

And the description if probably wrong, as 'iccifort' neither
includes Intel MPI nor MKL.  But this is inherited from upstream ;-)

Markus

On 03/16/17 15:41, Vanzo, Davide wrote:

Maxime,
in your case isn't the use of "intel" as alternative name for the
iccifort module creating conflicts with the intel toolchain module?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:35 am, Maxime Boissonneault
 wrote:

 If you want an example for this, you can have a look at our easyconfig
 for iccifort :
 
https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/i/iccifort/iccifort-2017.1.eb

 Maxime

 On 17-03-16 10:24, Markus Geimer wrote:
 > Davide,
 >
 >> as a matter of fact I was thinking too about renaming iccifort to
 >> something a little more familiar to the users. The problem is
 that if I
 >> do so I need to modify all other easyconfig dependencies whenever
 they
 >> depend on iccifort. Or do you have a better way?
 > You can do so via the 'modaltsoftname' easyconfig parameter and
 > a custom module naming scheme (needed since 'iccifort' is a
 > toolchain). But then the easyconfigs can still refer to 'iccifort'.
 >
 > @Kenneth: We really, really, really, need a way to install icc
 > and ifort as a bundle such that only a single compiler module
 > remains... (Note: Not the full PSXE, just the compilers)
 >
 >> So if I understand correctly your suggestion, I may want to consider
 >> using the --recursive_module_unload flag only when building GCC and
 >> iccifort, sonce those are my only two base compilers, correct?
 > Yes, you can put
 >
 > recursive_module_unload = True
 >
 > in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'. Then
 > the recursive unloading is done for the compilers only.
 >
 > Markus
 >
 >
 >> On Mar 16 2017, at 9:04 am, Markus Geimer
  wrote:
 >>
 >> Davide,
 >>
 >> yes, 'iccifort' needs to be visible. In fact, in my setup I am
 >> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
 >> (well, it is actually also renamed to 'Intel', but that is a
 >> different story...). The 'recursive_module_unload' is not
 >> necessarily a global setting; you can also put it in an the
 >> corresponding easyconfig where you want this behavior, e.g.,
 >> in all compiler modules which are the lowest level of the
 >> hierarchy and the recursive unloading shouldn't do any harm.
 >>

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone:  +49-2461-61-1773
Fax:+49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW:http://www.fz-juelich.de/jsc/





Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt






--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] Lmod integration - Families

[Maxime Boissonneault]

Hi Davide,
We don't have the intel toolchain module.

Maxime

On 17-03-16 10:41, Vanzo, Davide wrote:

Maxime,
in your case isn't the use of "intel" as alternative name for the 
iccifort module creating conflicts with the intel toolchain module?


--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:35 am, Maxime Boissonneault 
 wrote:


If you want an example for this, you can have a look at our
easyconfig
for iccifort :

https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/i/iccifort/iccifort-2017.1.eb

Maxime

On 17-03-16 10:24, Markus Geimer wrote:
> Davide,
>
>> as a matter of fact I was thinking too about renaming iccifort to
>> something a little more familiar to the users. The problem is
that if I
>> do so I need to modify all other easyconfig dependencies
whenever they
>> depend on iccifort. Or do you have a better way?
> You can do so via the 'modaltsoftname' easyconfig parameter and
> a custom module naming scheme (needed since 'iccifort' is a
> toolchain). But then the easyconfigs can still refer to 'iccifort'.
>
> @Kenneth: We really, really, really, need a way to install icc
> and ifort as a bundle such that only a single compiler module
> remains... (Note: Not the full PSXE, just the compilers)
>
>> So if I understand correctly your suggestion, I may want to
consider
>> using the --recursive_module_unload flag only when building GCC and
>> iccifort, sonce those are my only two base compilers, correct?
> Yes, you can put
>
> recursive_module_unload = True
>
> in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'. Then
> the recursive unloading is done for the compilers only.
>
> Markus
>
>
>> On Mar 16 2017, at 9:04 am, Markus Geimer
 wrote:
>>
>> Davide,
>>
>> yes, 'iccifort' needs to be visible. In fact, in my setup I am
>> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
>> (well, it is actually also renamed to 'Intel', but that is a
>> different story...). The 'recursive_module_unload' is not
>> necessarily a global setting; you can also put it in an the
>> corresponding easyconfig where you want this behavior, e.g.,
>> in all compiler modules which are the lowest level of the
>> hierarchy and the recursive unloading shouldn't do any harm.
>>
>> Markus
>>
>>
>
> --
> Dr. Markus Geimer
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
>
> Phone: +49-2461-61-1773
> Fax: +49-2461-61-6656
> E-mail: m.gei...@fz-juelich.de
> WWW: http://www.fz-juelich.de/jsc/
>
>
>
>


>


> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt
(Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>


>


>


-- 
-

Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de
Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique




--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

Maxime,
in your case isn't the use of "intel" as alternative name for the iccifort 
module creating conflicts with the intel toolchain module?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:35 am, Maxime Boissonneault 
 wrote:

If you want an example for this, you can have a look at our easyconfig
for iccifort :
https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/i/iccifort/iccifort-2017.1.eb

Maxime

On 17-03-16 10:24, Markus Geimer wrote:
> Davide,
>
>> as a matter of fact I was thinking too about renaming iccifort to
>> something a little more familiar to the users. The problem is that if I
>> do so I need to modify all other easyconfig dependencies whenever they
>> depend on iccifort. Or do you have a better way?
> You can do so via the 'modaltsoftname' easyconfig parameter and
> a custom module naming scheme (needed since 'iccifort' is a
> toolchain). But then the easyconfigs can still refer to 'iccifort'.
>
> @Kenneth: We really, really, really, need a way to install icc
> and ifort as a bundle such that only a single compiler module
> remains... (Note: Not the full PSXE, just the compilers)
>
>> So if I understand correctly your suggestion, I may want to consider
>> using the --recursive_module_unload flag only when building GCC and
>> iccifort, sonce those are my only two base compilers, correct?
> Yes, you can put
>
> recursive_module_unload = True
>
> in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'. Then
> the recursive unloading is done for the compilers only.
>
> Markus
>
>
>> On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:
>>
>> Davide,
>>
>> yes, 'iccifort' needs to be visible. In fact, in my setup I am
>> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
>> (well, it is actually also renamed to 'Intel', but that is a
>> different story...). The 'recursive_module_unload' is not
>> necessarily a global setting; you can also put it in an the
>> corresponding easyconfig where you want this behavior, e.g.,
>> in all compiler modules which are the lowest level of the
>> hierarchy and the recursive unloading shouldn't do any harm.
>>
>> Markus
>>
>>
>
> --
> Dr. Markus Geimer
> Juelich Supercomputing Centre
> Institute for Advanced Simulation
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
>
> Phone: +49-2461-61-1773
> Fax: +49-2461-61-6656
> E-mail: m.gei...@fz-juelich.de
> WWW: http://www.fz-juelich.de/jsc/
>
>
>
> 
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> 
> 
>

--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] Lmod integration - Families

[Markus Geimer]

Davide,

> as a matter of fact I was thinking too about renaming iccifort to
> something a little more familiar to the users. The problem is that if I
> do so I need to modify all other easyconfig dependencies whenever they
> depend on iccifort. Or do you have a better way?

You can do so via the 'modaltsoftname' easyconfig parameter and
a custom module naming scheme (needed since 'iccifort' is a
toolchain).  But then the easyconfigs can still refer to 'iccifort'.

@Kenneth: We really, really, really, need a way to install icc
and ifort as a bundle such that only a single compiler module
remains...  (Note: Not the full PSXE, just the compilers)

> So if I understand correctly your suggestion, I may want to consider
> using the --recursive_module_unload flag only when building GCC and
> iccifort, sonce those are my only two base compilers, correct?

Yes, you can put

recursive_module_unload = True

in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'.  Then
the recursive unloading is done for the compilers only.

Markus


> On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:
>
> Davide,
>
> yes, 'iccifort' needs to be visible. In fact, in my setup I am
> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
> (well, it is actually also renamed to 'Intel', but that is a
> different story...). The 'recursive_module_unload' is not
> necessarily a global setting; you can also put it in an the
> corresponding easyconfig where you want this behavior, e.g.,
> in all compiler modules which are the lowest level of the
> hierarchy and the recursive unloading shouldn't do any harm.
>
> Markus
>
>


--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone:  +49-2461-61-1773
Fax:+49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW:http://www.fz-juelich.de/jsc/





Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Maxime Boissonneault]

If you want an example for this, you can have a look at our easyconfig 
for iccifort :

https://github.com/ComputeCanada/easybuild-easyconfigs/blob/computecanada-master/easybuild/easyconfigs/i/iccifort/iccifort-2017.1.eb

Maxime

On 17-03-16 10:24, Markus Geimer wrote:

Davide,


as a matter of fact I was thinking too about renaming iccifort to
something a little more familiar to the users. The problem is that if I
do so I need to modify all other easyconfig dependencies whenever they
depend on iccifort. Or do you have a better way?

You can do so via the 'modaltsoftname' easyconfig parameter and
a custom module naming scheme (needed since 'iccifort' is a
toolchain).  But then the easyconfigs can still refer to 'iccifort'.

@Kenneth: We really, really, really, need a way to install icc
and ifort as a bundle such that only a single compiler module
remains...  (Note: Not the full PSXE, just the compilers)


So if I understand correctly your suggestion, I may want to consider
using the --recursive_module_unload flag only when building GCC and
iccifort, sonce those are my only two base compilers, correct?

Yes, you can put

 recursive_module_unload = True

in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'.  Then
the recursive unloading is done for the compilers only.

Markus



On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:

 Davide,

 yes, 'iccifort' needs to be visible. In fact, in my setup I am
 hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
 (well, it is actually also renamed to 'Intel', but that is a
 different story...). The 'recursive_module_unload' is not
 necessarily a global setting; you can also put it in an the
 corresponding easyconfig where you want this behavior, e.g.,
 in all compiler modules which are the lowest level of the
 hierarchy and the recursive unloading shouldn't do any harm.

 Markus




--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone:  +49-2461-61-1773
Fax:+49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW:http://www.fz-juelich.de/jsc/





Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt






--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

Thank you guys! That is really great.

+1 on the idea of having a single module for icc and ifort.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:25 am, Markus Geimer  wrote:

Davide,

> as a matter of fact I was thinking too about renaming iccifort to
> something a little more familiar to the users. The problem is that if I
> do so I need to modify all other easyconfig dependencies whenever they
> depend on iccifort. Or do you have a better way?

You can do so via the 'modaltsoftname' easyconfig parameter and
a custom module naming scheme (needed since 'iccifort' is a
toolchain). But then the easyconfigs can still refer to 'iccifort'.

@Kenneth: We really, really, really, need a way to install icc
and ifort as a bundle such that only a single compiler module
remains... (Note: Not the full PSXE, just the compilers)

> So if I understand correctly your suggestion, I may want to consider
> using the --recursive_module_unload flag only when building GCC and
> iccifort, sonce those are my only two base compilers, correct?

Yes, you can put

recursive_module_unload = True

in the easyconfigs of 'icc', 'ifort', 'iccifort', and 'GCC'. Then
the recursive unloading is done for the compilers only.

Markus

> On Mar 16 2017, at 9:04 am, Markus Geimer  wrote:
>
> Davide,
>
> yes, 'iccifort' needs to be visible. In fact, in my setup I am
> hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
> (well, it is actually also renamed to 'Intel', but that is a
> different story...). The 'recursive_module_unload' is not
> necessarily a global setting; you can also put it in an the
> corresponding easyconfig where you want this behavior, e.g.,
> in all compiler modules which are the lowest level of the
> hierarchy and the recursive unloading shouldn't do any harm.
>
> Markus
>
>

--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-mail: m.gei...@fz-juelich.de
WWW: http://www.fz-juelich.de/jsc/



Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Alan O'Cais]

You can actually set the recursive_unload directly in the easyconfig.

Here's a naming scheme that auto-handles the problem you foresee (Markus' based 
on something I did)

On 16 March 2017 at 15:12, Vanzo, Davide 
> wrote:
Markus,

as a matter of fact I was thinking too about renaming iccifort to something a 
little more familiar to the users. The problem is that if I do so I need to 
modify all other easyconfig dependencies whenever they depend on iccifort. Or 
do you have a better way?

So if I understand correctly your suggestion, I may want to consider using the 
--recursive_module_unload flag only when building GCC and iccifort, sonce those 
are my only two base compilers, correct?

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
(615)-875-9137
www.accre.vanderbilt.edu

On Mar 16 2017, at 9:04 am, Markus Geimer 
> wrote:

Davide,

yes, 'iccifort' needs to be visible. In fact, in my setup I am
hiding 'icc' and 'ifort' and only have 'iccifort' non-hidden
(well, it is actually also renamed to 'Intel', but that is a
different story...). The 'recursive_module_unload' is not
necessarily a global setting; you can also put it in an the
corresponding easyconfig where you want this behavior, e.g.,
in all compiler modules which are the lowest level of the
hierarchy and the recursive unloading shouldn't do any harm.

Markus




--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt





swat_hmns.py
Description: Binary data

Re: [easybuild] Lmod integration - Families

[Markus Geimer]

Davide,

> Do you add the family attribute manually after EB installed
> the default module for a specific package?

We currently add

modluafooter = 'family("compiler")'

or similar to the easyconfig file.

> For this purpose it would be nice to have a function in EB that would
> allow from the easyconfig file

There has been some work on this, see
https://github.com/hpcugent/easybuild-framework/pull/1257
However, the PR hasn't been touched for a while and will need some
care.  Maybe someone at the User Group meeting is keep on adopting
it ;-)

> (or even better in a easyconfig patch
> files)  to add additional lines to the generated module for a more easy
> to manage localization.

+1 for easyconfig patch files

Markus


> On Feb 6 2017, at 12:05 pm, Alvarez, Damian <d.alva...@fz-juelich.de>
> wrote:
> 
> I’ll cover it a bit, but not too deeply. In any case, I don’t think
> Davide will come.
> 
>  
> 
> Anyway, what are the issues you are seeing? I think that using a
> toolchain based module naming scheme this would work similarly to
> the hierarchical scheme (ie: it will prevent you from loading a
> compiler on top of another one)
> 
>  
> 
> Damian
> 
>  
> 
> *From: *<easybuild-requ...@lists.ugent.be> on behalf of Alan O'Cais
> <a.oc...@fz-juelich.de>
> *Reply-To: *"easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
> *Date: *Monday 6 February 2017 at 18:22
> *To: *"easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
> *Subject: *Re: [easybuild] Lmod integration - Families
> 
>  
> 
> We use families for  compilers and MPI implementations in a
> hierarchical scheme. We do not expose the toolchains to normal
> users. Damian will probably cover this a little in his presentation
> on Wednesday.
> 
>  
> 
> Alan
> 
>  
> 
> On 6 February 2017 at 16:57, Vanzo, Davide
> <davide.va...@vanderbilt.edu <mailto:davide.va...@vanderbilt.edu>>
> wrote:
> 
> Hello all,
> 
> one of the most useful features provided by Lmod is the ability
> to assign a family to a subset of modules in order to avoid
> loading potentially conflicting software. This gets a little
> complicated when dealing with the toolchain-based model of EB. I
> was wondering if anybody is using families and how.
> 
> Thank you.
> 
> 
> -- 
> Davide Vanzo, PhD
> Application Developer
> 
> Adjunct Assistant Professor of Chemical and Biomolecular Engineering
> 
> Advanced Computing Center for Research and Education (ACCRE)
> 
> Vanderbilt University - Hill Center 201
> www.accre.vanderbilt.edu <http://www.accre.vanderbilt.edu>
> 
> 
> 
>  
> 
> -- 
> 
> Dr. Alan O'Cais
> E-CAM Software Manager
> Juelich Supercomputing Centre
> Forschungszentrum Juelich GmbH
> 52425 Juelich, Germany
> 
> Phone: +49 2461 61 5213
> Fax: +49 2461 61 6656
> E-mail: a.oc...@fz-juelich.de <mailto:a.oc...@fz-juelich.de>
> WWW:http://www.fz-juelich.de/ias/jsc/EN
> 
> 
> 
> 
> 
> 
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> 
> 
> 
> 
> 
> 

-- 
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone:  +49-2461-61-1773
Fax:+49-2461-61-6656
E-Mail: m.gei...@fz-juelich.de
Web:http://www.fz-juelich.de/jsc

Re: [easybuild] Lmod integration - Families

[Vanzo, Davide]

Damian and Alex,
I am still on a pre-production stage and I am trying to find the best way to 
implement it in production such that the users experience will be the best 
possible. So I was just wandering what other groups do in order to avoid 
reinventing the wheel.
Your idea of not exposing the toolchain to users is already a good feedback. Do 
you add the family attribute manually after EB installed the default module for 
a specific package?
For this purpose it would be nice to have a function in EB that would allow 
from the easyconfig file (or even better in a easyconfig patch files)  to add 
additional lines to the generated module for a more easy to manage localization.

PS: You are right, I will not be able to join you guys at the user meeting. I 
will surely watch any recordings or slides you will make available.

Davide


On Feb 6 2017, at 12:05 pm, Alvarez, Damian <d.alva...@fz-juelich.de> wrote:

I’ll cover it a bit, but not too deeply. In any case, I don’t think Davide will 
come.



Anyway, what are the issues you are seeing? I think that using a toolchain 
based module naming scheme this would work similarly to the hierarchical scheme 
(ie: it will prevent you from loading a compiler on top of another one)



Damian



From: <easybuild-requ...@lists.ugent.be> on behalf of Alan O'Cais 
<a.oc...@fz-juelich.de>
Reply-To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Date: Monday 6 February 2017 at 18:22
To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Subject: Re: [easybuild] Lmod integration - Families



We use families for  compilers and MPI implementations in a hierarchical 
scheme. We do not expose the toolchains to normal users. Damian will probably 
cover this a little in his presentation on Wednesday.



Alan



On 6 February 2017 at 16:57, Vanzo, Davide 
<davide.va...@vanderbilt.edu<mailto:davide.va...@vanderbilt.edu>> wrote:

Hello all,

one of the most useful features provided by Lmod is the ability to assign a 
family to a subset of modules in order to avoid loading potentially conflicting 
software. This gets a little complicated when dealing with the toolchain-based 
model of EB. I was wondering if anybody is using families and how.

Thank you.

--
Davide Vanzo, PhD
Application Developer

Adjunct Assistant Professor of Chemical and Biomolecular Engineering

Advanced Computing Center for Research and Education (ACCRE)

Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>





--

Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de>
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Alvarez, Damian]

I’ll cover it a bit, but not too deeply. In any case, I don’t think Davide will 
come.

Anyway, what are the issues you are seeing? I think that using a toolchain 
based module naming scheme this would work similarly to the hierarchical scheme 
(ie: it will prevent you from loading a compiler on top of another one)

Damian

From: <easybuild-requ...@lists.ugent.be> on behalf of Alan O'Cais 
<a.oc...@fz-juelich.de>
Reply-To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Date: Monday 6 February 2017 at 18:22
To: "easybuild@lists.ugent.be" <easybuild@lists.ugent.be>
Subject: Re: [easybuild] Lmod integration - Families

We use families for  compilers and MPI implementations in a hierarchical 
scheme. We do not expose the toolchains to normal users. Damian will probably 
cover this a little in his presentation on Wednesday.

Alan

On 6 February 2017 at 16:57, Vanzo, Davide 
<davide.va...@vanderbilt.edu<mailto:davide.va...@vanderbilt.edu>> wrote:
Hello all,
one of the most useful features provided by Lmod is the ability to assign a 
family to a subset of modules in order to avoid loading potentially conflicting 
software. This gets a little complicated when dealing with the toolchain-based 
model of EB. I was wondering if anybody is using families and how.
Thank you.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu<http://www.accre.vanderbilt.edu>



--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de<mailto:a.oc...@fz-juelich.de>
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Lmod integration - Families

[Alan O'Cais]

We use families for  compilers and MPI implementations in a hierarchical 
scheme. We do not expose the toolchains to normal users. Damian will probably 
cover this a little in his presentation on Wednesday.

Alan

On 6 February 2017 at 16:57, Vanzo, Davide 
> wrote:
Hello all,
one of the most useful features provided by Lmod is the ability to assign a 
family to a subset of modules in order to avoid loading potentially conflicting 
software. This gets a little complicated when dealing with the toolchain-based 
model of EB. I was wondering if anybody is using families and how.
Thank you.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu



--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany

Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: a.oc...@fz-juelich.de
WWW:http://www.fz-juelich.de/ias/jsc/EN




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] Lmod integration - Families

[Vanzo, Davide]

Hello all,
one of the most useful features provided by Lmod is the ability to assign a 
family to a subset of modules in order to avoid loading potentially conflicting 
software. This gets a little complicated when dealing with the toolchain-based 
model of EB. I was wondering if anybody is using families and how.
Thank you.

--
Davide Vanzo, PhD
Application Developer
Adjunct Assistant Professor of Chemical and Biomolecular Engineering
Advanced Computing Center for Research and Education (ACCRE)
Vanderbilt University - Hill Center 201
www.accre.vanderbilt.edu