Re: [easybuild] Strategy for hiding modules

[Ward Poelmans]

If you're still interested, the isVisible() hook has been merged. An
example on how to use it can be found at
https://github.com/TACC/Lmod/pull/276/files

You can dynamically hide a module based on any criteria of your desire,
time, toolchain version, ...

Ward

On 07-04-17 14:32, Alan O'Cais wrote:
> Funnily enough, there's a PR for Lmod that suits exactly your use case:
> https://github.com/TACC/Lmod/pull/248
> 
> 
> On 7 Apr 2017 3:05 pm, "Andreas Hilboll"  > wrote:
> 
> Hi all,
> 
> when I started using EasyBuild, I didn't know about the option to hide
> modules.  Slowly, the list of installed modules becomes too long for
> `module avail` to be useful for the user.  So I would like to hide
> modules.  The question is, how should I do this?
> 
> 1. Re-generate all modules (can I use EasyBuild to regenerate modules,
>hiding them if I want to, without recompiling the code?)
> 
> 2. Manually hide modules (probably a bad idea)
> 
> 3. ???
> 
> It would be nice to have a best-practices document somewhere, but I
> couldn't find any. [*]
> 
> Also, from what I understand, using minimal toolchains is generally a
> good idea to keep the number of modules low -- correct?
> 
> I'd appreciate any guidance on how I can clean up my installation.
> 
> Cheers,
>   Andreas.
> 
> PS: I'm using lmod
> 
> [*] I'd be happy to write a blog post from a non-expert's perspective
> (I'm no sysadmin, and our cluster is rather smallish, almost no
> technical staff to maintain the software).
> 
> 
> 
> 
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> 
> 
> 

Re: [easybuild] Strategy for hiding modules

[Maxime Boissonneault]

Note that with Lmod, you can specify a hidden module in the global 
modulerc file.




On 17-04-07 08:32, Alan O'Cais wrote:

Funnily enough, there's a PR for Lmod that suits exactly your use case:
https://github.com/TACC/Lmod/pull/248 



On 7 Apr 2017 3:05 pm, "Andreas Hilboll" > wrote:


Hi all,

when I started using EasyBuild, I didn't know about the option to hide
modules.  Slowly, the list of installed modules becomes too long for
`module avail` to be useful for the user.  So I would like to hide
modules.  The question is, how should I do this?

1. Re-generate all modules (can I use EasyBuild to regenerate modules,
   hiding them if I want to, without recompiling the code?)

2. Manually hide modules (probably a bad idea)

3. ???

It would be nice to have a best-practices document somewhere, but I
couldn't find any. [*]

Also, from what I understand, using minimal toolchains is generally a
good idea to keep the number of modules low -- correct?

I'd appreciate any guidance on how I can clean up my installation.

Cheers,
  Andreas.

PS: I'm using lmod

[*] I'd be happy to write a blog post from a non-expert's perspective
(I'm no sysadmin, and our cluster is rather smallish, almost no
technical staff to maintain the software).





Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt






--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Président - Comité de coordination du soutien à la recherche de Calcul Québec
Team lead - Research Support National Team, Compute Canada
Instructeur Software Carpentry
Ph. D. en physique

Re: [easybuild] Strategy for hiding modules

[Alvarez, Damian]

You can EasyBuild to regenerate the modules with --module-only. Be careful, as 
in some corner cases the resulting module might be difficult when simply 
regenerating the module and when installing software for real.

You can also hide the modules using lmod’s 7 hidden modules feature. In your 
modulerc file you can have stuff like hide_version("mpich/3.2-dbg"). That way 
you won’t see the hidden modules with module avail, but all your modules will 
still work. I am not sure how scalable that is, since lmod will check for every 
module if it should be shown or not, and for that it will go through the whole 
list.

You can also hide then manually, but that’s more work and you need to make sure 
your modules are consistent (load (“X/1.0”) won’t work if the module is 
“X/.1.0”).

The way we do that is having a list of hidden dependencies in 
EASYBUILD_HIDE_DEPS, and be careful to don’t install dependencies directly 
without --hidden. I’ve sent to the mailing list our list of hidden dependencies 
this week, in case you want to use it as a basis.

Regarding --minimal-toolchains, why do you think it is a good idea to keep the 
number of modules low? I mean, it looks to me like a good idea, but 
independently of --minimal-toolchains.

Damian

On 07/04/17 14:05, "easybuild-requ...@lists.ugent.be on behalf of Andreas 
Hilboll"  
wrote:

Hi all,

when I started using EasyBuild, I didn't know about the option to hide
modules.  Slowly, the list of installed modules becomes too long for
`module avail` to be useful for the user.  So I would like to hide
modules.  The question is, how should I do this?

1. Re-generate all modules (can I use EasyBuild to regenerate modules,
   hiding them if I want to, without recompiling the code?)

2. Manually hide modules (probably a bad idea)

3. ???

It would be nice to have a best-practices document somewhere, but I
couldn't find any. [*]

Also, from what I understand, using minimal toolchains is generally a
good idea to keep the number of modules low -- correct?

I'd appreciate any guidance on how I can clean up my installation.

Cheers,
  Andreas.

PS: I'm using lmod

[*] I'd be happy to write a blog post from a non-expert's perspective
(I'm no sysadmin, and our cluster is rather smallish, almost no
technical staff to maintain the software).






Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

Re: [easybuild] Strategy for hiding modules

[Alan O'Cais]

Funnily enough, there's a PR for Lmod that suits exactly your use case:
https://github.com/TACC/Lmod/pull/248

On 7 Apr 2017 3:05 pm, "Andreas Hilboll" 
mailto:hilb...@uni-bremen.de>> wrote:
Hi all,

when I started using EasyBuild, I didn't know about the option to hide
modules.  Slowly, the list of installed modules becomes too long for
`module avail` to be useful for the user.  So I would like to hide
modules.  The question is, how should I do this?

1. Re-generate all modules (can I use EasyBuild to regenerate modules,
   hiding them if I want to, without recompiling the code?)

2. Manually hide modules (probably a bad idea)

3. ???

It would be nice to have a best-practices document somewhere, but I
couldn't find any. [*]

Also, from what I understand, using minimal toolchains is generally a
good idea to keep the number of modules low -- correct?

I'd appreciate any guidance on how I can clean up my installation.

Cheers,
  Andreas.

PS: I'm using lmod

[*] I'd be happy to write a blog post from a non-expert's perspective
(I'm no sysadmin, and our cluster is rather smallish, almost no
technical staff to maintain the software).




Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt

[easybuild] Strategy for hiding modules

[Andreas Hilboll]

Hi all,

when I started using EasyBuild, I didn't know about the option to hide
modules.  Slowly, the list of installed modules becomes too long for
`module avail` to be useful for the user.  So I would like to hide
modules.  The question is, how should I do this?

1. Re-generate all modules (can I use EasyBuild to regenerate modules,
   hiding them if I want to, without recompiling the code?)

2. Manually hide modules (probably a bad idea)

3. ???

It would be nice to have a best-practices document somewhere, but I
couldn't find any. [*]

Also, from what I understand, using minimal toolchains is generally a
good idea to keep the number of modules low -- correct?

I'd appreciate any guidance on how I can clean up my installation.

Cheers,
  Andreas.

PS: I'm using lmod

[*] I'd be happy to write a blog post from a non-expert's perspective
(I'm no sysadmin, and our cluster is rather smallish, almost no
technical staff to maintain the software).