I completely agree with Loris! I have been wanting to hide irrelevant
modules from the listing of "module avail" for a very long time (the
curious users can always use "module spider"). The "module list" should
list all loaded modules, hidden or not.
At the recent Supercomputing 2018 conference I asked Davide Vanzo
(EasyBuild) and Robert McLay (Lmod) exactly this question, and Davide's
answer is that you must rebuild all of the module .lua files with
--module-only and --hide-deps. You *do not* have to rebuild the
software itself!
It's also possible to hide modules by configuring Lmod, see the
isVisibleHook function in
https://lmod.readthedocs.io/en/latest/170_hooks.html
I think Davide's solution of rebuilding the module files is the simplest
way forward. However, it isn't trivial to do correctly on a running
system, because you will be messing with the locations of modules in the
.lua files which could break user jobs. It is strongly recommended to
make a copy of the module tree and work on this without impacting the
production system. When completed, you can point users to the location
of the new module tree (on a shared file system, I presume).
This leads us to an interesting project for interested EasyBuild sites.
Can we design some tools to perform:
1. Create a configuration file with "system" module names (not end-user
applications) that we want to hide. My favorites include all module
names beginning with:
Autoconf, Automake, Autotools, Bison, CMake, LibTIFF, LibUUID, M4, Szip,
Tcl, Tk, Tkinter, XML-Parser, XZ, bzip2, binutils, cURL, expat, flex,
fontconfig, freetype, gettext, help2man, hwloc, lib*, ncurses, numactl,
pkg-config, tmux, util-linux, zlib
2. Write a script that traverses all directories in
$EASYBUILD_PREFIX/modules/all and rebuilds all the .lua files using "eb
--module-only --hide-deps". Maybe the above "system" module names must
be rebuilt with --hidden, whereas end-user applications must be rebuilt
with a --hide-deps list equal to the desired hidden modules. I'm unsure
of the correct logic here, but maybe EasyBuild experts can comment on this?
The "eb --help" explains how to hide modules:
--hidden Install 'hidden' module file(s) by prefixing their
version with '.' (def False)
--hide-deps=HIDE-DEPS Comma separated list of dependencies that you
want automatically hidden, (e.g. --hide-deps=zlib,ncurses)
(type comma-separated list)
Does anyone have the time and courage to work on a setup with such a
functionality?
Best regards,
Ole
On 11/23/2018 10:17 AM, Loris Bennett wrote:
Hi Sam,
Thanks for the info about rebuilding the module files.
Regarding the visibility of modules, I just want to exclude some from
the results of 'module av'. I assume 'module list' will still show all
the modules which are loaded, hidden or not. That doesn't bother me,
since, as far as I know, very few of our users look at the output of
'module list' anyway.
Cheers,
Loris
Sam Moors <[email protected]> writes:
Hi Loris,
You can instruct EB to only rebuild the modules with the '--module-only' option.
I don't know about your second question though.
That said, are you sure you want to hide the dependencies?
I think it is useful for the users to know which deps are loaded, so
they can avoid conflicts with other modules, even if it may look
daunting to see 50+ modules loaded in some cases.
Cheers,
Sam
On Fri, Nov 23, 2018 at 8:25 AM Loris Bennett <[email protected]>
wrote:
Hi,
Now that I have installed a couple of packages with easybuild I have a
surprising number of modules and realise that I should have been using
--hide-deps. I have two questions about this:
1. Is there any way to retrospectively hide modules, other than tweaking
the module files by hand, say, rebuilding the modules?
2. Can a set of modules be defined which will be hidden by default?
Cheers,
Loris
--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin Email [email protected]
--
Ole Holm Nielsen
PhD, Senior HPC Officer
Department of Physics, Technical University of Denmark,
Building 307, DK-2800 Kongens Lyngby, Denmark
E-mail: [email protected]
Homepage: http://dcwww.fysik.dtu.dk/~ohnielse/
Tel: (+45) 4525 3187 / Mobile (+45) 5180 1620
