on the "too many modules", i guess one of the best things we can do is learn lua and use lmod. and then see how we can tailor the output with easybuild environment. eg, after loading a toolchain, lmod should be able to pick this up from the environment and filter out the modules that will conflict on the toolchain restriction.

(for this to work however, all toolchian modules should declare a EBTOOLCHAIN environment variable or some other mechanism to detect the toolchain)



stijn


On 03/03/2013 10:12 AM, Kenneth Hoste wrote:
Hi Fotis,

On 03 Mar 2013, at 09:10, Fotis Georgatos wrote:

sw@h-cluster1-1:~$ module whatis ROOT
ROOT/v5.34.01-ictce-4.0.6(18):ERROR:102: Tcl command execution failed: if { 
![is-loaded ictce/4.0.6] } {
    module load ictce/4.0.6
}

ROOT                 : The ROOT system provides a set of OO frameworks with all 
the functionality
needed to handle and analyze large amounts of data in a very efficient way. - 
Homepage: http://root.cern.ch/drupal/

Same problem on our end, apparently (see below). "module show" and "module 
load" are working correctly though.
Maybe this is specific to the whatis feature of modules?


-bash-3.2$ module whatis ROOT/v5.34.01-ictce-4.0.6
ROOT/v5.34.01-ictce-4.0.6(18):ERROR:102: Tcl command execution failed: if { 
![is-loaded ictce/4.0.6] } {
     module load ictce/4.0.6
}

ROOT/v5.34.01-ictce-4.0.6: The ROOT system provides a set of OO frameworks with 
all the functionality
needed to handle and analyze large amounts of data in a very efficient way. - 
Homepage: http://root.cern.ch/drupal/

ERROR: module whatis ROOT/v5.34.01-ictce-4.0.6 failed



Does it relate to the 64bit build of env. modules? Are we alone in this?

No idea, but we also have a 64-bit build in any case:

-bash-3.2$ file `which modulecmd.exe`
/usr/bin/modulecmd.exe: ELF 64-bit LSB executable, AMD x86-64, version 1 
(SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), stripped


I suspect, but have no proof yet, that we hit a bug of this type:
http://sourceforge.net/mailarchive/message.php?msg_id=26547928
...since the error persists even if I feed modulefiles that are known to behave!


The problem seems to be specific to the 'whatis' functionality. Is it too on 
your end?


ps.
We released on Friday our most popular cluster back in production,
after a maintenance window, delivering >500 modules in production,
all thanks to v1.2.0; that was very good job everybody!!! Solid stuff.

Nice. Did you test the software behind all those modules too? ;-)



We already get complaints that we have too many modules, people cannot
find them etc; unfortunately, it's true :) so, what are your workarounds?

The prospect of arriving the watermark of 1000 modules is within reach now
(in fact, we could just arrive there by just tool-chaining latest gcc/icc!)
so this is becoming a very very serious challenge. Should we split the
modulespace by toolchain? does that make sense in your eyes?

We may need to provide a wrapper command over here (module-find or such),
otherwise it's tough to navigate the plethora of provided builds.

I wonder what you guys do, any ideas welcome!

tia,
Fotis

ps. LRZ has an interesting idea of "module -d" for default versions only!

This is indeed becoming a problem, one that only pops up because EasyBuild is 
making it so easy to produce more modules.

Currently, we basically have no way of handling this on our end. It grew bit by 
bit on our end, so we haven't had too many people complaining up until now.

There are a couple of things you could do:

  * group modules by field, to only show the ones groups of people are 
interested in (that's hard, imho)
  * set a decent default for all software packages, and only show those by 
default with 'module av'
  * set a default toolchain + version, and only show software built with that 
toolchain by default

For power users, you always want to allow them to see everything around.

That, combined with a more decent search functionality in modules (i.e., 
case-INsensitive), would already be a great help probably.



K.

Reply via email to