On 03/05/2020 00:17, Mike Kelsey wrote:
kelsey writes:
Thanks to both Kenneth and Ake for their suggestions! I'll investigate
both, but I think I have a couple of questions first.
More follow up below to both suggestions, and my idea for a Third Way (with
apologies to Tony Blair :-).
Ake_Sandgren wrote:
it might be possible to specify it using a dependency like this,
('CDMS', EXTERNAL_MODULE),
in the dependeies list, or if you need a specific version,
('CDMS/xx.yy', EXTERNAL_MODULE)
So this is what I had done, and when a user tries to load the enclosing
Bundle, they get the error:
[kelsey@terra2 eb]$ module load scdms-release/V02-04-01-foss-2018b
Lmod has detected the following error: These module(s) or
extension(s) exist but cannot be loaded as requested: "cdms/V02-04-01"
Try: "module spider cdms/V02-04-01" to see how to load the module(s).
(yes, the case is correct; we're trying to use lowercase for module names,
rather than all caps).
See
https://easybuild.readthedocs.io/en/latest/Using_external_modules.html?highlight=external%20module
It didn't work. I used --external-modules-metadata, and created a .cfg file:
[cdms/V02-04-01]
name = ReleaseBuilder
version = V02-04-01
prefix = /scratch/group/mitchcomp/CDMS/releases/modulefiles
Then I rebuilt my scdms-release/V02-04-01-foss-2018b-Python-3.6.6 Bundle.
The resulting .lua file was identical to the previous version (none of the
information from the above got included in it), and when I tried loading
that "scdms-release" module with a plain MODULEPATH, it failed with the same
Lmod error.
It looks like with this method, the dependent modules still need to be
findable at load time, but the .cfg information somehow causes the load to
be different.
The provided metadata will not change the generated module the way you
think/expect it would.
The metadata is used by EasyBuild to get more information about
dependencies that are marked as external modules, so it gets a better
idea of what they provide. The "prefix" is to specify in which directory
the software that the module provides is located (not where the module
itself is located).
The external modules indeed have to be loadable with the current
$MODULEPATH, EasyBuild will make no attempt to make them loadable by itself.
That may be a good idea to support though in the metadata, so EasyBuild
can do a 'module use' before loading that external module, but it
supported isn't currently.
[...]
Kenneth Hoste writes:
[...]
You can certainly do it via a post_module_hook (see
https://easybuild.readthedocs.io/en/latest/Hooks.html), where you can
interfere with the generated module file.
Is there an example of a "module hook" anywhere in the repository? GitHub's
search was entirely unhelpful?
I found easyconfig.py appearing to do something with parse_hook, but I found
no acutal "def ..._hook" functions anywhere in ither Framework or in
EasyBlocks. I did find some hook-testing stuff in toy_build.py, but nothing
relevant to post_module_hook.
One thing I don't quite see about this is that the post_module_hook wouldn't
get run until after the resulting .lua file (or Tcl file) had already been
written out. Essentially, I'd be writing Python-based "awk" code to insert
extra lines, and that code would rely on knowing the module's language.
Yes, indeed, you'd have to pick up the module file that was produced
already, and modify it...
I wonder if it might be "better" to write an EasyBlock for this situation,
and overload the make_module_dep() function, such that it prepends to
MODULEPATH before generating the load() statements (as is done in
make_module_extnd_modpath(), but after the dependencies).
What I don't know is whether changing the envvar is actually seen by and
affects Lmod during the load, or whether Lmod caches the intial value of
MODULEPATH for its own use.
Any changes to the environment are picked up subsequently in the same
installation, so that would work, but you shouldn't alter $MODULEPATH
directly, you should use "self.modules_tool.use" in an easyblock instead
(which will change $MODULEPATH for you).
regards,
Kenneth
-- Mike
