In the message dated: Mon, 27 Jan 2014 17:50:58 +0100,
The pithy ruminations from Reuti on
<Re: [gridengine users] Using modules form the compute nodes> were:
=> Hi,
=>
=> Am 27.01.2014 um 17:26 schrieb Txema Heredia:
=>
=> > I have been trying to use modulefiles from my compute nodes with no avail.
=> >
=> > When a job starts, the modulecmd command is in the path, but the module
function is nowhere to be found. I have tried to add calls to
/etc/profile.d/modules.sh in both /etc/bashrc and ~/.bashrc, and even setting
the queue shell to "/bin/bash -l" with no luck (it is not a valid shell). The
only mens to have access to the "module" function is either to add ".
/etc/profile.d/modules.sh"
That's what we've been doing -- having submit scripts source the modules
initialization file.
=>
=> This could also be done in a "starter_method" of the queue which calls the
real script:
=>
=> #!/bin/bash
=> . /etc/profile.d/modules.sh
=> exec "${@}"
=>
I like that idea...that can be very helpful.
However, I just tried it, and I'm running into something odd...
With a starter method of:
###########################
#!/bin/bash
# initialize modules, then run whatever was given
. /usr/share/Modules/init/bash
module help python
exec "${@}"
###########################
I get the output of the module file help text for our python module (as
expected), and then the user-supplied command is run.
The problem is that the module environment (function definitions) is
not passed to the user's command.
For example, if the SGE job is:
#############
#! /bin/bash
#$ -S /bin/bash
#$ -j y
#$ -o /home/$USER/output
echo "-- Module command before sourcing init file --"
module help gcc
. /usr/share/Modules/init/bash
echo "-- Module command after sourcing init file ---"
module help matlab
#############
when the SGE job runs:
1. the help text for the 'python' module is sent to stderr by
the starter method script, indicating that the module initialization
file is correctly sourced
2. the attempt to get help on the 'gcc' module produces the error:
module: command not found
indicating that the module function is not defined in the subshell
3. the matlab module help is sent to stderr, indicating that sourcing
the module initialization file succeeds
Note that the init file does contain "export -f" after the module() function
defintion.
There's been a lot of discussion on the environment-modules mailing list
about this kind of problem over the years, with a number of work-arounds a
(to my reading) no definitive general solution.
In this specific case, if I change the starter_method to:
##################
#! /bin/bash
# initialize modules
. /usr/share/Modules/init/bash
# check if "module" is declared as a function
declare -f -F module 1> /dev/null 2>&1
if [ $? = 0 ] ; then
# there is a module function, [re]export it
export -f module
fi
exec "${@}"
##################
it works just fine.
Mark
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
