Hello all,
I'm wondering if it is sensible to make Galaxy tools automatically use
the environment variable $NSLOTS to automatically adjust their
number of threads?
Using $NSLOTS works on SGE, but is it generally used on other clusters?
The idea here is rather than hard coding the number of threads in a tool
or its XML file, which may need to be altered for different local setups, and
it can be specified in universe_wsgi.ini under [galaxy:tool_runners]
e.g. By default our SGE allocates one slot, so with the following BWA
should use one thread:
[galaxy:tool_runners]
bwa_wrapper = drmaa://-V/
However, if we ask SGE for 8 slots, the tool should use eight threads:
[galaxy:tool_runners]
bwa_wrapper = drmaa://-V -pe smp 8/
For this to be truly general, we would need a way to set environement
variables for local:// runners.
However, we can cope with $NSLOTS being undefined with a little
magic in the XML definitions. .e.g For the BWA wrapper this is
currently hard coded to use 4 threads:
<command interpreter="python">
bwa_wrapper.py
--threads="4"
...
</command>
Instead, this could be something like this:
<command interpreter="python">
bwa_wrapper.py
#if "$NSLOTS"=="":
--threads="4"
#else:
--threads="$NSLOTS"
#end if
...
</command>
Likewise for the BLAST+ wrappers etc. i.e. If the environment variable
is set (e.g. via the cluster settings) use that, otherwise keep the current
hard coded default.
Would this work in principle on other cluster setups? i.e. Is $NSLOTS
sufficiently general? It would be messy but the XML if statement could
be expanded to handle a second environment variable as well if needs
be.
Would the Galaxy team accept a pull request implementing this for the
BWA and BLAST+ wrappers?
Thanks,
Peter
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
