On Tue, Nov 27, 2012 at 8:58 AM, Andreas Kuntzagk
<andreas.kuntz...@mdc-berlin.de> wrote:
> Hi,
> the four processes I saw where all called "lastz" and ran in parallel and
> consumed 100% of a core each.
> My guess is that the lastz_wrapper.py is responsible for this.
> Looking at it I see a some code regarding queuing and in the very beginning
> this line:
> and further one the class BaseQueue which starts "threads". BTW. there seems
> to be no way to adjust this number other than editing the source file - bad.
> And this get's me wondering if there are other such surprises hidden in
> galaxy.
> regards, Andreas

As the author of several tool wrappers, I've been asking for a Galaxy
wide mechanism for Galaxy to tell the tool how many threads it can
use, for example via an environment variable. The value could then
be set with a general default, per runner default, or even per tool
using the existing runner configuration under [galaxy:tool_runners]
in universe_wsgi.ini

See: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-March/009037.html
and: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html

In your example, and others like the BWA and BLAST+ wrappers where
the tool XML is hard coded to 8 threads, you would probably want to
use a custom runner in universe_wsgi.ini setting the cluster submission
to request that many slots/CPUs.

For our local cluster, I modify the BLAST+ wrapper XML to use 4 threads,
and have something like this in my universe_wsgi.ini file:

ncbi_blastp_wrapper  = drmaa://-V -pe smp 4/
ncbi_blastn_wrapper  = drmaa://-V -pe smp 4/
ncbi_blastx_wrapper  = drmaa://-V -pe smp 4/
ncbi_tblastn_wrapper = drmaa://-V -pe smp 4/
ncbi_tblastx_wrapper = drmaa://-V -pe smp 4/

Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:


Reply via email to