So would the current correct method for setting up multi-threaded jobs on a
cluster be to specify custom runners in the [galaxy:tool_runners] section of
the universe config file for EVERY tool that uses a multiple threads
(assuming the default is set to one)?

For example, for the bowtie program and a queue named "galaxy":
*bowtie = pbs:///galaxy/-l ppn=4,mem=16gb/*
Is this currently the only way for galaxy to inform the queuing system how
many threads a program will use?
And does this mean that without custom runners in the config file any
muti-threaded program that has multiple instances in an asychronous workflow
has the opportunity to overload a cluster node since the queuing system
doesn't "know" how many threads the program will be using?

Just want to make sure I'm not missing out on the latest and greatest method
for process management. :)

Louise-Amélie Schmitt wrote:

> >
> > default_cluster_job_runner will remain for backwards compatibility, but
> > we'll ship a sample job_conf.xml that runs everything locally by
> > default.
> >
> > --nate
> Haha, and I did that before realizing I could do just what I needed by
> writing tool-specific *pbs*:// URLs at the end of the config file... I'm
> an idiot.

Haha, okay, I don't think i even noticed since I was distracted by your
implementation being a step in the way we want to go with it.

> But I really like what you did of it and I have a couple of questions.
> Concerning the single-threaded tools, what would happen if the number of
> threads set in the xml file was >1 ?

It'd consume extra slots, but the tool itself would just run as usual.

> Could it be possible to forbid a tool to run on a given node?

Hrm.  In *PBS* you could do it using node properties/neednodes or resource
requirements.  I'd have to think a bit about how to do this in a more
general way in the XML.


> Thanks,
> L-A
> >
> >>
> >> Peter
> >>
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

Reply via email to