On Nov 1, 2012, at 4:22 PM, Carlos Borroto <carlos.borr...@gmail.com> wrote:

> Hi,
> I've been researching the possibility of using dynamic job runner in
> combination with job splitting for blast jobs. My main interest is to
> create a rule where both the size of the query and the database are
> taken into consideration in order to select DRMAA and splitting
> options.
> My first question is what is the status on dynamic job runner? I found
> these two threads, but is not clear to me if this feature is part of
> galaxy-dist already:
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-October/007160.html
> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010080.html

The dynamic runner is alive and well thanks to the continuous work of the 
author John Chilton and others. For instance, yesterday John committed code to 
galaxy-central that propagates tracebacks from the dynamic job runner's job 
rules script to the logs and also allows us to raise explicit exceptions from 
the job rules script thus making it possible to fail a job and print an 
arbitrary message in the job history's error message as well as making it much 
easier to debug the job rules.

> My second question is if there is any documentation other than the
> threads above to configure something like what I describe? In any
> case, there is very good information from John in these emails and I
> think that should get me started at least.

John's information is right on spot. I can also share my rules script with you. 
I've completely switched our production instance to the dynamic runner. It 
allows me to set the resource requests based on datasets (blast query at the 
moment, but I'm getting to the point of using the database as well), resources 
allocated to the user's group or a secondary group and so on as we well as 
check internal ACLs for tool usage restrictions and such and send jobs to 
specific queues or compute nodes as necessary.

I think your goals will inevitably bring you to the dynamic runner and it's not 
a bad thing. 

The only large obstacle in the job handling area as I perceive it is the lack 
of a generic way to plug some <input> statements into any tool's interface 
without changing all tool definition files, but that's beyond the dynamic 
runner and is something for the Galaxy Core Team to consider.


Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:


Reply via email to