Could you give me a pointer where in the codebase I should start looking to implement this, please?
I'd be happy to share this with the community if it is useful for others....
Thon
On Jun 01, 2012, at 10:53 AM, Jeremy Goecks <jeremy.goe...@emory.edu> wrote:
No, this is not configurable via XML; you'd have to extend the Galaxy codebase to implement this behavior.Best,J.On Jun 1, 2012, at 1:50 PM, Anthonius deBoer wrote:Hi Jeremy,Thanks for the info...But it's not clear from your message if I could use this Key/Value approach at the moment to distinguish if a job should be run in the fast or in the slow queue.I would like to add a parameter to every tool that would have it determine if it should be in the fast queue or in the slow queue...It would be checked for interactive jobs and if someone created a workflow with this tool, they could turn it off and it would run in the slow/high memory queue....Could I add this today and what would the XML look like or are you saying it only works for the trackster example you gave...ThanksThon>> Is there a way for a tool to sometimes be placed in the fast queue and
>> sometimes in the long queue?
>
> Not through Galaxy as far as I know.
Yes, this is possible using job parameterization. From universe.ini.sample:
--
# Per-tool job handler and runner overrides. Parameters can be included to define multiple
# runners per tool. E.g. to run Cufflinks jobs initiated from Trackster
# differently than standard Cufflinks jobs:
#
# cufflinks = local:///
# cufflinks[source@trackster] = local:///
--
This approach is definitely a beta feature, but the idea is that any set of key@value parameters should be able to be used to direct jobs to different queues as needed.
Job parameterization is done in only one place right now, the tracks.py controller in rerun_tool The idea is that jobs run via Trackster are short, so they get a different queue:
--
subset_job, subset_job_outputs = tool.execute( trans, incoming=tool_params,
history=target_history,
job_params={ "source" : "trackster" } )
--
> Right now I'd like to be able to allocate jobs to different queues
> based on the input data size (and thus the expected compute time
> and resource needed), but that is rather complicated. e.g. If you
> have a low memory queue and a high memory query.
To make this work, you'd want to modify the execute() method in the DefaultToolAction class (/lib/galaxy/tools/actions/__init__.py) to add job parameters based on either tool parameters and/or input dataset size.
> You might even want different queues according to the user
> (e.g. one group might have paid for part of the cluster and get
> priority access).
This could also be done in the same location as trans.user will give you the user running the tool/job.
Best,
J.
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
