Re: [galaxy-dev] Galaxy not killing split cluster jobs
On Tue, May 1, 2012 at 3:46 PM, Dannon Baker dannonba...@me.com wrote: On Tue, May 1, 2012 at 3:10 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On May 1, 2012, at 10:03 AM, Dannon Baker dannonba...@me.com wrote: On May 1, 2012, at 9:51 AM, Peter Cock wrote: I'm a little confused about tasks.py vs drmaa.py but that TODO comment looks pertinent. Is that the problem here? The runner in tasks.py is what executes the primary job, splitting and creating the tasks. The tasks themselves are actually injected back into the regular job queue and run as normal jobs with the usual runners (in your case drmaa). And, yes, it should be fairly straightforward to add, but this just hasn't been implemented yet. -Dannon So the stop_job method for the runner in task.py needs to call the stop_job method of each of the child tasks it created for that job (which in this case are drmaa jobs - but could be pbs etc jobs). I'm not really clear how all that works. Should I open an issue on this? Peter I'll take care of it. Thanks for reminding me about the TODO! Hi Dannon, Is this any nearer the top of your TODO list? I was reminded by having to manually log onto our cluster today and issue a bunch of SGE qdel commands to manually kill a job which was hogging the queue, but had bee deleted in Galaxy. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Error starting Galaxy after upgrade.
Hi, I upgraded Galaxy to the latest version using the command: hg pull -u, then when I run sh run.sh I got the following error messages: Initializing openid_conf.xml from openid_conf.xml.sample Initializing tool-data/bowtie2_indices.loc from bowtie2_indices.loc.sample Some eggs are out of date, attempting to fetch... Traceback (most recent call last): File ./scripts/fetch_eggs.py, line 30, in module c.resolve() # Only fetch eggs required by the config File /home/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py, line 345, in resolve egg.resolve() File /home/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py, line 195, in resolve return self.version_conflict( e.args[0], e.args[1] ) File /home/galaxy/galaxy-dist/lib/galaxy/eggs/__init__.py, line 226, in version_conflict r = pkg_resources.working_set.resolve( ( dist.as_requirement(), ), env, egg.fetch ) File /usr/lib/python2.6/dist-packages/pkg_resources.py, line 546, in resolve raise DistributionNotFound(req) pkg_resources.DistributionNotFound: mercurial==2.1.2 Fetch failed. What happened here? Thanks, Luobin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Re : Re : Error executing tool: cannot find 'database'
Hi Marine If I use the xml file as you have posted in this e-mail, it works. ie it works up to the error message: An error occurred running this job: /bin/sh: /home/myName/myToolRepository/bin/myTool: not found I don't get the cheetah error you show. Are you sure, you are using the same xml file? Regards, Hans On 05/30/2012 06:06 PM, Marine Rohmer wrote: Ok, when I remove interpreter = C++ from command, and keep it empty or write interpreter = bash instead, I have no more this error message. Galaxy says The following job has been successfully added to the queue,and I can show its progression on the history. Thank you Ross ! But now I have a new error message that I can't solve : An error occurred running this job: /failure preparing job /Details are: Traceback (most recent call last): File /home/myName/work/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 59, in run_job job_wrapper.prepare() File /home/myName/work/galaxy-dist/lib/galaxy/jobs/__init__.py, line 429, in prepare self.command_line = self.tool.build_command_line( param_dict ) File /home/myName/work/galaxy-dist/lib/galaxy/tools/__init__.py, line 1971, in build_command_line command_line = fill_template( self.command, context=param_dict ) File /home/myName/work/galaxy-dist/lib/galaxy/util/template.py, line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File /home/myName/work/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py, line 1004, in __str__ return getattr(self, mainMethName)() File cheetah cheetah_DynamicallyCompiledCheetahTemplate_1338392112_91_81444.py, line 83, in respond NotFound: cannot find 'HOME' I've tried to put an erroneous path/to/myTool in command in order to see if my original path is right. It indeed gives me another error message saying An error occurred running this job: //bin/sh: 1: path/to/myTool: not found/ , which makes sense since the path is erroneous. I've also tried to put some erroneous data in the XML file (for example wrong names in param) to see if it influences the error message, but I still have the same. (that is to say : NotFound: cannot find 'HOME') With Hans' help, my XML file is bettered that way : tool id=mytool name=My Tool description description /description command /home/myName/myToolRepository/bin/myTool -i = $genome_index -r = $input -k = $kmer_length /command inputs param name=genome_index type=select label=Genome index option value=E.coli /path/to/indexEcoli /option option value=Human path/to/indexHuman /option /param param format=txt, raw, doc, fastq name=input type=data label=Source File help=Select a file/ param name=kmer_length type=integer min=15 max=150 value=21 label=k-mer length/ /inputs outputs data format=sam name=crac_outfile label=crac outfile / /outputs tests test /test /tests help /help /tool Regards, Marine ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Is there a way to dynamically assign Queues in Galaxy
Hi,I often find my self having to battle with other users of the cluster, even for the one-off jobs that someone runs interactively...We have setup Fast/short queues and Long queues...Is there a way for a tool to sometimes be placed in the fast queue and sometimes in the long queue?I know I can setup a tool in the universe file and add a -pe option there...But is there a way to dynamically assign jobs sometimes to one queue and sometimes to another?Even an option in the UI would work, since I could make the short queue the default for interactive work and people could turn it off in workflows that will run for a long time or will run with many samplesThanksThon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] tool_dependency_dir not being picked up?
Hi - I have been trying to run a quick test using the tool_dependency_dir configuration option to prove out how it works ... but unfortunately I can't get it to function right. I was hoping someone could point out my error. I have written a simple tool in galaxy, version_test, that will just echo out a version number to a text file so I can test out tool dependencies and install locations. But my required package is not getting resolved by galaxy Snippet from my configuration file: # Directory which contains dependent tool binaries or a env.sh to set env vars in order to find specific versions. tool_dependency_dir = /opt My simple tool: tool id=version_test name=version_test version=1.0.0 requirements requirement type=packagegalaxy_test/requirement /requirements descriptionprovides simple stats on BAM files/description commandtest.sh $output1 /command inputs param name=input1 type=data format=tabular,binary label=Holder for input file - not used. / /inputs outputs data name=output1 format=txt / /outputs /tool My filesystem: [galaxy@galaxy-0-4:galaxy-dist]$ ls -lat /opt/galaxy_test/ total 20 drwxr-xr-x 3 root root 4096 May 16 14:10 2.0 drwxr-xr-x 3 root root 4096 May 16 14:09 1.0 drwxr-xr-x 4 root root 4096 May 16 14:05 . -rw-r--r-- 1 root root 50 May 16 14:05 env.sh drwxr-xr-x 72 root root 4096 May 16 14:04 .. My env file: #!/bin/bash PATH=/opt/galaxy_test/1.0/bin:$PATH export PATH Galaxy log messages when invoking my test tool: galaxy.tools DEBUG 2012-05-31 21:59:17,349 Dependency galaxy_test galaxy.tools WARNING 2012-05-31 21:59:17,349 Failed to resolve dependency on 'galaxy_test', ignoring If I manually source the env.sh file , my test.sh is found and executes as appropriately. Where am I going wrong? Thanks much! Ann ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
