Re: [galaxy-dev] UCSC Display problem
This looks like Galaxy asking for part of a BAM file, using a byte range
request, but the server hosting the BAM file is not handling this. It is
probably a configuration error on that server, or perhaps in a proxy?
Peter
On Mon, Mar 10, 2014 at 11:09 PM, Pete Schmitt
peter.r.schm...@dartmouth.edu wrote:
Latest version of galaxy running on CentOS 5.10 with python 2.6.8
When attempting to use display at UCSC
mainhttp://dirigo.mdibl.org:8080/display_application/257f23ebf7c87bff/ucsc_bam/main
testhttp://dirigo.mdibl.org:8080/display_application/257f23ebf7c87bff/ucsc_bam/test
There is an error at UCSC: unable to get the display-data
Error from paster.log:
128.114.119.136 - - [10/Mar/2014:18:52:56 -0400] GET
/display_application/257f23ebf7c87bff/ucsc_bam/main/0a34505ca0c7b470/data/galaxy_257f23ebf7c87bff.bam
HTTP/1.0 200 817113 - genome.ucsc.edu/net.c
Exception happened during processing of request from ('128.114.119.136',
29811)
Traceback (most recent call last):
File
/nextgen3/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py,
line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File /usr/lib64/python2.6/SocketServer.py, line 322, in finish_request
self.RequestHandlerClass(request, client_address, self)
File /usr/lib64/python2.6/SocketServer.py, line 618, in __init__
self.finish()
File /usr/lib64/python2.6/SocketServer.py, line 661, in finish
self.wfile.flush()
File /usr/lib64/python2.6/socket.py, line 297, in flush
self._sock.sendall(buffer(data, write_offset, buffer_size))
error: [Errno 32] Broken pipe
I'm not sure what the error above means.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
inline: Screen Shot 2014-03-10 at 7.08.12 PM.png___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Dataset's extra files
Hi all, We have a local tool which role is to transfer (ie copy) a dataset file to a directory on our NFS. This is extremely convenient as it can be included within workflows and therefore save the time of clicking download button (we also have configurable renaming/compression as part of it). It is heavily used by our users. The problem is with datasets that have associated files like FASTQC as these extra files are simple not ignored... We'd like to improve our 'NFS_transfer' tool so it can deal with this in a similar fashion as the download button. Foreseen solution : * Check if a directory named 'dataset_id_files' exists within the dataset store * if so, 'cp -r' it into a tmp dir, cp the dataset itself into same tmp dir (with renaming on the fly) * zip/tar.gz the tmp dir * copy it to final NFS location Question is : is this the right way to do it ? As a non python specialist, it is a little tricky to find the right way to it (I can t locate the piece of code that does this in galaxy ie behind the download button). In particular, can I get the list of extra files using the '$galaxyFile' object given in the tool by : param type=data name=galaxyFile label=File to transfer/ i.e. in the same way we get the dataset name or file extension ($galaxyFile.dataset.name and $galaxyFile.ext) ? Any advise on how best to implement this, in a portable way, very appreciated. Thanks for your time, Charles = Charles Girardot Head of Genome Biology Computational Support (GBCS) European Molecular Biology Laboratory Tel: +49 6221 387 -8585 Fax: +49-(0)6221-387-8166 Email: charles.girar...@embl.de Room V205 Meyerhofstraße 1, 69117 Heidelberg, Germany = ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] job_wrapper.finish - No JSON object could be decoded:
Hi all, Some of our split BLAST jobs using the parallelism feature have been failing (apparently for a while, so this is not a recent problem from changes in the BLAST wrappers) as follows: quote The Galaxy framework encountered the following error while attempting to run the tool: Traceback (most recent call last): File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/tasks.py, line 141, in queue_job job_wrapper.finish( stdout, stderr, job_exit_code ) File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, line 962, in finish if ( not self.external_output_metadata.external_metadata_set_successfully( dataset, self.sa_session ) and self.app.config.retry_metadata_internally ): File /mnt/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py, line 638, in external_metadata_set_successfully rval, rstring = simplejson.load( open( metadata_files.filename_results_code ) ) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py, line 328, in load use_decimal=use_decimal, **kw) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py, line 384, in loads return _default_decoder.decode(s) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py, line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py, line 420, in raw_decode raise JSONDecodeError(No JSON object could be decoded, s, idx) JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) Tool execution generated the following error message: Unable to finish job /quote Reading the code, this is breaking after all the child jobs have finished, and the data has been merged, during the external set metadata step. This specific example was BLAST XML output, and the merge had produced a 4.2 GB file which ended mid record (evidently the output merge was not really successful). However, as a partial XML file, it would be invalid and might this cause problems with something in the metadata setting code - and explain the reported error? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] UCSC Display problem
Are you using Apache? This might help: http://lists.bx.psu.edu/pipermail/galaxy-user/2012-November/005508.html https://wiki.galaxyproject.org/Admin/Config/ApacheProxy Peter On Tue, Mar 11, 2014 at 1:16 PM, Pete Schmitt peter.r.schm...@dartmouth.edu wrote: Where would the configuration be in the galaxy server that would cause something like this. This worked in the previous year-old version of galaxy that was installed. On 3/11/14, 6:21 AM, Peter Cock wrote: This looks like Galaxy asking for part of a BAM file, using a byte range request, but the server hosting the BAM file is not handling this. It is probably a configuration error on that server, or perhaps in a proxy? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool_dependency_dir (Installed, missing tool dependencies)
Hello, It's difficult to determine the cause of the problem in your environment based on the details you've provided. If you have defined your tool_dependeny_dir configuration setting in your universe_wsgi.ini to be ../tool_dependencies, then the tool shed installation process will install tool dependencies there. If you see them installed elsewhere, then that setting must have been set differently at the time they were installed. You can try setting it to the location where your tool dependencies have been installed (i.e., the Galaxy installation directory), restat your server and uninstall them. Then you can reset it to the more appropriate location (e.g., ../tool_dependencies), restart yiour server and reinstall them. If you have tool dependencies installed in multiple locations, you'll have to be careful to use a process that will eventually result in all tool dependencies installed into a single location. Greg Von Kuster On Mar 6, 2014, at 5:41 PM, Wang, Xiaofei xfw...@ku.edu wrote: Dear there, I am trying to install some tools from toolshed on my local galaxy instance, like bwa_wrappers, package_picard_1_56_0, package_samtools_0_1_18, and sam_to_bam. For the first time, I got a warning before installing, something like this set the value of your tool_dependency_dir setting in universe_wsgi_ini and restart before installing. Then, I followed the answer on this thread (Problem installing tool_ Galaxy local) as below. I edited the universe_wsgi_ini like this: #Path to the directory in which managed tool dependencies are placed. To use # the dependency system, see the documentation at: # http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies #tool_dependency_dir = None tool_dependency_dir = ../tool_dependencies Then, I restarted and installed the tool again. Galaxy create a directory named tool_dependencies at the same level in local file system hierarchy as the galaxy installation directory (same level as galaxy-dist). Some softwares such as bwa and picard are installed automatically here. But, when I checked in the admin manage installed tool shed repositories, the installation status is Installed, missing tool dependencies for the four tools I mentioned above. I do not know why this happened in my case? I tried to figure it out follow this thread The migrated BWA installed but missing dependencies. But, I did not get the idea. Could you tell me why this happen and how to fix it up? Thank you so much! -- Your setting for tool_dependency_dir should not be as you've stated: tool_dependency_dir = tool_dependencies It should be an actual path on your local file system, something like: tool_dependency_dir = ../tool_dependencies If you use the above setting, then Galaxy should create a directory named tool_dependency_dir at the same level in your local file system hierarchy as your Galaxy installation directory. I cannot guarantee that this change will solve your problem, but it will be a step in the right direction. After making this change, let us know if you still see problems. Greg Von Kuster ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] launching openmpi tool with Galaxy
Dear developers, I'm trying to launch a mpi version of test tool. To do so, I have a test xml called sleep.xml but all he does is write the hostname of the machine in an output file. (command: /usr/lib64/openmpi/bin/mpirun hostname $output) When I launch this tool as a local job (we use son of grid engine as the job scheduler), everything is ok. When i specify the following line in the universe_wsgi.xml: sleep = drmaa://-q galaxy -pe openmpi 8/ The program is launched but it is lost somewhere and juste doesn't do anything. It is just running and never ends. Does someone has an explanation for that ? Thanks in advance, -- Olivia Doppelt-Azeroual, PhD Tel: 92 15 CIB - Institut Pasteur ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem with change_format and conditional inputs?
Hi Peter,
I think you need to have:
change_format
when input=output.out_format value=0 format=txt/
when input=output.out_format value=0 -html format=html/
when input=output.out_format value=2 format=txt/
when input=output.out_format value=2 -html format=html/
when input=output.out_format value=4 format=txt/
when input=output.out_format value=4 -html format=html/
when input=output.out_format value=5 format=blastxml/
/change_format
Sorry, can't test it right now.
Best,
Bjoern
Am 10.03.2014 16:28, schrieb Peter Cock:
Hi all,
An eagle eyed user has just spotted a bug in our BLAST wrappers
and/or the Galaxy framework itself.
1. Run BLASTN (wrapper version v0.1.00 which is currently on the Test Tool Shed)
2. Select BLASTXML output, execute
3. While running click the i icon, which wrongly reports:
Format:tabular
4. Check the repeat, which correctly has XML output
5. Once finished, notice the output is XML, but Galaxy labelled it tabular
6. Workaround via the pencil icon, correct the datatype to blastxml.
This happens both on our production server:
$ hg summary
parent: 11218:26f58e05aa10 release_2013.11.04
Galaxy stable release for 2013.11.04.
branch: stable
And also on my development server with galaxy-central:
$ hg summary
parent: 13675:bc1173309bd5 tip
Variable name fix for the tool shed's install and test framework.
branch: default
This used to work perfectly. We've not changed this area of the code
in the BLAST wrappers directly - it uses a default output type of tabular
and the change_format tag - expanding the macro we have:
outputs
data name=output1 format=tabular
label=${blast_type.value} $query.name vs @ON_DB_SUBJECT@
!-- expand macro=output_change_format / --
change_format
when input=out_format value=0 format=txt/
when input=out_format value=0 -html format=html/
when input=out_format value=2 format=txt/
when input=out_format value=2 -html format=html/
when input=out_format value=4 format=txt/
when input=out_format value=4 -html format=html/
when input=out_format value=5 format=blastxml/
/change_format
/data
/outputs
What has changed recently in the NCBI BLAST wrappers is
that input parameter out_format is now nested within a
conditional block.
Other tools using change_format seem to still work fine:
https://github.com/peterjc/pico_galaxy/tree/master/tools/seq_filter_by_id
Can anyone testing the BLAST+ update on the Test Tool Shed
reproduce this? Any thoughts on the cause and fix? I have to
attend a meeting now, so I've not dug any deeper yet...
Regards,
Peter
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Problem with change_format and conditional inputs?
On Tue, Mar 11, 2014 at 2:58 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Peter, I think you need to have: change_format when input=output.out_format value=0 format=txt/ when input=output.out_format value=0 -html format=html/ when input=output.out_format value=2 format=txt/ when input=output.out_format value=2 -html format=html/ when input=output.out_format value=4 format=txt/ when input=output.out_format value=4 -html format=html/ when input=output.out_format value=5 format=blastxml/ /change_format Sorry, can't test it right now. That seems to work - thanks Björn :) This seems to have exposed a bug in the test framework, e.g. test param name=query value=rhodopsin_nucs.fasta ftype=fasta / param name=db_opts_selector value=file / param name=subject value=three_human_mRNA.fasta ftype=fasta / param name=database value= / param name=evalue_cutoff value=1e-40 / param name=out_format value=5 / param name=adv_opts_selector value=basic / output name=output1 file=blastn_rhodopsin_vs_three_human.xml ftype=blastxml / /test This was saying the output should have been blastxml, but until I just fixed it the output was being tagged as tabular (although run_functional_tests.sh did check the content it didn't check the datatype matched). Dave - do think this is a reasonable enhancement? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] job_wrapper.finish - No JSON object could be decoded:
On Tue, Mar 11, 2014 at 11:44 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, Some of our split BLAST jobs using the parallelism feature have been failing (apparently for a while, so this is not a recent problem from changes in the BLAST wrappers) as follows: quote The Galaxy framework encountered the following error while attempting to run the tool: Traceback (most recent call last): File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/tasks.py, line 141, in queue_job job_wrapper.finish( stdout, stderr, job_exit_code ) File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, line 962, in finish if ( not self.external_output_metadata.external_metadata_set_successfully( dataset, self.sa_session ) and self.app.config.retry_metadata_internally ): File /mnt/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py, line 638, in external_metadata_set_successfully rval, rstring = simplejson.load( open( metadata_files.filename_results_code ) ) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py, line 328, in load use_decimal=use_decimal, **kw) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py, line 384, in loads return _default_decoder.decode(s) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py, line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py, line 420, in raw_decode raise JSONDecodeError(No JSON object could be decoded, s, idx) JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) Tool execution generated the following error message: Unable to finish job /quote Reading the code, this is breaking after all the child jobs have finished, and the data has been merged, during the external set metadata step. This specific example was BLAST XML output, and the merge had produced a 4.2 GB file which ended mid record (evidently the output merge was not really successful). However, as a partial XML file, it would be invalid and might this cause problems with something in the metadata setting code - and explain the reported error? Peter Clues from my log: 2014-03-11T07:11:16.635421+00:00 ppserver galaxy.jobs.splitters.multi ERROR 2014-03-11 07:11:16,563 Error merging files#012Traceback (most recent call last):#012 File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/splitters/multi.py, line 153, in do_merge#012output_type.merge(output_files, output_file_name, **extra_merge_args)#012 File /mnt/galaxy/galaxy-dist/lib/galaxy/datatypes/data.py, line 572, in merge#012shutil.copyfileobj(open(fsrc, 'rb'), fdst)#012 File /usr/lib64/python2.6/shutil.py, line 31, in copyfileobj#012 fdst.write(buf)#012IOError: [Errno 28] No space left on device 2014-03-11T07:11:25.594665+00:00 ppserver galaxy.jobs.runners.tasks ERROR 2014-03-11 07:11:25,558 Job wrapper finish method failed#012Traceback (most recent call last):#012 File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/runners/tasks.py, line 141, in queue_job#012job_wrapper.finish( stdout, stderr, job_exit_code )#012 File /mnt/galaxy/galaxy-dist/lib/galaxy/jobs/__init__.py, line 962, in finish#012if ( not self.external_output_metadata.external_metadata_set_successfully( dataset, self.sa_session ) and self.app.config.retry_metadata_internally ):#012 File /mnt/galaxy/galaxy-dist/lib/galaxy/datatypes/metadata.py, line 638, in external_metadata_set_successfully#012rval, rstring = simplejson.load( open( metadata_files.filename_results_code ) )#012 File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py, line 328, in load#012use_decimal=use_decimal, **kw)#012 File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py, line 384, in loads#012return _default_decoder.decode(s)#012 File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py, line 402, in decode#012obj, end = self.raw_decode(s, idx=_w(s, 0).end())#012 File /mnt/galaxy/galaxy-dist/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py, line 420, in raw_decode#012raise JSONDecodeError(No JSON object could be decoded, s, idx)#012JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) So this looks like the root cause was a full partition, but Galaxy handled this poorly (showing a JSON error rather than disk full). This proposed patch should give a more helpful error message from the metadata script, which is a start: $ hg diff lib/galaxy/datatypes/metadata.py diff -r 26f58e05aa10 lib/galaxy/datatypes/metadata.py --- a/lib/galaxy/datatypes/metadata.pyMon Nov 04 14:58:55 2013 -0500 +++ b/lib/galaxy/datatypes/metadata.pyTue Mar 11
[galaxy-dev] Internal server error when importing workflows!
Hi, I have upgraded my local galaxy to the latest stable version and it works fine but I get an error when I try to import workflows that were created older version of Galaxy. Here is the error I keep getting. Internal Server Error Galaxy was unable to successfully complete your request URL: http://10.7.10.21/workflow/import_workflow Module galaxy.web.framework.middleware.error:149 in __call__ http://10.7.10.21/workflow/import_workflow app_iter = self.application(environ, sr_checker) Module paste.recursive:84 in __call__ http://10.7.10.21/workflow/import_workflow return self.application(environ, start_response) Module paste.httpexceptions:633 in __call__ http://10.7.10.21/workflow/import_workflow return self.application(environ, start_response) Module galaxy.web.framework.base:132 in __call__ http://10.7.10.21/workflow/import_workflow return self.handle_request( environ, start_response ) Module galaxy.web.framework.base:190 in handle_request http://10.7.10.21/workflow/import_workflow body = method( trans, **kwargs ) Module galaxy.webapps.galaxy.controllers.workflow:1072 in import_workflow http://10.7.10.21/workflow/import_workflow workflow, missing_tool_tups = self._workflow_from_dict( trans, data, source=src, add_to_menu=add_to_menu ) Module galaxy.web.base.controller:1639 in _workflow_from_dict http://10.7.10.21/workflow/import_workflow module.save_to_step( step ) Module galaxy.workflow.modules:274 in save_to_step http://10.7.10.21/workflow/import_workflow step.tool_inputs = self.tool.params_to_strings( self.state.inputs, self.trans.app ) Module galaxy.tools:2418 in params_to_strings http://10.7.10.21/workflow/import_workflow return params_to_strings( self.inputs, params, app ) Module galaxy.tools.parameters:85 in params_to_strings http://10.7.10.21/workflow/import_workflow value = params[ key ].value_to_basic( value, app ) Module galaxy.tools.parameters.grouping:468 in value_to_basic http://10.7.10.21/workflow/import_workflow rval[ input.name ] = input.value_to_basic( value[ input.name ], app ) KeyError: 'index' Can someone let me know how to fix this issue? Thank you Hakeem ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Persistent jobs in cluster queue even after canceling job in galaxy
Hi All, I've been able to submit jobs to the cluster through galaxy, it works great. But when the job is in queue to run (it is gray in the galaxy history pane) and I cancel the job, it still remains in queue on the cluster. Why does this happen? How can I delete the jobs in queue as well? I tried qdel job_id as galaxy user but it says I am not authenticated to delete the job. Any help would be greatly appreciated. Thanks Ravi. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Persistent jobs in cluster queue even after canceling job in galaxy
On Tue, Mar 11, 2014 at 5:57 PM, Ravi Alla ravi.a...@berkeley.edu wrote: Hi All, I've been able to submit jobs to the cluster through galaxy, it works great. But when the job is in queue to run (it is gray in the galaxy history pane) and I cancel the job, it still remains in queue on the cluster. Why does this happen? How can I delete the jobs in queue as well? I tried qdel job_id as galaxy user but it says I am not authenticated to delete the job. Any help would be greatly appreciated. Thanks Ravi. What kind of cluster is it? e.g. SGE? Are you using task splitting (parallelization)? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Job never starts
Hi All, I have installed the recent version of galaxy and starting multiple web and job handlers(six each) on a Centos 5.1 machine. It is working almost perfectly. 1. The first problem is sometimes jobs never start and they are in grey stage. After I killed them and start again, they work fine. There are no any log about why they haven’t started. So, if you direct me to find the reason, it would be great. 2. Second problem is about UCSC visualization issue using the links from history. I haven’t found a solution yet, my XSendFile is all set and working in the previous version but not this one. The only difference is I am using the new version and balancer in Apache since I run multiple instances. So, balancer might not use XSendFile properly or there is a configuration problem. http://lists.bx.psu.edu/pipermail/galaxy-user/2012-November/005508.html https://wiki.galaxyproject.org/Admin/Config/ApacheProxy Let me know, if anybody has a similar settings and solved this problem with Apache. Best, Alper Kucukural, PhD Bioinformatics Core, University of Massachusetts Medical School 368 Plantation St.Room AS4.2067 Worcester, MA 01605-2324 Phone: 774-312-4493 E-mail: al...@kucukural.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Persistent jobs in cluster queue even after canceling job in galaxy
Hi Peter, No I am not using that option. It is currently set to false in my universe_wgsi.ini file. It says it is a new feature and not recommended, so I didn't mess with it. Thanks Ravi On Mar 11, 2014, at 11:27 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Ravi, Could you reply to the list? And actually I meant do you have use_tasked_jobs = True in your universe_wsgi.ini file which is linked to the special XML tag parallelism in some Galaxy Tools for splitting jobs. Sorry, that was unclear of me. Peter On Tue, Mar 11, 2014 at 6:24 PM, Ravi Alla ravi.a...@berkeley.edu wrote: Peter, This is a Centos linux cluster using PBS Torque job manager. By parallelization are you referring to multiple job handlers? No I am not. I barely set this server up and it is in its most basic form so far. I will add options for multiple web servers and runners once I get things working soundly. Thanks Ravi On Mar 11, 2014, at 11:07 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, Mar 11, 2014 at 5:57 PM, Ravi Alla ravi.a...@berkeley.edu wrote: Hi All, I've been able to submit jobs to the cluster through galaxy, it works great. But when the job is in queue to run (it is gray in the galaxy history pane) and I cancel the job, it still remains in queue on the cluster. Why does this happen? How can I delete the jobs in queue as well? I tried qdel job_id as galaxy user but it says I am not authenticated to delete the job. Any help would be greatly appreciated. Thanks Ravi. What kind of cluster is it? e.g. SGE? Are you using task splitting (parallelization)? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] error in workflow
Dear Galaxy Developers,I'm running galaxy local on ubuntu and trying to run a workflow on multiple datasets (separately). Occasionally when I try to run workflow on input dataset I get Unable to finish job error in some steps, in most time the problem will be solved when I run the workflow again, but this going to happen more frequently. I attached the error message here. I suspect it may be related to increase in number of datasets of the current history (1757 datasets with ~350GB size).I will appreciate your helpTraceback (most recent call last): File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 116, in queue_job job_wrapper.finish( stdout, stderr, exit_code ) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/lib/galaxy/jobs/__init__.py, line 1068, in finish self.sa_session.flush() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py, line 114, in do return getattr(self.registry(), name)(*args, **kwargs) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 1718, in flush self._flush(objects) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 1804, in _flush transaction.commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py, line 365, in commit t[1].commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2045, in commit self._do_commit() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 2075, in _do_commit self.connection._commit_impl() File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 1280, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py, line 1277, in _commit_impl self.engine.dialect.do_commit(self.connection) File /media/milad/acfed08f-e8e7-43d5-b582-5b5acdba9072/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py, line 307, in do_commit connection.commit() OperationalError: (OperationalError) database is locked None None ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] ?????? Re: Local Galaxy web layout
Hi Sam, Using static/scripts/galaxy.menu.js,I can add/remove/change labels of my local Galaxy successfully,but I don't know how to change the positions ,as well as the Galaxy interface.So are there any details or guidances for this help?Thank youvery much. Regards, Shenwiyn -- -- ??: sam guerler;aysam.guer...@gmail.com; : 2014??2??19??(??) 11:06 ??: shenw...@gmail.comshenw...@gmail.com; : hrhh...@fmi.ch; galaxy-devgalaxy-dev@lists.bx.psu.edu; : Re: Re: [galaxy-dev] Local Galaxy web layout Hi Shenwiyn, Can you specify what is not working? Did you find the file? Thanks, Sam On Wed, Feb 19, 2014 at 1:42 AM, shenw...@gmail.com shenw...@gmail.com wrote: Hi Sam and Hans-Rudolf?? Thank you very much for your help.It seems that the static/scripts/galaxy.menu.js fail to work in my local Galaxy.As a result,would you please show me some more details about this? Regards, Shenwiyn From: sam guerler Date: 2014-02-18 01:20 To: Hans-Rudolf Hotz CC: shenw...@gmail.com; galaxy-dev Subject: Re: [galaxy-dev] Local Galaxy web layout Hi Shenwiyn, This has changed recently. Please take a look at: static/scripts/galaxy.menu.js Using that file, you can add/remove/change labels and their position. It is recommended to run pack_scripts.py after changing the source file. Regards, Sam On Mon, Feb 17, 2014 at 11:33 AM, Hans-Rudolf Hotz h...@fmi.ch wrote: Hi Shenwiyn Have a look at: ~/galaxy_dist/database/compiled_templates/base_panels.mako.py Regards, Hans-Rudolf On 02/17/2014 08:04 AM, shenw...@gmail.com wrote: Galaxy developers: I want to change my local Galaxy web layout,for example,delete the top button of Visualization and Cloud or change their positions.Could anyone show me some details about this? Thank you very much. shenw...@gmail.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] History seems to load forever
Hi List, I just pulled and merged from galaxy-dist (I think it's dc067a95261d), updated my database, and migrated my tools. The issues I am able to see are minor, but I have a user who can no longer access a particular history. It apparently just hangs. If they delete the history, Galaxy responds, but it will hang again when they try to add back any datasets. Unfortunately, I don't see any errors in the paster logs about this. I fear that their dataset entries in the Postgresql db have been corrupted during the update, or perhaps the files themselves were purged. I cannot reproduce their issues with my own histories - I'm able to bring up saved histories and upload etc. I don't know if they can share a history if it can't be loaded. How should I troubleshoot this? I'm set up to authenticate through apache, which makes it harder to get in as the user and see what's up. Is there an easy way to find out what histories belong to what users? I can search the database? Any advice would be greatly appreciated! -Carrie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] slurm drmaaa
Jillian, We've connected our Galaxy instance to a slurm cluster via slurm-drmaa [1], and use the DRMAAJobRunner plugin configured according to the Galaxy admin wiki [2]. It works pretty well, but it should be noted that the nativeSpecification options slurm-drmaa makes available to you (that is, options like those you'd pass to sbatch) are *not* complete. For example, one of our IT guys just patched slurm-drmaa to add support for the --cpus-per-task option, which wasn't present in slurm-drmaa 1.0.7. [1] http://apps.man.poznan.pl/trac/slurm-drmaa [2] https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster#DRMAA On Mon, Mar 10, 2014 at 5:33 AM, Jillian Rowe jir2...@qatar-med.cornell.edu wrote: Hello all, I would like to get galaxy to submit jobs to slurm using the drmaa. Galaxy exists on its own machine, and slurm is run on a cluster. I am at a bit of a lose as to how to do this. I have galaxy installed and working fine, and slurm works fine on its own. I saw a blog post detailing how to get the two connected, but it seems like maybe it was outdated. Any tips? Best, Jillian ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Brian Claywell, Systems Analyst/Programmer Fred Hutchinson Cancer Research Center bclay...@fhcrc.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Persistent jobs in cluster queue even after canceling job in galaxy
We've had the same problem since updating from the April 2013 stable to Feb 2014 stable. Our jobs are going off to a slurm cluster via the SlurmJobRunner plugin (though this was happening with the DRMAAJobRunner plugin too, if I remember right). Removing pending datasets occasionally removes the entry in Admin/Jobs, but not reliably, and regardless the jobs stay queued in slurm with no noise in galaxy.log at DEBUG or higher. We're not using parallelism either. I noticed this around the same time as I noticed the new in-progress animation in the history pane; perhaps there's an ajax-y callback that's not firing? Otherwise I'd expect to see something in the log about Galaxy at least trying to cancel the job. On Tue, Mar 11, 2014 at 10:57 AM, Ravi Alla ravi.a...@berkeley.edu wrote: Hi All, I've been able to submit jobs to the cluster through galaxy, it works great. But when the job is in queue to run (it is gray in the galaxy history pane) and I cancel the job, it still remains in queue on the cluster. Why does this happen? How can I delete the jobs in queue as well? I tried qdel job_id as galaxy user but it says I am not authenticated to delete the job. Any help would be greatly appreciated. Thanks Ravi. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Brian Claywell, Systems Analyst/Programmer Fred Hutchinson Cancer Research Center bclay...@fhcrc.org ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
