Re: [galaxy-dev] Freebayes
Juan, I'm not sure of a likely cause of this error off the top of my head. Have you tried running Freebayes from the command line on the same data input? That might be an easy way to explore what particular options with the newer version of Freebayes that you have are causing issues, or perhaps would let us know if the input files are somehow truly invalid. -Dannon On Apr 4, 2011, at 11:25 AM, Juan Carlos Perin wrote: This fix got me further, however I'm now getting the following error, which is now more confusing... [bam_sort_core] merging from 40 files... BGZF ERROR: unable to open file 1.0 BGZF ERROR: unable to open file 1.0 Could not open input BAM files Juan On Apr 1, 2011, at 1:53 PM, Dannon Baker wrote: Juan, What version of freebayes are you running? The freebayes configuration in the galaxy repository was written with the .4 series in mind, and it appears that the options have changed with the .6 series. We'll update the tool config, but in the meanwhile you could probably get it working again for your local use by editing the tools/human_genome_variation/freebayes.xml file to comment out the no-longer-valid parameter in line 34: -Y $params.postIntegBanddepth Let me know if this doesn't get things moving again for you, Dannon On Apr 1, 2011, at 1:17 PM, Juan Carlos Perin wrote: I'm trying to run Freebayes SNP calling on a bam file from a BWA alignment of solid data. I keep getting the following error: [bam_sort_core] merging from 40 files... freebayes: invalid option -- Y did you mean --bam ? I'm not sure where this is set in the code to try to debug things. I'm wondering if anyones had this error, and how to fix it? Just to note, I'm running Brad Chapmans branch of galaxy for the LIMS functionality. But it appears to be an issue with either galaxy-dist or this one. Thanks in advance for any advice. Juan Perin ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Ignoring white space differences in test output
On Tue, Apr 5, 2011 at 10:58 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I'd like to write a tool test where the sample output and the generated output are the same except for white space. i.e. running diff with the -w switch reports no differences. Is this possible at the moment with the Galaxy test framework? I'm hoping for a test output attribute. I don't see anything like this mentioned here: https://bitbucket.org/galaxy/galaxy-central/wiki/WritingTests Thanks, Peter I've filed an enhancement request for this, https://bitbucket.org/galaxy/galaxy-central/issue/510 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating BLAST+ wrapper
On Mon, Apr 4, 2011 at 6:07 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Apr 4, 2011 at 5:55 PM, Kanwei Li kan...@gmail.com wrote: Done the manual way Thanks. I was hoping to have finished this today, but it turns out there are some subtle but annoying changes to how BLAST records the hit id/name/description in their XML file which make the script to convert from XML to tabular unhappy. Hi Kanwei, I think I'm nearly done now, but I am still working on the blast25 branch you had trouble merging/transplanting from. I guess I will need to make a new clean branch from the galaxy tip for submission. Do you prefer several small atomic commits (which I understand is the recommend practice in git), or a few large commits (which seems quite common on the Galaxy hg history)? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Trouble viewing interval files in UCSC with Apache+XSendFile
Hello, I've encountered a strange combination of factors that results in file access problems. Perhaps I'm doing something wrong - any advice will be appreciated. I'm using Apache + XSendFile. when viewing a strict BED file, everything works, because the actual dataset filename is passed on to Apache/XSendFile. When viewing any other kind of interval file, Galaxy first creates a temporary file in strict BED format ( in Interval::as_ucsc_display_file() ). This method uses tempfile.mkstemp(), which is documented to create a temporary file with user-only read/write access (no group or world access). So when the file name is passed on to apache/XsendFile - apache can't read the file and returns 404. I'm wondering if anyone else encountered such problem, or is my configuration is somehow wrong. Thanks, -gordon ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Ratings on Tool Shed broken?
Hi all, I had wondered why none of the tools on the Galaxy Tool Shed had a rating, and had assumed it was just that so far no-one had bothered. I have a new theory, it doesn't work? I just viewed Brad's BAM to BigWig tool, clicked on Tool Actions then Rate Tool, and got: Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) Can anyone reproduce this? I've tried a couple of other tools and the same thing happens. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] SGE and Galaxy (a different approach)
I'm aware of how to configure Galaxy to use SGE in universe_wsgi.ini, however what I want to do is a little different. Because I only want certain processes to be submitted to the queue, I'd rather control this at the tool configuration level (the xml wrapper). For example: command interpreter=bash qsub myscript.sh /command This will work, except that the status of the job (in Galaxy) shows as completed even though the job has simply been submitted to SGE. Basically Galaxy 'loses track' of the process because the submission process (myscript.sh) has completed even if the actual job hasn't. Has anyone else tried anything like this before, or have anything helpful to suggest? One thought is to somehow cause the myscript.sh process to pause until the SGE job has completed... somehow. Any advice appreciated. Thanks, Andrew ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] SGE and Galaxy (a different approach)
Ah, I should have added that this is for a strictly in-house Galaxy implementation with absolutely no intention of publishing these specific tools (just these ones! I promise I'll publish other tools in the future :) On Tue, Apr 5, 2011 at 12:38 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Tue, Apr 5, 2011 at 5:27 PM, andrew stewart andrew.c.stew...@gmail.com wrote: I'm aware of how to configure Galaxy to use SGE in universe_wsgi.ini, however what I want to do is a little different. Because I only want certain processes to be submitted to the queue, I'd rather control this at the tool configuration level (the xml wrapper). Hi Andrew, That seems like a bad idea for portable XML wrappers - having local configuration information hard coded within a tool wrapper would prevent the tool (and any workflow using it) from being shared with another Galaxy server. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] SGE and Galaxy (a different approach)
Use qrsh instead of qsub to submit the job should help, I believe. On Tue, Apr 5, 2011 at 10:27 AM, andrew stewart andrew.c.stew...@gmail.comwrote: I'm aware of how to configure Galaxy to use SGE in universe_wsgi.ini, however what I want to do is a little different. Because I only want certain processes to be submitted to the queue, I'd rather control this at the tool configuration level (the xml wrapper). For example: command interpreter=bash qsub myscript.sh /command This will work, except that the status of the job (in Galaxy) shows as completed even though the job has simply been submitted to SGE. Basically Galaxy 'loses track' of the process because the submission process (myscript.sh) has completed even if the actual job hasn't. Has anyone else tried anything like this before, or have anything helpful to suggest? One thought is to somehow cause the myscript.sh process to pause until the SGE job has completed... somehow. Any advice appreciated. Thanks, Andrew ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] SGE and Galaxy (a different approach)
andrew stewart wrote: I'm aware of how to configure Galaxy to use SGE in universe_wsgi.ini, however what I want to do is a little different. Because I only want certain processes to be submitted to the queue, I'd rather control this at the tool configuration level (the xml wrapper). For example: Hi Andrew, If you're not aware, it's possible to specify that only certain tools should be run on a specific cluster with certain parameters. See the bottom of the Cluster documentation for detials (under the Galaxy Configuration and Tool Configuration headers): https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster command interpreter=bash qsub myscript.sh /command This will work, except that the status of the job (in Galaxy) shows as completed even though the job has simply been submitted to SGE. Basically Galaxy 'loses track' of the process because the submission process (myscript.sh) has completed even if the actual job hasn't. To avoid needing a wrapper that executes qsub for you, you may be able to just run your command behind qrsh or qlogin instead of qsub - this will remain in the foreground until the job completes. Has anyone else tried anything like this before, or have anything helpful to suggest? One thought is to somehow cause the myscript.sh process to pause until the SGE job has completed... somehow. To use qsub, you'd need to make a wrapper around myscript.sh along the lines of the following: #!/bin/sh jobid=`qsub -terse myscript.sh` while [ 1 ]; do qstat -j $jobid /dev/null 21 [ $? -eq 1 ] break; sleep 5; done -- The above script is the most basic option, it would probably be best to add more error handling and reporting. --nate Any advice appreciated. Thanks, Andrew ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] SGE and Galaxy (a different approach)
On Tue, Apr 5, 2011 at 12:27 PM, andrew stewart andrew.c.stew...@gmail.com wrote: I'm aware of how to configure Galaxy to use SGE in universe_wsgi.ini, however what I want to do is a little different. Hi, Andrew. Take a look at this page: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster In particular, does the last section, Tool Configuration, describe something like what you want to do? Sean Because I only want certain processes to be submitted to the queue, I'd rather control this at the tool configuration level (the xml wrapper). For example: command interpreter=bash qsub myscript.sh /command This will work, except that the status of the job (in Galaxy) shows as completed even though the job has simply been submitted to SGE. Basically Galaxy 'loses track' of the process because the submission process (myscript.sh) has completed even if the actual job hasn't. Has anyone else tried anything like this before, or have anything helpful to suggest? One thought is to somehow cause the myscript.sh process to pause until the SGE job has completed... somehow. Any advice appreciated. Thanks, Andrew ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Optimizing Disk space using by combining mapping with SAM-to-BAM
Hi Gordon - It looks like on the samtools mailing lists there is an active discussion on speeding up sorting: http://sourceforge.net/mailarchive/message.php?msg_id=27247076 http://sourceforge.net/mailarchive/message.php?msg_id=26990598 Interestingly enough there is a recommendation on using Picard instead of samtools. Are there Galaxy tools scripts for Picard? This might be useful. That still doesn't negate the fact that SAM files are being created and need to be converted to BAM files. Right now, I think I can live with sacrificing a little time for a single-threaded sort than for losing disk space from SAM files unnecessarily. Ryan On 4/5/11 1:18 PM, Assaf Gordon wrote: Hello Ryan, I'm in the exact same situation with my bowtie/tophat tools, going back and forth between outputing a SAM, sorted SAM, BAM or sorted BAM, and I'm still not sure what's the best method. Storage wise - you're correct, just saving the sorted BAM is the best (even more with the fact the processing SAM files as text is so horrendous that I think alnost no tool uses them directly, always requiring intervals or sorted BAM). But one annoyance (for me) is that samtools (the program) is very in-efficient - using only a single thread (and the sort part isn't doing a great job at that). So if I give the mapping tool as a whole 20 threads or more, and a part of the running time (the samtools sort part) is only using a single-thread - I'm wasting the other threads, as they sit idle waiting for the sort to finish. I also tried sorting the SAM file directly, using GNU sort (version 8.10 can use multiple threads, and the memory management actually works, as opposed to samtools sort -m) - but I'm not sure it's worth the effort. I didn't find an optimal solution that I like, and I'm interested to hear what others think. -gordon Ryan Golhar wrote, On 04/05/2011 01:08 PM: Hi all - I find it redundant to hold on to SAM output from NGS Mapping tools such when I end up converting the SAM files to BAM files anyway. The cleanup scripts require the history items to be deleted, but I don't want to delete them yet as I want the entire workflow to be kept until we are done analyzing our data. So, I was thinking of a way to remove the intermediate SAM files and thought how I would do this on the command line...simply pipe the output of BWA to samtools to create a BAM file and never have a SAM file to deal with. The BWA tool runner can be modified to pipe BWA output directly to samtools so a SAM file is never physical stored on disk. Has anyone done this? Does this seem like a good idea? Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information. attachment: golharam.vcf___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] samtools sam-to-bam problem
I've performed an alignment using BWA on a file of paired-end illumina reads. The SAM file looks fine, and contains header information. I'm converting it to BAM using the sam to bam converter, however it consistently errors out after running for a while. The error is: Error extracting alignments from (/home/galaxy/galaxy-dist/database/files/000/dataset_785.dat), but no error is provided. Looking at the sam_to_bam.py on line 156 is where the error is thrown. Nothing is in e (I think). BTW - If I run the samtools command from the shell by hand, the BAM file is created properly. I do see information on stderr: $ samtools view -bt /data/genomes/H_sapiens/hg19/hg19.fa.fai -o /tmp/killme.bam /home/galaxy/galaxy-dist/database/files/000/dataset_785.dat [samopen] SAM header is present: 25 sequences. I'm using samtools version 0.1.14 (r933:170) on Linux, 64-bit. What do I do? Ryan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Ratings on Tool Shed broken?
Hi Peter, I get the same error here. Florent On 06/04/11 01:50, Peter Cock wrote: Hi all, I had wondered why none of the tools on the Galaxy Tool Shed had a rating, and had assumed it was just that so far no-one had bothered. I have a new theory, it doesn't work? I just viewed Brad's BAM to BigWig tool, clicked on Tool Actions then Rate Tool, and got: Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) Can anyone reproduce this? I've tried a couple of other tools and the same thing happens. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
