Thanks,  greatly appreciated!

Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, 
Inc.
(650) 428-6023 office | (408) 605-3938 mobile
bcr...@completegenomics.com<mailto:bcr...@completegenomics.com>


From: Scott McManus <scottmcma...@gatech.edu<mailto:scottmcma...@gatech.edu>>
Date: Friday, August 17, 2012 11:02 AM
To: Birgit Crain 
<bcr...@completegenomics.com<mailto:bcr...@completegenomics.com>>
Cc: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" 
<galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>>
Subject: Re: [galaxy-dev] code example for improved error handling

I'm writing documentation now - I'll have something this afternoon.
Sorry for the delay.

-Scott

________________________________
No, I call the executable directly from the xml. It kept failing although it 
seemed to finish the job and I realized that on success the executable prints a 
summary message to stderror, because it's using stdout for the actual results 
so they can be pipped to another command.
>From the July news brief I understood that you can now add code to the xml to 
>check the stdout and std error for messages and I was wondering if you have 
>any documentation or sample code for that in any published tool.

Thanks

Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, 
Inc.
(650) 428-6023 office | (408) 605-3938 mobile
bcr...@completegenomics.com<mailto:bcr...@completegenomics.com>


From: John Patterson <jmpa...@g.uky.edu<mailto:jmpa...@g.uky.edu>>
Date: Thursday, August 16, 2012 12:29 PM
To: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" 
<galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>>
Subject: Re: [galaxy-dev] code example for improved error handling

On 08/16/2012 03:02 PM, Birgit Crain wrote:
Hi

I'm writing a tools for an executable that writes results to stdout and reports 
success message to stderr, so it really need the improved error handling. Is 
there any sample code or documentation available?

Thanks

Birgit Crain, Ph.D. | Sr. Professional Services Scientist | Complete Genomics, 
Inc.
(650) 428-6023 office | (408) 605-3938 mobile
bcr...@completegenomics.com<mailto:bcr...@completegenomics.com>


From: Nicole Rockweiler <n.rockwei...@gmail.com<mailto:n.rockwei...@gmail.com>>
Date: Friday, July 20, 2012 10:53 AM
To: Brian Haas <bh...@broadinstitute.org<mailto:bh...@broadinstitute.org>>
Cc: "galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>" 
<galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>>
Subject: Re: [galaxy-dev] pipeline execution succeeds but galaxy shows failure

Hi Brian,

A couple of days ago, smcmanus<https://bitbucket.org/smcmanus> pushed the 
following change to the repo:

Tools can now specify their own handling of stderr and stdout regular 
expressions as well as exit code ranges.

https://bitbucket.org/galaxy/galaxy-central/issue/325/allow-tool-authors-to-decide-whether-to-use-return-codes-or-stderr-for-detecting-job

It looks like the documentation has yet to be written.


Hope this helps,
Nicole

On Fri, Jul 20, 2012 at 12:46 PM, Brad Chapman 
<chapm...@50mail.com<mailto:chapm...@50mail.com>> wrote:

Brian;

> I wrote a pipeline (xml attached) that, from what I can gather,
> succeeds, but galaxy shows it as an error and doesn't make the output
> file accessible as a new data set.

Is it possible the software is writing to standard error? Galaxy doesn't
check status codes, but rather check for stderr and assumes that output
indicates a problem. You can wrap the problematic programs with a little
script to eat up stderr and check that everything is okay:

http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr

Brad


>
>>From the server log, I can see that the command line is being
> constructed correctly, and it even indicates that it's captured the
> output, but in the display of the web browser, it just shows up in the
> error state.   The script being run exits (0) on success.  Any ideas?
>
> Here's what the output section of my xml file looks like:
>
> <outputs>
>         <data format="bam" name="coordSortedBam" 
> label="${tool.name<http://tool.name>}
> on ${on_string}: coord-sorted read alignments"
> from_work_dir="alignment/alignment.coordSorted.bam"/>
> </outputs>
>
> and here's what the server log states:
>
> galaxy.jobs.handler INFO 2012-07-20 09:52:05,240 (30) Job dispatched
> galaxy.jobs.runners.local DEBUG 2012-07-20 09:52:05,453 executing:
> alignReads.pl --target
> /Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_26.dat
> -o alignment --aligner bowtie --single
> /Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_23.dat
>   --seqType fq
>
> galaxy.jobs<http://galaxy.jobs> DEBUG 2012-07-20 09:52:16,673 finish(): Moved
> /Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/job_working_directory/000/30/alignment/alignment.coordSorted.bam
> to /Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_50.dat
> as directed by from_work_dir
>
> Again, as far as I can tell, everything worked - but the browser
> doesn't think so.
>
> I've run the exact command above on the command-line, and it exits(0)
> indicating success.
> Also, I've verified that when run through my galaxy instance, the
> galaxy-relocated output file is as expected.
>
> Many thanks for your help.   I'm still getting my feet wet with
> galaxy, reading through all the documentation and searching the
> mailing list for additional help.
>
> best regards,
>
> -brian
>
>
> --
> --
> Brian J. Haas
> Manager, Genome Annotation and Analysis, Research and Development
> The Broad Institute
> http://broad.mit.edu/~bhaas<http://broad.mit.edu/%7Ebhaas>
> <tool id="alignreads" name="alignReads" version="0.0.1">
>
>     <description>alignReads: short read alignment tool wrapper</description>
>     <requirements>
>         <requirement type="package">trinity</requirement>
>     </requirements>
>     <command>
>
>         alignReads.pl --target $target -o alignment --aligner 
> $aligner_selection.aligner
>
>
>         ## Inputs.
>         #if str($inputs.paired_or_single) == "paired":
>             --left $inputs.left_input --right $inputs.right_input
>             #if  $inputs.left_input.ext == 'fa':
>                 --seqType fa
>             #else:
>                 --seqType fq
>             #end if
>             #if str($inputs.library_type) != "None":
>                 --SS_lib_type $inputs.library_type
>             #end if
>                       --max_dist_between_pairs $inputs.max_dist_between_pairs
>         #else:
>             --single $inputs.input
>             #if  str($inputs.input.ext) == 'fa':
>                 --seqType fa
>             #else:
>                 --seqType fq
>             #end if
>             #if str($inputs.library_type) != "None":
>                 --SS_lib_type $inputs.library_type
>             #end if
>         #end if
>
>         ## Additional parameters.
>           ##if str($inputs.use_additional) == "yes":
>               ##      -- $inputs.additional_params
>         ##end if
>
>
>         ## direct to output
>           ##    > $trinity_log 2>&amp;1
>
>
>
>     </command>
>     <inputs>
>               <param format="fasta" name="target" type="data" label="target" 
> help="Fasta sequences targeted for short-read alignment"  />
>
>         <conditional name="inputs">
>                   <param name="paired_or_single" type="select" label="Paired 
> or Single-end data?">
>                 <option value="paired">Paired</option>
>                 <option value="single">Single</option>
>             </param>
>             <when value="paired">
>                 <param format="fasta,fastq" name="left_input" type="data" 
> label="Left/Forward strand reads" help=""/>
>                 <param format="fasta,fastq" name="right_input" type="data" 
> label="Right/Reverse strand reads" help=""/>
>                 <param name="library_type" type="select" 
> label="Strand-specific Library Type">
>                     <option value="None">None</option>
>                     <option value="FR">FR</option>
>                     <option value="RF">RF</option>
>                 </param>
>                 <param name="max_dist_between_pairs" type="integer" 
> value="2000" min="1" label="max_dist_between_pairs" help="Maximum length 
> expected between fragment pairs as aligned to the target, including introns 
> where relevant."/>
>
>
>             </when>
>             <when value="single">
>                 <param format="fasta,fastq" name="input" type="data" 
> label="Single-end reads" help=""/>
>                 <param name="library_type" type="select" 
> label="Strand-specific Library Type">
>                     <option value="None">None</option>
>                     <option value="F">F</option>
>                     <option value="R">R</option>
>                 </param>
>             </when>
>         </conditional>
>
>               <conditional name="aligner_selection">
>                       <param name="aligner" type="select" label="Select 
> alignment tool to run">
>                               <option value="bowtie">bowtie</option>
>                               <option value="bwa">bwa</option>
>                               <option value="blat">blat</option>
>                       </param>
>                       <when value="blat">
>                               <param name="max_intron_length" type="integer" 
> value="10000" min = "1" label="maximum intron length" help="" />
>                               <param name="min_percent_identity" 
> type="integer" value="95" min="1" label="minimum percent identity" help="" />
>                       </when>
>                       <when value="bwa">
>                       </when>
>                       <when value="bowtie">
>                       </when>
>               </conditional>
>
>
>       <!--
>               <conditional name="use_additional_params">
>             <param name="use_additional" type="select" label="Use Additional 
> Params?">
>                 <option value="no">No</option>
>                 <option value="yes">Yes</option>
>             </param>
>             <when value="no">
>             </when>
>             <when value="yes">
>                               <param name="additional_params" type="text" 
> value="" label="Additional command-line parameters to aligner" help="" />
>             </when>
>         </conditional>
>
>       -->
>
>     </inputs>
>     <outputs>
>         <data format="bam" name="coordSortedBam" 
> label="${tool.name<http://tool.name>} on ${on_string}: coord-sorted read 
> alignments" from_work_dir="alignment/alignment.coordSorted.bam"/>
>       <!--        <data format="bam" name="nameSortedBam" 
> label="${tool.name<http://tool.name>} on ${on_string}: name-sorted read 
> alignments" from_work_dir="alignment/alignment.nameSorted.bam"/> -->
>       </outputs>
>     <tests>
>     </tests>
>     <help>
>         .. _Trinity: http://trinityrnaseq.sourceforge.net
>     </help>
> </tool>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>   http://lists.bx.psu.edu/
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/



--
Nicole Rockweiler
Genome Technology Access Center
Washington University in St. Louis
Campus Box 8510
4444 Forest Park Avenue
Saint Louis, MO 63108

________________________________

The contents of this e-mail and any attachments are confidential and only for 
use by the intended recipient. Any unauthorized use, distribution or copying of 
this message is strictly prohibited. If you are not the intended recipient 
please inform the sender immediately by reply e-mail and delete this message 
from your system. Thank you for your co-operation.


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Hi

Let me get this straight first. Are you writing python scripts that pass 
commands to this executable? If so, you are doing something similar to my tool. 
In this case, all prints to stderr will cause galaxy to report a failed run 
even if the run is successful.

Set up the command and then make a sub process call with try/except:
Need to import errno.

try:
    retcode = subprocess.call(dividereads_cmd, stderr=subprocess.STDOUT, 
stdout=splice_log)
except OSError, o:
        if o.errno == errno.ENOTDIR or o.errno == errno.ENOENT:
            print >> sys.stdout, fail_str, "Error: /insert name here/ not found 
on this system"
        exit(1)

Here is some documentation on errno: http://docs.python.org/library/errno.html

The important thing is to redirect the stderr from the executable with the 
subprocess call. I'd suggest printing your success messages and or debugging to 
a log file.

-John

________________________________

The contents of this e-mail and any attachments are confidential and only for 
use by the intended recipient. Any unauthorized use, distribution or copying of 
this message is strictly prohibited. If you are not the intended recipient 
please inform the sender immediately by reply e-mail and delete this message 
from your system. Thank you for your co-operation.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


________________________________

The contents of this e-mail and any attachments are confidential and only for 
use by the intended recipient. Any unauthorized use, distribution or copying of 
this message is strictly prohibited. If you are not the intended recipient 
please inform the sender immediately by reply e-mail and delete this message 
from your system. Thank you for your co-operation.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Reply via email to