Dannon,

Thanks for agreeing to have a look at this. Configs and traceback are
attached.

Just to clarify, these tools work without error when run individually,
and it's only when we try and generate a workflow that problems occur.

Cheers

Steve

-----Original Message-----
From: Dannon Baker [mailto:dannonba...@me.com] 
Sent: 21 November 2011 15:28
To: Steven Platt
Cc: galaxy-dev@lists.bx.psu.edu; Paul-Michael Agapow
Subject: Re: [galaxy-dev] Tools appearing without inputs in workflow
editor

Could you send me the traceback and perhaps a copy of the misbehaving
tool config files?  I'll be happy to take a look.

Thanks!

Dannon


On Nov 21, 2011, at 9:53 AM, Steven Platt wrote:

> A colleague of mine posted this on the Users list ... 4 days later and
> no replies! I'm hoping that one of the Devs can shed a bit of light
>
------------------------------------------------------------------------
> ---------------------------------------------------
> A peculiar bug, presented for your input:
> 
> A colleague has built a few tools and is trying to incorporate them
into
> a workflow. However:
> 
> 1.       When he creates an new empty workflow, edits it and adds the
> tools, they appear without inputs (i.e. the icons/boxes on the diagram
> have no input "sockets")
> 2.       When he tries to save a history using these tools as a
> workflow, a error results in which Galaxy complains about an
> unrecognised name, being one of the tool inputs. Depending on which
tool
> he includes in the workflow, the missing name occurs in whichever tool
> is earlier in the workflow. (Traceback available on request)
> 
> Puzzling. And even more puzzling when I tried to debug this:
> 1.       Other tools (not written by him) behave in the workflows as
> expected.
> 2.       Pulling a copy of his config file across to my own instance
of
> Galaxy and creating a dummy tool to house it, it worked perfectly 
> 3.       We're running the same version of Galaxy, albeit different
> versions of Python (2.5 vs 2.6.4)
> 4.       The tools work fine in normal analysis mode.
> 5.       Line-ending unix, XML seems to parse fine. 
> 
> I'm stymied here and unsure what to look at next. Ideas?
>
------------------------------------------------------------------------
> ---------------------------------------------------
> 
> Thanks
> 
> Steve
> 
> Dr Steven Platt
> Bioinformatician
> Health Protection Agency
> Microbiology Services Division
> London
> www.hpa.org.uk/bioinformatics 
> -----------------------------------------
>
************************************************************************
**
> The information contained in the EMail and any attachments is
> confidential and intended solely and for the attention and use of
> the named addressee(s). It may not be disclosed to any other person
> without the express authority of the HPA, or the intended
> recipient, or both. If you are not the intended recipient, you must
> not disclose, copy, distribute or retain this message or any part
> of it. This footnote also confirms that this EMail has been swept
> for computer viruses, but please re-sweep any attachments before
> opening or saving. HTTP://www.HPA.org.uk
>
************************************************************************
**
> 
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> 
>  http://lists.bx.psu.edu/



-----------------------------------------
**************************************************************************
The information contained in the EMail and any attachments is
confidential and intended solely and for the attention and use of
the named addressee(s). It may not be disclosed to any other person
without the express authority of the HPA, or the intended
recipient, or both. If you are not the intended recipient, you must
not disclose, copy, distribute or retain this message or any part
of it. This footnote also confirms that this EMail has been swept
for computer viruses, but please re-sweep any attachments before
opening or saving. HTTP://www.HPA.org.uk
**************************************************************************
URL: http://xxx.xxx.xxx.xxx:6050/workflow/build_from_current_history
File 
'/home/f0/steve/galaxy/eggs/WebError-0.8a-py2.5.egg/weberror/evalexception/middleware.py',
 line 364 in respond
  app_iter = self.application(environ, detect_start_response)
File '/usr/lib/python2.5/site-packages/paste/debug/prints.py', line 98 in 
__call__
  environ, self.app)
File '/usr/lib/python2.5/site-packages/paste/wsgilib.py', line 539 in 
intercept_output
  app_iter = application(environ, replacement_start_response)
File '/usr/lib/python2.5/site-packages/paste/recursive.py', line 80 in __call__
  return self.application(environ, start_response)
File '/usr/lib/python2.5/site-packages/paste/httpexceptions.py', line 632 in 
__call__
  return self.application(environ, start_response)
File '/home/f0/steve/galaxy/lib/galaxy/web/framework/base.py', line 145 in 
__call__
  body = method( trans, **kwargs )
File '/home/f0/steve/galaxy/lib/galaxy/web/controllers/workflow.py', line 1188 
in build_from_current_history
  associations = cleanup_param_values( tool.inputs, param_values )
File '/home/f0/steve/galaxy/lib/galaxy/web/controllers/workflow.py', line 1819 
in cleanup_param_values
  cleanup( "", inputs, values )
File '/home/f0/steve/galaxy/lib/galaxy/web/controllers/workflow.py', line 1794 
in cleanup
  tmp = values[key]
KeyError: 'file1'
<tool id="remove_adapters_and_normalise" name="Clean and Normalise FASTQ" 
force_history_refresh="True">
  <description>Filter and trim FASTQ files ensuring consistent read 
pairing</description>
  <command interpreter="ruby">remove_adapters_and_normalise.rb 
  $output
  $output.id
  $__new_file_path__
  $read_pattern1 
  $read_pattern2
  #if $data.data_source_select == "predefined"
        $GALAXY_DATA_INDEX_DIR/NGS/truseq_adapters.fasta
  #else if $data.data_source_select == "user"
        $data.NGS_adapters
  #else
        no_adapters
  #end if
  #if $options.advanced == "yes"
        $options.filter_quality
        $options.filter_threshold
        $options.trim_quality
        $options.trim_fraction
  #else
        20
        90
        20
        95
  #end if
  <!-- $userEmail
  #for $i in $input_files
        ${i.file}
        "${i.file.name}"
  #end for -->
  $file1
  $file1.name
  $file2
  $file2.name
  </command>
  <inputs>
    <param name="file1"  type="data" format="fasta,fastq" label="File1"/>
    <param name="file2"  type="data" format="fasta,fastq" label="File2"/>
    <param name="read_pattern1" type="text" size="15" value="_1" label="Pattern 
to identify file(s) of forward reads" help="Example: _1"/>
    <param name="read_pattern2" type="text" size="15" value="_2" label="Pattern 
to identify file(s) of reverse reads" help="Example: _2"/>
    <conditional name="data">
      <param name="data_source_select" type="select" label="NGS adapter 
sequences: Predefined, User uploaded or None" help="If adapters are unknown 
leave as predefined">
          <option value="predefined">Predefined NGS adapter sequences</option>
          <option value="user">Your uploaded NGS adapter sequences</option>
          <option value="none">No adapter sequences to be removed</option>
      </param>
      <when value="predefined">
          <param name="species" type="select" label="Predefined adapter sets">
              <options from_file="NGS_adapters.loc">
                <column name="name" index="0"/>
                <column name="value" index="1"/>
              </options>
          </param>
      </when>
      <when value="user">
          <param name="NGS_adapters" type="data" format="fasta"
                label="NGS adapters in fasta format"
                help="A multifasta file where the adapter names are included 
with headers."
          />
      </when>
      <when value="none"/>
    </conditional>
    <conditional name="options">
          <param name="advanced" type="select" label="Filtering/Trimming 
Advanced Options">
               <option value="no">Use Defaults</option>
               <option value="yes">Set Filtering/Trimming Values</option>
          </param>
          <when value="no"/>
          <when value="yes">
                  <param name="filter_quality" type="select" format="text " 
help="Default: 20">
                        <label>Filtering: Phred score quality cut-off</label>
                        <option value="10">10</option>
                        <option value="20" selected="true">20</option>
                        <option value="30">30</option>
                        <option value="40">40</option>
                  </param>
                  <param name="filter_threshold" type="select" format="text " 
help="Default: 90">
                        <label>Filtering: Percentage of bases in sequence that 
must have quality greater than or equal to the cut-off</label>
                        <option value="50">50</option>
                        <option value="60">60</option>
                        <option value="70">70</option>
                        <option value="80">80</option>
                        <option value="90" selected="true">90</option>
                        <option value="100">100</option>
                  </param>
                  <param name="trim_quality" type="select" format="text " 
help="Default: 20">
                        <label>Trimming: Phred score quality cut-off</label>
                        <option value="10">10</option>
                        <option value="20" selected="true">20</option>
                        <option value="30">30</option>
                        <option value="40">40</option>
                  </param>
                  <param name="trim_fraction" type="select" format="text " 
help="Default: 95">
                        <label>Trimming: Percentage of consecutive bases in 
sequence that must have quality greater than or equal to the cut-off after 
trimming</label>
                        <option value="70">70</option>
                        <option value="75">75</option>
                        <option value="80">80</option>
                        <option value="85">85</option>
                        <option value="90">90</option>
                        <option value="95" selected="true">95</option>
                        <option value="100">100</option>
                  </param>
          </when>
      </conditional>
<!--    <conditional name="email_on_completion">
      <param name="send_email" type="select" label="Send email when job is 
complete?">
        <option value="yes">Yes</option>
        <option value="no">No</option>
      </param>
      <when value="no"/>
      <when value="yes">
        <param name="address" type="text" value="$userEmail" label="Email 
address"/>
      </when>
    </conditional> -->
  </inputs>
  <outputs>
    <data format="input" name="output" />
  </outputs>

  <tests>
    <test>
      <param name="input" value="fa_gc_content_input.fa"/>
      <output name="out_file1" file="fa_gc_content_output.txt"/>
    </test>
  </tests>

  <help>
This tools performs several actions on paired Next Generation Sequence (NGS) 
data.

Input Data
----------

* One or more pairs of NGS reads. The pairs can be specified in any order. 

* One pattern that is specific to the 1st read of all pairs, and a second 
pattern that is specific to the 2nd read of all pairs.

.. class:: warningmark  

Assumes pairs of NGS files are identically named apart from the pattern 
specifed.

* Adapter sequences to be screened from the reads. If non-standard adapters are 
required users can select an appropriate FASTA file from their History.

Actions
-------

* If adapters have been specified these short sequences will be removed from 
the NGS reads, cleaning up the data. This cleanup can take a significant amount 
of time, up 1 hour per pair of reads.

* Reads are filtered for quality (Phred score >= 20 for >90% of bases in a 
read). Reads that fail are discarded.

* Reads are trimmed based on quality (Phred score >= 20 across consecutive 95% 
of read length). Where base quality at the end of the read falls below this 
value the base is trimmed, resulting in a shorter read. Where trimming shortens 
the read to less than the specified length, the read is discarded. 

* The two files are compared to ensure that only pairs of reads are retained. 
Unpaired orphan reads are discarded.

* If the read file is >3Gb in size it is shrunk so that 15 million reads are 
retained.

* The pair of read files are interleaved into a single file, containing both 
forward and reverse reads.

* Finally, the interleaved file is returned to the current History along with a 
report on the manipulations applied to all NGS reads.
  </help>

</tool>
<tool id="velvet_assembly" name="Velvet" force_history_refresh="True">
  <description>NGS Assembly using Velveth and Velvetg</description>
  <command interpreter="perl">velvet_pre_script3.pl 
  $output
  $output.id
  $__new_file_path__
  #if $reads.sspace.run_sspace == "yes"
        $reads.sspace.insert_variation
        $reads.sspace.pair_type
  #end if
  #if $reads.sspace.run_sspace == "no"
        void
        void
  #end if
  -s $kmin 
  -e $kmax 
  -k $optFuncKmer 
  -f '
  #for $i in $inputs
        ${i.file_format}
        ${i.read_type}
        ${i.input}
  #end for
  '
  #if $contig_lgth.use_contig_lgth == "yes"
        -o '
        -min_contig_lgth $contig_lgth.min_contig_lgth
  #end if
  #if $reads.paired == "yes":
       #if $contig_lgth.use_contig_lgth == "no"
                -o '
       #end if
       #if int(str($reads.ins_length)) > 0:
           -ins_length $reads.ins_length
       #end if
    #if $reads.options.advanced == "yes":
           #if int(str($reads.options.ins_length_sd)) > 0:
               -ins_length_sd $reads.options.ins_length_sd
           #end if
           #if int(str($reads.options.ins_length2)) > 0:
               -ins_length2 $reads.options.ins_length2
           #end if
           #if int(str($reads.options.ins_length2_sd)) > 0:
               -ins_length2_sd $reads.options.ins_length2_sd
           #end if
           #if int(str($reads.options.ins_length_long)) > 0:
               -ins_length_long $reads.options.ins_length_long
           #end if
           #if int(str($reads.options.ins_length_long_sd)) > 0:
               -ins_length_long_sd $reads.options.ins_length_long_sd
           #end if
           #if int(str($reads.options.max_branch_length)) > 0:
               -max_branch_length $reads.options.max_branch_length
           #end if
<!--       #if int($reads.options.max_divergence) > 0:
               -max_divergence $reads.options.max_divergence
           #end if-->
           #if int(str($reads.options.max_gap_count)) > 0:
               -max_gap_count $reads.options.max_gap_count
           #end if
           #if int(str($reads.options.min_pair_count)) > 0:
               -min_pair_count $reads.options.min_pair_count
           #end if
<!--       #if int($reads.options.max_coverage) > 0:
               -max_coverage $reads.options.max_coverage
           #end if-->
           #if int(str($reads.options.long_mult_cutoff)) > 0:
               -long_mult_cutoff $reads.options.long_mult_cutoff
           #end if
           $reads.options.scaffolding
     #end if
  #end if
  #if $contig_lgth.use_contig_lgth == "yes" or $reads.paired == "yes":
        '
  #end if

  </command>
  <inputs>
    <param type="integer" name="kmin" label="Minimum kmer" help="Must be an ODD 
number (11-51)" min="11" max="51" value="29"/>
    <param type="integer" name="kmax" label="Maximum kmer" help="Must be an ODD 
number (21-71)" min="21" max="71" value="45"/>
    <param label="Kmer optimisation determinant" name="optFuncKmer" 
type="select" help="Which feature of the assembly is maximised (minimised for 
'Number of [long] contigs') to determine optimal assembly">
        <option value="ncon">Total number of contigs (ncon)</option>
        <option value="tbp">Total number of base pairs in contigs (tbp)</option>
        <option value="max">Length of the longest contig (max)</option>
        <option value="Lcon">Number of large (>1kb) contigs (Lcon)</option>
        <option value="Lbp">Total number of base pairs in large contigs 
(Lbp)</option>
        <option value="n50" selected="yes">The N50 (n50)</option>
        <option value="n50+max">The N50 + the length of the longest contig 
(n50+max)</option>
    </param>
    <repeat name="inputs" title="Input Files" min="1">
              <param label="file format" name="file_format" type="select">
                      <option value="-fasta" selected="yes">fasta</option>
                      <option value="-fastq">fastq</option>
              </param>
              <param label="read type" name="read_type" type="select">
                      <option value="-short" selected="yes">short reads</option>
                      <option value="-shortPaired">shortPaired reads</option>
                      <option value="-short2">short2 reads</option>
                      <option value="-shortPaired2">shortPaired2 reads</option>
                      <option value="-long">long reads</option>
                      <option value="-longPaired">longPaired reads</option>
              </param>
     <param name="input" type="data" format="fasta,fastq" label="Dataset"/>
    </repeat>
    <conditional name="contig_lgth">
        <param name="use_contig_lgth" type="select" label=" Set minimum contig 
length" help="minimum contig length exported to contigs.fa file (default: hash 
length * 2).">
                <option value="no">No</option>
                <option value="yes">Yes</option>
        </param>
        <when value="no"/>
        <when value="yes">
                <param name="min_contig_lgth" value = "500" label="minimum 
contig length" type="integer" help="minimum contig length exported to 
contigs.fa file (default: hash length * 2)"/>
        </when>
    </conditional>
    <conditional name="reads">
          <param name="paired" type="select" label="Using Paired Reads">
               <option value="no">No</option>
               <option value="yes" 
selected="${input.metadata.paired_end_reads}">Yes</option>
          </param>
          <when value="no"/>
          <when value="yes">
          <param name="ins_length" value = "-1" label="Insert Length in 
Paired-End Read dataset (ignored when -1)" type="integer" help="Expected 
distance between two paired end reads"/>
      <conditional name="options">
          <param name="advanced" type="select" label="Velvet Advanced Options">
               <option value="no">Use Defaults</option>
               <option value="yes">Set Advanced Option Values</option>
          </param>
          <when value="no"/>
          <when value="yes">
                  <param name="ins_length_sd" value = "-1" label="Estimate of 
Standard Deviation of Paired-End Read dataset(ignored when -1)" type="integer" 
help="Estimate of standard deviation of Paired-End Read dataset (default: 10% 
of corresponding length)"/>
                  <param name="ins_length2" value = "-1" label="Insert Length 
in 2nd Paired-End Short Read dataset (ignored when -1)" type="integer" 
help="Expected distance between two paired end reads in the second short-read 
dataset"/>
                  <param name="ins_length2_sd" value = "-1" label="Estimate of 
Standard Deviation of 2nd Paired-End Read dataset (ignored when -1)" 
type="integer" help="Estimate of standard deviation of 2nd Paired-End Read 
dataset (default: 10% of corresponding length)"/>
                  <param name="ins_length_long" value = "-1" label="Insert 
Length in Long Paired-End Read dataset (ignored when -1)" type="integer" 
help="Expected distance between two long paired-end reads"/>
                  <param name="ins_length_long_sd" value = "-1" label="Estimate 
of Standard Deviation of 2nd Paired-End Read dataset (ignored when -1)" 
type="integer" help="Estimate of standard deviation of Long Paired-End Read 
dataset (default: 10% of corresponding length)"/>
                  <param name="max_branch_length" value = "-1" label="Maximum 
branch length (ignored when -1)" type="integer" help="maximum length in base 
pair of bubble (default: 100)"/>
<!--              <param name="max_divergence" value = "-1." label="Maximum 
max_divergence (between .0 and 1., ignored when -1.)" type="float" 
help="maximum divergence rate between two branches in a bubble (default: 
.2)"/>-->
                  <param name="max_gap_count" value = "-1" label="Maximum gap 
count (ignored when -1)" type="integer" help="maximum number of gaps allowed in 
the alignment of the two branches of a bubble (default: 3)"/>
                  <param name="min_pair_count" value = "-1" label="Minimum 
read-pair count (ignored when -1)" type="integer" help="minimum number of 
paired end connections to justify the scaffolding of two long contigs (default: 
10)"/>
<!--              <param name="max_coverage" value = "-1." label="Maximum 
coverage exclusion(ignored when -1.)" type="float" help="Exclude data that has 
coverage more than this maximum coverage value"/>-->
                  <param name="long_mult_cutoff" value = "-1" label="Minimum 
number of long reads required to merge contigs (ignored when -1)" 
type="integer" help="minimum number of long reads required to merge contigs 
(default: 2)"/>
                  <param name="scaffolding" type="boolean" checked="true" 
truevalue="-scaffolding yes" falsevalue="-scaffolding no" label="Use 
Scaffolding" help="Scaffold contigs that it cannot quite be connected (This 
results in sequences of Ns in the contigs)"/>
          </when>
      </conditional>
      <conditional name="sspace">
        <param name="run_sspace" type="select" label="Follow up with SSPACE 
scaffolding?">
                <option value="no">Do not scaffold velvet output</option>
                <option value="yes">Scaffold contigs using SSPACE</option>
        </param>
        <when value="no"/>
        <when value="yes">
                <param name="insert_variation" value="0.3" type="float" min="0" 
max="1" label="Allowed variation in insert size (0.0 - 1.0)" />
<!-- help="0.0 = inserts are always of size specified above, 1.0 insert size 
may vary by its full length e.g. inserts of 2kb could be between 0bp & 4kbp" -->
                <param name="pair_type" label="Using Mate-paired or Paired-end 
reads?" type="select">
                        <option value="1">Mate-paired reads</option>
                        <option value="0">Paired-end reads</option>
                </param>
        </when>
      </conditional>
          </when>
    </conditional>
  </inputs>
  <outputs>
    <data format="txt" name="output" />
  </outputs>

  <tests>
    <test>
      <param name="input" value="fa_gc_content_input.fa"/>
      <output name="out_file1" file="fa_gc_content_output.txt"/>
    </test>
  </tests>

  <help>
XXXXXXXXXXXXXXX This tool performs a multiple sequence alignment of the 
specified data on the Bioinformatics Condor compute cluster, 
using the parallel MPI implementation of clustalW.

.. class:: warningmark

While this tool should be able to handle most sequence formats it is 
recommended that sequences are submitted in the FASTA format.

----

The length of time it takes to complete the alignment will depend on the number 
and length of sequences submitted, 
however it will be significantly faster than most desktop PCs.

Example times:

* 50 1Kb sequences in approximately 50 seconds
* 200 1Kb sequences in approximately 2 minutes
* 600 1Kb sequences in approximately 10 minutes
* 900 0.5-1.7Kb sequences in approximately 40 minutes

Please be patient. The alignment will complete eventually.

.. class:: warningmark

The compute cluster is a finite resource and it may be that your job is delayed 
by the presence of others that are already running.
You may want to check the status of the cluster (**Cluster tools > check 
cluster status**) before running a sequence alignment.
At least 10 unclaimed nodes are required to run an alignment. Alignments 
submitted insufficient resources are available 
will be queued for execution and will run as soon as 10 nodes are free.
  </help>

</tool>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Reply via email to