Maybe my message was not understandable enough. I really need your help, so 
I'll try to be more concise :

How do I make a composite output from 2 datatypes that I have declared myself ?
I've followed the "Composite Datatypes" wiki but it seems that I've missed 
My composite datatype appears well in "file format" from Get Data's upload file 
section, but when I run my tool, I have 2 outputs which are the components of 
my primary datatype, instead of only one output.

Best regards,



 De : Marine Rohmer <marine.roh...@yahoo.fr>
À : "galaxy-dev@lists.bx.psu.edu" <galaxy-dev@lists.bx.psu.edu> 
Envoyé le : Vendredi 8 juin 2012 15h15
Objet : Composite output with self-declarated datatype

Hi everyone,

I'm trying to add a tool which generates 2 files, that I will call ".xxx" (a 
text file) and ".yyy" (a binary file)  . Both files are needed to use the 
result of my tool with an other tool I've added. 
So I wanted to create a composite datatype , that I will call ".composite", 
whose components are ".xxx" and ".yyy".

I've declared the datatype ".xxx", ".yyy" and ".composite" in the 
datatypes_conf.xml file, and written the required python files . Now, ".xxx", 
".yyy" and ".composite" appear in Get Data's "file format" .

These are my files :

In datatype_conf.xml :

<datatype extension="xxx" type="galaxy.datatypes.xxx:xxx" mimetype="text/html" 
display_in_upload = "True"
    <datatype extension="yyy" type="galaxy.datatypes.yyy:yyy" 
mimetype="application/octet-stream" display_in_upload = "True" subclass="True" 
    <datatype extension="composite" type="galaxy.datatypes.composite:Composite" 
mimetype="text/html" display_in_upload="True"/>

xxx.py (summarized) :

import logging
from metadata import MetadataElement
from data import Text

log = logging.getLogger(__name__)

class xxx(Text):      
    file_ext = "xxx"

        def __init__( self, **kwd ):
        Text.__init__( self, **kwd )

yyy.py (summarized) :  

import logging
from metadata import
from data import Text

log = logging.getLogger(__name__)

# yyy is a binary file, don't know what to put instead of "Text". "Binary" and 
"Bin" don't work.
class yyy(Text):     
    file_ext = "yyy"

        def __init__( self, **kwd ):
        Text.__init__( self, **kwd )

composite.py (summarized) :

import logging
from metadata import MetadataElement
from data import Text

log = logging.getLogger(__name__)

class Composite(Text):      
    composite_type = 'auto_primary_file'
    MetadataElement( name="base_name", desc="base name for all transformed 
versions of this index dataset", default="your_index", readonly=True,
    file_ext = 'composite'

    def __init__( self, **kwd ):
        Text.__init__( self, **kwd )
        self.add_composite_file( '%s.xxx', description = "XXX file", 
substitute_name_with_metadata = 'base_name')
        self.add_composite_file( '%s.yyy', description = "YYY file", 
substitute_name_with_metadata = 'base_name', is_binary = True )

Atfer having read Composite Datatypes in the wiki, my myTool.xml looks like :

<tool id="my tool">
   <command> path/to/crac-index-wrapper.sh 
   ${os.path.join( $output_name_yyy.extra_files_path, '%s.yyy')} 
${os.path.join( $output_name_xxx.extra_files_path, '%s.xxx' )} $input_file
      <param name="output_name" type="text" value ="IndexOutput" label="Output 
      <param name="input_file" type="data" label="Source file" format="fasta"/>
      <data format="ssa" name="output_name_ssa" 
from_work_dir="crac-index_output.ssa" label="CRAC-index: ${output_name}.ssa">
    <data format="conf" name="output_name_conf" 
from_work_dir="crac-index_output.conf" label="CRAC-index: ${output_name}.conf">

I have 2 main problems  :

When I upload a xxx file via "Get Data", there's no problem. However, when I 
upload a yyy
 file (the binary one),history bloc rests eternally blue ("uploading dataset") 
, even for a small file. 

The second problem is that I want my tool to only generate the .composite file 
on the history, and not each of .xxx and .yyy. 
. But when I run my tool I still have 2 outputs displayed in the history : one 
for xxx and one for yyy. Furthermore, neither of them work, and I have the 
following message :

path/to/myTool-wrapper.sh: 6: path/to/myTool-wrapper.sh.sh: cannot create 
Directory nonexistent
path/to/myTool-wrapper.sh: 6: path/to/myTool-wrapper.sh: cannot create 
Directory nonexistent
path/to/myTool-wrapper.sh: 11: path/to/myTool-wrapper.sh: Syntax error: 
redirection unexpected

So I've checked manually in /home/myName/work/galaxy-dist/database/files/000/ 
and there's only "dataset_302.dat", an empty file.
(And whatsmore, I don't understand why I get in the message "%s.yyy.xxx" and 
"%s.yyy.yyy" instead of "%s.yyy" and "%s.xxx" ...)

Then I've looked the example of rgenetics.xml, and tried to change the command 
line and the output :
<tool id="my tool">
   <command> path/to/myTool-wrapper.sh 
'$output_name.extra_files_path/$output_name.metadata.base_name' $input_file
      <param name="output_name" type="text" value ="IndexOutput" label="Output 
name" />
      <param name="input_file" type="data" label="Source file"
      <data format="html" name="output" label="myTool: ${output_name}.html" 

This gave me :

Traceback (most recent call last):
  File "/home/myName/work/galaxy-dist/lib/galaxy/jobs/runners/local.py", line 
59, in run_job
  File "/home/myName/work/galaxy-dist/lib/galaxy/jobs/__init__.py", line 429, 
in prepare
    self.command_line = self.tool.build_command_line( param_dict )
  File "/home/myName/work/galaxy-dist/lib/galaxy/tools/__init__.py", line 1971, 
in build_command_line
    command_line = fill_template( self.command, context=param_dict )
  File "/home/myName/work/galaxy-dist/lib/galaxy/util/template.py", line 9, in
    return str( Template( source=template_text, searchList=[context] ) )
 line 1004, in __str__
    return getattr(self, mainMethName)()
  File "cheetah_DynamicallyCompiledCheetahTemplate_1339157051_58_87978.py", 
line 83, in respond
NotFound: cannot find 'extra_files_path' while searching for 

So now I don't know which way is the one to follow : the first one inspired by 
the example in the wiki, or the second one inspired by rgenetics.xml. And 
what's wrong with it... 
I will really appreciate any suggestion !

Best regards,

Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:


Reply via email to