Hello Jim,

I've implemented support for proprietary datatypes that use class modules 
included in tool shed repositories.  To see how this works, you'll need at 
least change set revision 6479:4d131422777f, which is currently available only 
from our central repo at https://bitbucket.org/galaxy/galaxy-central.

I've documented the way this works in the following 2 sections of the tool shed 
wiki.  In the second section, I've taken the liberty of using your gmap tool 
repository as an example.  i hope you don't mind.  I've written the document 
section assuming that your gmap repository includes the 2 changes I've 
described below.


There are 2 categories of datatypes that are currently supported:

1. data types that subclass from the datatype classes included in the Galaxy 
distribution - these require no code files that define proprietary datatype 
classes to be included in the tool shed repository, and are documented in the 
first wiki section listed above.

2. datatypes that use proprietary classes defined in code files included in the 
tool shed repository - documented in the second wiki section listed above.  
Your gmap tool suite falls into this category.

If you make the following changes to your gmap tool suite, your proprietary 
data types will automatically load into a local Galaxy instance when the Galaxy 
admin installs your tool suite to that instance.  The data types will be loaded 
at the time of installation as well as whenever the Galaxy server is stopped / 
restarted.  I'll send you a separate message detailing the changes you'll need 
to make to your mothur tool suite.

Add a file named datatypes_conf.xml to your repository.  This is the approach 
I'm using to support proprietary datatypes included in tool shed repositories 
instead f your proposed addition of datatypes in the tool config's 
<requirements> tag set.  The datatypes_conf.xml file can be located anywhere in 
the repository, but the the obvious location for your gmap repository is your 
~/tool-data directory.

This file should contain the following datatype definitions.

<?xml version="1.0"?>
        <datatype_file name="gmap.py"/>
        <datatype extension="gmapdb" type="galaxy.datatypes.gmap:GmapDB" 
        <datatype extension="gmapsnpindex" 
type="galaxy.datatypes.gmap:GmapSnpIndex" display_in_upload="False"/>
        <datatype extension="iit" 
type="galaxy.datatypes.gmap:IntervalIndexTree" display_in_upload="True"/>
        <datatype extension="splicesites.iit" 
        <datatype extension="introns.iit" 
type="galaxy.datatypes.gmap:IntronsIntervalIndexTree" display_in_upload="True"/>
        <datatype extension="snps.iit" 
type="galaxy.datatypes.gmap:SNPsIntervalIndexTree" display_in_upload="True"/>
        <datatype extension="gmap_annotation" 
type="galaxy.datatypes.gmap:IntervalAnnotation" display_in_upload="False"/>
        <datatype extension="gmap_splicesites" 
type="galaxy.datatypes.gmap:SpliceSiteAnnotation" display_in_upload="True"/>
        <datatype extension="gmap_introns" 
type="galaxy.datatypes.gmap:IntronAnnotation" display_in_upload="True"/>
        <datatype extension="gmap_snps" 
type="galaxy.datatypes.gmap:SNPAnnotation" display_in_upload="True"/>
        <sniffer type="galaxy.datatypes.gmap:IntervalAnnotation"/>
        <sniffer type="galaxy.datatypes.gmap:SpliceSiteAnnotation"/>
        <sniffer type="galaxy.datatypes.gmap:IntronAnnotation"/>
        <sniffer type="galaxy.datatypes.gmap:SNPAnnotation"/>

I noticed that your README in your current gmap repository on the main Galaxy 
tool shed includes the following datatype definitions, but they refer to 
classes that are not included in your repository so I've eliminated them from 
the above datatypes_conf.xml file.  You may need to add the classes to your 
current gmap.py datatypes class file and add them to the above 
datatypes_conf.xml file if your tools actually require them.

<datatype extension="tally.iit" 
type="galaxy.datatypes.gmap:TallyIntervalIndexTree"  display_in_upload="True"/>
<datatype extension="gsnap_tally" type="galaxy.datatypes.gmap:TallyAnnotation"  
<datatype extension="gsnap" type="galaxy.datatypes.gmap:GsnapResult"  

Modules that include proprietary datatype class definitions cannot use relative 
import references for imported modules.  Imports must be defined as absolute 
from the galaxy subdirectory inside the Galaxy root's lib subdirectory.  So for 
your ~/lib/galaxy/datatypes/gmap.py datatypes module in your gmap repository, 
the following changes are necessary.

Your current imports look like this:

import logging
import os,os.path,re
import data
from data import Text
from galaxy import util
from metadata import MetadataElement

But they need to be changed to this - note the elimination of relative imports:

import logging
import os,os.path,re
import galaxy.datatypes.data
from galaxy.datatypes.data import Text
from galaxy import util
from galaxy.datatypes.metadata import MetadataElement

Thanks very much for helping out with this, and please let me know if you bump 
into any problems.

Greg Von Kuster

On Oct 21, 2011, at 1:13 PM, Jim Johnson wrote:

> Greg,
> I put the gmap tool suite in the galaxy Tool Shed,  let me know if there is 
> more I should do.  
> It has 5 galaxy tools:
>     GMAP   -  Genomic Mapping and Alignment Program for mRNA and EST 
> sequences 
>     GSNAP    - Genomic Short-read Nucleotide Alignment Program       
>     GMAP Build    -  a database genome index for GMAP and GSNAP     ( calls:  
> gmap_build, iit_store, snpindex, cmetindex, atoiindex ) 
>     GMAP SNP Index    - build index files for known SNPs                     
> (calls:  iit_store, snpindex) 
>     GMAP IIT    - Create a map store for known genes or SNPs              
> (calls:  iit_store) 
> It uses these added datatypes:
> % grep -E '(^class | file_ext)' lib/galaxy/datatypes/gmap.py 
> class GmapDB( Text ):
>     file_ext = 'gmapdb'
> class GmapSnpIndex( Text ):
>     file_ext = 'gmapsnpindex'
> class IntervalIndexTree( Text ):
>     file_ext = 'iit'
> class SpliceSitesIntervalIndexTree( IntervalIndexTree ):
>     file_ext = 'splicesites.iit'
> class IntronsIntervalIndexTree( IntervalIndexTree ):
>     file_ext = 'introns.iit'
> class SNPsIntervalIndexTree( IntervalIndexTree ):
>     file_ext = 'snps.iit'
> class IntervalAnnotation( Text ):
>     file_ext = 'gmap_annotation'
> class SpliceSiteAnnotation(IntervalAnnotation):
>     file_ext = 'gmap_splicesites'
> class IntronAnnotation(IntervalAnnotation):
>     file_ext = 'gmap_introns'
> class SNPAnnotation(IntervalAnnotation):
>     file_ext = 'gmap_snps'
> I added a requirement tag for the datatypes to the tool-configs:
> % grep 'requirement.*datatype' *.xml
> gmap_build.xml:      <requirement type="datatype">gmapdb</requirement>
> gmap_build.xml:      <requirement type="datatype">gmap_snps</requirement>
> gmap.xml:    <requirement type="datatype">gmapdb</requirement>
> gmap.xml:    <requirement type="datatype">gmap_annotation</requirement>
> gmap.xml:    <requirement type="datatype">gmap_splicesites</requirement>
> gmap.xml:    <requirement type="datatype">gmap_introns</requirement>
> gmap.xml:    <requirement type="datatype">gmap_snps</requirement>
> gsnap.xml:      <requirement type="datatype">gmapdb</requirement>
> gsnap.xml:      <requirement type="datatype">gmapsnpindex</requirement>
> gsnap.xml:      <requirement type="datatype">splicesites.iit</requirement>
> gsnap.xml:      <requirement type="datatype">introns.iit</requirement>
> iit_store.xml:      <requirement type="datatype">gmap_annotation</requirement>
> iit_store.xml:      <requirement type="datatype">gmap_snps</requirement>
> iit_store.xml:      <requirement type="datatype">iit</requirement>
> iit_store.xml:      <requirement type="datatype">splicesites.iit</requirement>
> iit_store.xml:      <requirement type="datatype">introns.iit</requirement>
> iit_store.xml:      <requirement type="datatype">snps.iit</requirement>
> snpindex.xml:      <requirement type="datatype">gmapsnpindex</requirement>
> snpindex.xml:      <requirement type="datatype">gmapdb</requirement>
> snpindex.xml:      <requirement type="datatype">gmap_snps</requirement>
> snpindex.xml:      <requirement type="datatype">snps.iit</requirement>
> Thanks,
> JJ

Greg Von Kuster
Galaxy Development Team

Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:


Reply via email to