Re: [galaxy-dev] No samtools build after building index through Data Manager.
Hi Dan,
Thanks for your reply.
I don't handle the updates, but it was updated on 25 Jun 13. I'm pretty
sure I can rule out the version number as the BWA builds, created using
the Data Manager, work fine.
Here is the pertinent contents of shed_tool_data_table_conf.xml
value, dbkey, name, path
testtoolshed.g2.bx.psu.edudata_manager_bwa_index_builderblankenbergfe6508204ac
c
value, dbkey, name, path
testtoolshed.g2.bx.psu.edudata_manager_sam_fa_index_builderblankenberg926e5039
7b83
line_type, value, path
testtoolshed.g2.bx.psu.edudata_manager_sam_fa_index_builderblankenberg926e5039
7b83
value, dbkey, name, path
testtoolshed.g2.bx.psu.edudata_manager_fetch_genome_all_fastablankenbergca8b37
09309e
in ${GALAXY_HOME}/tool_data_table_conf.xml.sample the entry for sam_fa
reads as so:
line_type, value, path
The file tool-data/sam_fa_new_indices.loc (and .sample) does not exist.
If I keep the manually inserted builds listed in
tool-data/sam_fa_indices.loc and restart Galaxy, then I get the following
(abridged) entries in the paster.log:
galaxy.tools.data DEBUG 2013-07-15 09:51:11,109 Loaded tool data table
'all_fasta'
galaxy.tools.data DEBUG 2013-07-15 09:51:11,115 Loaded tool data table
'bwa_indexes'
galaxy.tools.data DEBUG 2013-07-15 09:51:11,116 Loaded tool data table
'bwa_indexes_color'
galaxy.tools.data DEBUG 2013-07-15 09:51:11,167 Loaded tool data table
'sam_fa_indexes'
...
galaxy.tools.data DEBUG 2013-07-15 09:51:11,324 Loading another instance
of data table 'all_fasta', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 09:51:11,340 Loading another instance
of data table 'bwa_indexes', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 09:51:11,348 Loading another instance
of data table 'bwa_indexes_color', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 09:51:11,410 Loading another instance
of data table 'all_fasta', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 09:51:11,422 Loading another instance
of data table 'sam_fa_indexes', attempting to merge content.
galaxy.tools.data ERROR 2013-07-15 09:51:11,422 Attempted to add fields
(['index', 'cfraxinea_s1v1',
'/home/galaxy/software/galaxy-central/tool-data/cfraxinea_s1v1/sam_index/cf
raxinea_s1v1/c_fraxinea_s1v1.fa']) to data table 'sam_fa_indexes', but
this entry already exists and allow_duplicates is False.
galaxy.tools.data ERROR 2013-07-15 09:51:11,422 Attempted to add fields
(['index', 'b_distachyon',
'/home/galaxy/software/galaxy-central/tool-data/b_distachyon/sam_index/b_di
stachyon/b_distachyon.fa']) to data table 'sam_fa_indexes', but this entry
already exists and allow_duplicates is False.
galaxy.tools.data ERROR 2013-07-15 09:51:11,423 Attempted to add fields
(['index', 'n_sylvestris',
'/home/galaxy/software/galaxy-central/tool-data/n_sylvestris/sam_index/n_sy
lvestris/n_sylvestris.fa']) to data table 'sam_fa_indexes', but this entry
already exists and allow_duplicates is False.
galaxy.tools.data ERROR 2013-07-15 09:51:11,423 Attempted to add fields
(['index', 'n_tomentosiformis',
'/home/galaxy/software/galaxy-central/tool-data/n_tomentosiformis/sam_index
/n_tomentosiformis/n_tomentosiformis.fa']) to data table 'sam_fa_indexes',
but this entry already exists and allow_duplicates is False.
galaxy.tools.data DEBUG 2013-07-15 09:51:11,491 Loading another instance
of data table 'all_fasta', attempting to merge content.
The builds listed above with the 'entry already exists' error, are all the
ones which I attempted to load using the Data Manager.
If I comment out these in entries in tool-data/sam_fa_indices.loc and
restart Galaxy, the errors disappear, but when I try to use those builds
to SAM-to-BAM, I'm back to the "Sequences are not currently available for
the specified build." error.
paster.log entries after re-start:
galaxy.tools.data DEBUG 2013-07-15 10:02:56,484 Loaded tool data table
'all_fasta'
galaxy.tools.data DEBUG 2013-07-15 10:02:56,491 Loaded tool data table
'bwa_indexes'
galaxy.tools.data DEBUG 2013-07-15 10:02:56,492 Loaded tool data table
'bwa_indexes_color'
galaxy.tools.data DEBUG 2013-07-15 10:02:56,497 Loaded tool data table
'sam_fa_indexes'
...
galaxy.tools.data DEBUG 2013-07-15 10:02:56,508 Loading another instance
of data table 'all_fasta', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 10:02:56,509 Loading another instance
of data table 'bwa_indexes', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 10:02:56,510 Loading another instance
of data table 'bwa_indexes_color', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 10:02:56,512 Loading another instance
of data table 'all_fasta', attempting to merge content.
galaxy.tools.data DEBUG 2013-07-15 10:02:56,512 Loading another instance
of data table 'sam_fa_indexes', attempting to merge content.
galaxy.tools.data DEBUG 2013
Re: [galaxy-dev] No samtools build after building index through Data Manager.
Hi Graham, Which revision of Galaxy are you currently using? Currently data managers require at least 9952:a28faa6ac188 on the default branch. When installed from the tool shed, data managers use the shed_tool_data_table_conf.xml file, could you check the contents of that file? Also can you check the paster.log during server start up for any errors, its most likely the case that this changeset https://bitbucket.org/galaxy/galaxy-central/commits/af20b15f7eda (10145:af20b15f7eda) creates a non-compatiible tool data table entry in the main file. Thanks for using Galaxy, Dan On Jul 12, 2013, at 8:14 AM, graham etherington (TSL) wrote: > Hi, > I've tried using the Data Manager (Admin > Data > Manage local data > (beta)) to install builds for BWA and Samtools on my local Galaxy instance. > Previous to using the Data Manager, I used to add the build to > tool-data/shared/ucsc/builds.txt, create the .fai indexes (for samtools) > from the command line, add them to tool-data/sam_fa_indices.loc and > restart Galaxy (obviously doing a similar thing for BWA and adding the > build to bwa_index.loc). > > I thought I'd try using the Data Manager to add builds for BWA and > Samtools. The BWA builds work fine (I can map to the build), but when I > try to use SAM-to-BAM I get the error "Sequences are not currently > available for the specified build." > > Using the Data Manager creates the directory tool-data/n_sylvestris/ which > contains the sub-dirs 'seq', 'bwa_index' and 'sam_index'. > 'seq' contains a symlink to the n_sylvestris.fa sequence. > 'sam_index' and 'bwa_index' both contains the sub-directory > 'n_sylvestris', which contains a symlink to the symlink for > n_sylvestris.fa in 'seq' along with their respective n_sylvestris.fa.xxx > index files. > > OK - all goodŠ > > In tool-data/testtoolshed.g2.bx.psu.edu/repos/blankenberg/ there are three > subdirectories: > data_manager_bwa_index_builder, data_manager_sam_fa_index_builder and > data_manager_fetch_genome_all_fasta > All three directories contain all_fasta.loc, tool_data_table_conf.xml, > tool_data_table_conf.xml.sample and (for sam and bam dirs) their pertinent > index.loc file. > > The data_manager_fetch_genome_all_fasta/all_fasta.loc file contains the > path to the fasta symlinks. > > The all_fasta.loc files in the sam and bwa data_manager_index_builder > directories don't contain any uncommented lines. > > The index.loc files in the sam and bwa data_manager_index_builder > directories point to: > tool-data/n_sylvestris/bwa_index/n_sylvestris/n_sylvestris.fa > tool-data/n_sylvestris/sam_index/n_sylvestris/n_sylvestris.fa > > As BWA runs fine, it's obviously reading the bwa_index.loc file from the > directory: > tool-data/testtoolshed.g2.bx.psu.edu/repos/blankenberg/data_manager_bwa_ind > ex_builder/fe6508204acc/bwa_index.loc > > ...but it's not reading the samtools indexes at: > tool-data/testtoolshed.g2.bx.psu.edu/repos/blankenberg/data_manager_sam_fa_ > index_builder/926e50397b83/sam_fa_indices.loc > > For Galaxy to find the sam indexes, I have to go to the > tool-data/sam_fa_indices.loc file and manually insert into it the contents > of: > tool-data/testtoolshed.g2.bx.psu.edu/repos/blankenberg/data_manager_sam_fa_ > index_builder/926e50397b83/sam_fa_indices.loc > > > So, I guess my question is: other than inserting the genome builds into > builds.txt, should I be doing any other configuration to get Data Manager > to write and configure Galaxy to read it's newly created builds. I find it > strange that the BWA builds work OK, but the Samtools ones don't. > > I've done a few greps for mentions of .loc files in Galaxy and the only > difference between the bwa and sam .loc files is that there is a file > tool-data/tool_data_table_conf.xml (plus a .sample version) which contains: > > > > > >line_type, value, path > > > > > > Could Galaxy be reading this file and ignoring the one in > tool-data/testtoolshed.g2.bx.psu.edu/repos/blankenberg/ ?? > > > > Best wishes, > Graham > > > > > Dr. Graham Etherington > Bioinformatics Support Officer, > The Sainsbury Laboratory, > Norwich Research Park, > Norwich NR4 7UH. > UK > Tel: +44 (0)1603 450601 > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
