Re: [galaxy-dev] Tool added / pb with output file
Hello,
My problem is solved. Here is a solution :
my $output_galaxy = $ARGV[2];
.
if (! -e $output_perl_file) {print STDERR no output file\n;}
else {`(cp -a $output_perl_file $output_galaxy) ./error.log 21`}
And an explanation :
The output file generated by commands in my perl script is not the same
file that the output file generated by Galaxy. So, I have to copy or
move the content of the output file generated by my perl script in the
Galaxy output file (arg).
I'll will add my tool in Galaxy Tool Shed (perl script have been modified).
Faithfully,
Sarah Maman
Sarah Maman a écrit :
Hello,
I have added a tool in my local instance to map BAM files on reference
genome fasta file thanks to BWA.
This tool runs, and the output file generated in database/files/000 is
correct (not empty and good data inside) but the output listed in the
history panel of Galaxy interface , is empty.
Could you please help me ?
Here is my xml file:
tool id=BWA_aln_sampe name= BWA with BAM files
descriptionBWA mapping BAM files on reference genome fasta
file/description
command interpreter=perlbwa_aln_sampe.pl $input1 $input2 $output1
/command
inputs
param format=fasta name=input1 type=data label=Reference
genome from your history - fasta file/
param format=bam name=input2 type=data label=Bam file/
/inputs
outputs
data format=bam name=output1 /
/outputs
help
This tool map BAM files on a reference genome.
/help
/tool
Here is my perl file :
#!/usr/bin/perl -w
use strict;
use File::Basename;
my $input_ref_genome = $ARGV[0];
my $input_bam = $ARGV[1];
my $output_bam = $ARGV[2];
my $input_dir=dirname($input_bam);
my $input_basename=basename($input_bam,'.bam');
my $input_bamsorted=$input_dir/${input_basename}-sorted;
#Ouverture des fichiers entrants
open (IN, $input_ref_genome) or die Cannot open $input_ref_genome !;
open (IN, $input_bam) or die Cannot open $input_bam !;
#Indexation du genome de reference
#system(bwa index $input_ref_genome ./bwa.log 21);
#Tri par nom du bam entrant pour travailler sur des donnees pairees
system(samtools sort -n $input_bam $input_bamsorted ./samtools.log
21);
#Alignement des donnees pairees uniquement
system(bwa aln $input_ref_genome -b1 ${input_bamsorted}.bam
${input_bamsorted}-1.sai ./bwaaln.log 21);
system(bwa aln $input_ref_genome -b2 ${input_bamsorted}.bam
${input_bamsorted}-2.sai ./bwaaln.log 21);
#bwa sampe
system(bwa sampe $input_ref_genome ${input_bamsorted}-1.sai
${input_bamsorted}-2.sai ${input_bamsorted}.bam ${input_bamsorted}.bam
| samtools view -bS - $output_bam ./bwa.log 21);
close( IN );
close( IN );
Thanks in advance,
Sarah
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] samtools pileup deprecated
Hi All, Maybe I missed it on the list here, but it seems that for samtools the pileup command has been deprecated without backward compatibility from the more recent samtools packages... http://massgenomics.org/2012/03/5-things-to-know-about-samtools-mpileup.html Did anyone rewrite some of the samtools tool wrappers for this? I am currently testing whether the Bam-pileup command can be easily changed in the sam_pileup.py script. But some functionality has gone and some additional is there. It seems I am up-to-date with the latest wrappers in the galaxy_dist repository. Any pointers to recent wrappers would be welcomed. Alex ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] samtools pileup deprecated
Update: adaptation of a default run bam-pileup on my small test dataset gives the same result in my case using mpileup instead of pileup in the sam_pileup.py script (line 97). Not tested any additional parameters though so there might be subtle differences in the outcome. #prepare basic pileup command cmd = 'samtools mpileup %s -f %s %s %s' try: Van: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] Namens Bossers, Alex Verzonden: vrijdag 20 juli 2012 12:20 Aan: 'galaxy-dev@lists.bx.psu.edu' Onderwerp: [galaxy-dev] samtools pileup deprecated Hi All, Maybe I missed it on the list here, but it seems that for samtools the pileup command has been deprecated without backward compatibility from the more recent samtools packages... http://massgenomics.org/2012/03/5-things-to-know-about-samtools-mpileup.html Did anyone rewrite some of the samtools tool wrappers for this? I am currently testing whether the Bam-pileup command can be easily changed in the sam_pileup.py script. But some functionality has gone and some additional is there. It seems I am up-to-date with the latest wrappers in the galaxy_dist repository. Any pointers to recent wrappers would be welcomed. Alex ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy crach while visualization
Hmm, this appears to be an indexing error, but there are many potential causes so that doesn't narrow down the issue much. Because you're using galaxy-dist, the simplest thing to do is wait for our upcoming distribution (which should happen in the next couple days), upgrade and see if that fixes the problem. Trackster is under active development, so there are quite a few bug fixes and improvements that will be available with the next distribution. If you're still experiencing problems after upgrading, let us know and we'll keep trying to figure out what's going on. Best, J. On Jul 19, 2012, at 11:03 AM, Chebbi Mohamed Amine wrote: HI! I'am usig Galaxy-dist. SO I have done the test again, I did a simple scrolling to visualize by the Trackster the different chromosome regions and then the galaxy crashed suddenly. Here Above what java script log looks : 22%2C%22exon_number%22%2C%22frac%22%2C%22cov%22%2C%22conf_lo%22%2C%22FPKM%22%2C%22conf_hi%22%5DFailed to load resource http://xxx.x.com/tracks/data?chrom=chr19low=1449105high=1449853mode=Packresolution=1.8693312836279785dataset_id=7350596ac236c69dhda_ldda=hdafilter_cols=%5B%22Mapping+Quality%22%5DFailed to load resource However The paster.log looks more talking this time : xx.xx.xxx.xxx - - [19/Jul/2012:16:46:33 +0200] GET /tracks/data?chrom=chr19low=1457332high=1458266mode=Packresolution=2.3339731285988483dataset_id=2a8867ca0da68ac1hda_ldda=hdafilter_cols=%5B%22Score%22%2C%22exon_number%22%2C%22frac%22%2C%22cov%22%2C%22conf_lo%22%2C%22FPKM%22%2C%22conf_hi%22%5D HTTP/1.1 200 - http://..com/tracks/browser?id=df7a1f0c02a5b08e; Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 xx.xx.xxx.xxx - - [19/Jul/2012:16:46:33 +0200] GET /tracks/data?chrom=chr19low=1457332high=1458266mode=Autoresolution=2.3339731285988483dataset_id=18644201db609b11hda_ldda=hdafilter_cols=%5B%22Score%22%5D HTTP/1.1 200 - http://..com/tracks/browser?id=df7a1f0c02a5b08e; Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 xx.xx.xxx.xxx - - [19/Jul/2012:16:46:33 +0200] GET /tracks/data?chrom=chr19low=1457332high=1458266mode=Packresolution=2.3339731285988483dataset_id=7350596ac236c69dhda_ldda=hdafilter_cols=%5B%22Mapping+Quality%22%5D HTTP/1.1 500 - http://..com/tracks/browser?id=df7a1f0c02a5b08e; Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 Error - type 'exceptions.IOError': could not open index `/srv/Galaxy/Galaxy_test/database/files/_metadata_files/000/metadata_75.dat` 2012/7/19 Jeremy Goecks jeremy.goe...@emory.edu JavaScript logs are found in the Web browser. For Safari/Chrome, see the View -- Developer -- JavaScript console option; for Firefox, see Tools -- Web Developer -- Web Console. The fact that the Paster log isn't showing errors is problematic. Are there certain scenarios where Trackster/Galaxy is more likely to crash? Finally, are you using galaxy-dist or galaxy-central? Best, J. On Jul 18, 2012, at 8:50 AM, Chebbi Mohamed Amine wrote: -- Forwarded message -- From: Chebbi Mohamed Amine chebbimam...@gmail.com Date: 2012/7/18 Subject: Re: [galaxy-dev] Galaxy crach while visualization To: Jeremy Goecks jeremy.goe...@emory.edu Hello Jeremy! Thanks for your response. Yes the paster.log is not complaining. Excuse me where I can found the javascript log. In /var/log I haven't any file related to javascript.log. Thanks Amine 2012/7/17 Jeremy Goecks jeremy.goe...@emory.edu Hello Amine, The paster log looks normal, and Postgre log is standard given the crash. Was there anything out of the ordinary that you can spot in the paster log and/or Javascript log in your Web browser? Thanks, J. On Jul 17, 2012, at 6:53 AM, Chebbi Mohamed Amine wrote: Hello Galaxy team ! I have e strange problem whith my Galaxy trackster visualization . While viewing bookmarked regions in my visualisation the whole Galaxy process crashed and needed to be restarted, mainly when I use Chrome browser. Above the Paster.log : xxx.xxx.xx.xxx, xxx.xx.xx.x - - [12/Jul/2012:12:56:36 +0200] GET /galaxy/tracks/data?chrom=chr19low=1183972high=1185154mode=Autoresolution=2.954022988505747dataset_id=f4b17be90d9b3c9dhda_ldda=hdafilter_cols=%5B%22Score%22%5D HTTP/1.1 200 - http://xxx.xxx.com/galaxy/u/amine/v/rna-seq-analysis-tophats-output; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.47 Safari/536.11 xxx.xxx.xx.xxx, xxx.xx.xx.x - - [12/Jul/2012:12:56:36 +0200] GET
[galaxy-dev] Problem installing repositories from local tool shed: JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
Hi all, I'm new to Galaxy- so hello, and excuse me while I get to grips with all the conventions! I've successfully got my local Galaxy (galaxy-central) install set up and working correctly, as well as a local toolshed. I've also linked the two with a line in tool_sheds_conf.xml. But I'm getting the following problem when I try to install tools from the local toolshed (either ones I tried to make myself, or ones I downloaded from the Galaxy public toolshed). The following appears when I click 'install to local Galaxy': URL: http://www.my.domain/galaxy/admin_toolshed/prepare_for_install?tool_shed_url=http://www.my.domain/toolshed/repository_ids=adb5f5c93f827949changeset_revisions=f5ee1108336b File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/controllers/admin_toolshed.py', line 978 in prepare_for_install repo_information_dict = from_json_string( raw_text ) File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py', line 384 in loads File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py', line 402 in decode File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py', line 420 in raw_decode JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) The following error sometimes (not always, but that may be due to my use of Varnish) pops up in my apache logs when the above appears, could this be linked? File does not exist: /var/www/galaxy_home/galaxy-central/static/scripts/packed/modernizr.js, referer: http://www.my.domain/galaxy/admin Pertinent configuration details: * Galaxy and the toolshed proxied from Apache on another machine. * A Cosign-based authentication system is in place, so 'use_remote_user = True' in both Galaxy and the toolshed. * Galaxy's home dir is at /var/www/galaxy_home for consistency with other things I run. All input appreciated. Thanks, Jon -- Dr Jonathan Manning Bioinformatics Team Centre for Cardiovascular Science University of Edinburgh Queens Medical Research Institute 47 Little France Crescent Edinburgh EH16 4TJ United Kingdom T: +44 131 242 6700 F: +44 131 242 6782 E: jmann...@staffmail.ed.ac.uk The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] building a full trinity workflow from assembly through analysis
Excellent. Thanks, Jeremy! -b On Fri, Jul 20, 2012 at 8:37 AM, Jeremy Goecks jeremy.goe...@emory.edu wrote: Brian, For the initial trinity de novo assembly, I took Jeremy's initial workflow and tweaked it to work with the latest release - and submitted it to the galaxy tool shed. Excellent. I'll remove my old Trinity wrapper from our code base and point people to your wrapper. There are some complexities here, such as not knowing ahead of time how many different samples are to be processed - so this needs to be determined dynamically, which impacts the overall complexity of the total workflow. Is this possible in Galaxy, and if so, are there examples I can work from? Take a look at the repeat element; it enables a user to specify a number of inputs to a tool. http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax#A.3Crepeat.3E_tag_set Tool wrapper that use the repeat element include the GATK's depth of coverage and Cuffdiff. Best, J. -- -- Brian J. Haas Manager, Genome Annotation and Analysis, Research and Development The Broad Institute http://broad.mit.edu/~bhaas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy crach while visualization
Ok. Thanks for your clarifications. So I will wait for your upcoming distribution of Galaxy and redo the test. Best, Amine 2012/7/20 Jeremy Goecks jeremy.goe...@emory.edu Hmm, this appears to be an indexing error, but there are many potential causes so that doesn't narrow down the issue much. Because you're using galaxy-dist, the simplest thing to do is wait for our upcoming distribution (which should happen in the next couple days), upgrade and see if that fixes the problem. Trackster is under active development, so there are quite a few bug fixes and improvements that will be available with the next distribution. If you're still experiencing problems after upgrading, let us know and we'll keep trying to figure out what's going on. Best, J. On Jul 19, 2012, at 11:03 AM, Chebbi Mohamed Amine wrote: HI! I'am usig Galaxy-dist. SO I have done the test again, I did a simple scrolling to visualize by the Trackster the different chromosome regions and then the galaxy crashed suddenly. Here Above what java script log looks : 22%2C%22exon_number%22%2C%22frac%22%2C%22cov%22%2C%22conf_lo%22%2C%22FPKM%22%2C%22conf_hi%22%5DFailed to load resource http://xxx.x.com/tracks/data?chrom=chr19low=1449105high=1449853mode=Packresolution=1.8693312836279785dataset_id=7350596ac236c69dhda_ldda=hdafilter_cols=%5B%22Mapping+Quality%22%5DFailedto load resource However The paster.log looks more talking this time : xx.xx.xxx.xxx - - [19/Jul/2012:16:46:33 +0200] GET /tracks/data?chrom=chr19low=1457332high=1458266mode=Packresolution=2.3339731285988483dataset_id=2a8867ca0da68ac1hda_ldda=hdafilter_cols=%5B%22Score%22%2C%22exon_number%22%2C%22frac%22%2C%22cov%22%2C%22conf_lo%22%2C%22FPKM%22%2C%22conf_hi%22%5D HTTP/1.1 200 - http://..com/tracks/browser?id=df7a1f0c02a5b08e; Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 xx.xx.xxx.xxx - - [19/Jul/2012:16:46:33 +0200] GET /tracks/data?chrom=chr19low=1457332high=1458266mode=Autoresolution=2.3339731285988483dataset_id=18644201db609b11hda_ldda=hdafilter_cols=%5B%22Score%22%5D HTTP/1.1 200 - http://..com/tracks/browser?id=df7a1f0c02a5b08e; Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 xx.xx.xxx.xxx - - [19/Jul/2012:16:46:33 +0200] GET /tracks/data?chrom=chr19low=1457332high=1458266mode=Packresolution=2.3339731285988483dataset_id=7350596ac236c69dhda_ldda=hdafilter_cols=%5B%22Mapping+Quality%22%5D HTTP/1.1 500 - http://..com/tracks/browser?id=df7a1f0c02a5b08e; Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/12.04 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 Error - type 'exceptions.IOError': could not open index `/srv/Galaxy/Galaxy_test/database/files/_metadata_files/000/metadata_75.dat` 2012/7/19 Jeremy Goecks jeremy.goe...@emory.edu JavaScript logs are found in the Web browser. For Safari/Chrome, see the View -- Developer -- JavaScript console option; for Firefox, see Tools -- Web Developer -- Web Console. The fact that the Paster log isn't showing errors is problematic. Are there certain scenarios where Trackster/Galaxy is more likely to crash? Finally, are you using galaxy-dist or galaxy-central? Best, J. On Jul 18, 2012, at 8:50 AM, Chebbi Mohamed Amine wrote: -- Forwarded message -- From: Chebbi Mohamed Amine chebbimam...@gmail.com Date: 2012/7/18 Subject: Re: [galaxy-dev] Galaxy crach while visualization To: Jeremy Goecks jeremy.goe...@emory.edu Hello Jeremy! Thanks for your response. Yes the paster.log is not complaining. Excuse me where I can found the javascript log. In /var/log I haven't any file related to javascript.log. Thanks Amine 2012/7/17 Jeremy Goecks jeremy.goe...@emory.edu Hello Amine, The paster log looks normal, and Postgre log is standard given the crash. Was there anything out of the ordinary that you can spot in the paster log and/or Javascript log in your Web browser? Thanks, J. On Jul 17, 2012, at 6:53 AM, Chebbi Mohamed Amine wrote: Hello Galaxy team ! I have e strange problem whith my Galaxy trackster visualization . While viewing bookmarked regions in my visualisation the whole Galaxy process crashed and needed to be restarted, mainly when I use Chrome browser. Above the Paster.log : xxx.xxx.xx.xxx, xxx.xx.xx.x - - [12/Jul/2012:12:56:36 +0200] GET /galaxy/tracks/data?chrom=chr19low=1183972high=1185154mode=Autoresolution=2.954022988505747dataset_id=f4b17be90d9b3c9dhda_ldda=hdafilter_cols=%5B%22Score%22%5D HTTP/1.1 200 - http://xxx.xxx.com/galaxy/u/amine/v/rna-seq-analysis-tophats-output; Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.47 Safari/536.11 xxx.xxx.xx.xxx, xxx.xx.xx.x - - [12/Jul/2012:12:56:36 +0200] GET
[galaxy-dev] pipeline execution succeeds but galaxy shows failure
Hi,
I wrote a pipeline (xml attached) that, from what I can gather,
succeeds, but galaxy shows it as an error and doesn't make the output
file accessible as a new data set.
From the server log, I can see that the command line is being
constructed correctly, and it even indicates that it's captured the
output, but in the display of the web browser, it just shows up in the
error state. The script being run exits (0) on success. Any ideas?
Here's what the output section of my xml file looks like:
outputs
data format=bam name=coordSortedBam label=${tool.name}
on ${on_string}: coord-sorted read alignments
from_work_dir=alignment/alignment.coordSorted.bam/
/outputs
and here's what the server log states:
galaxy.jobs.handler INFO 2012-07-20 09:52:05,240 (30) Job dispatched
galaxy.jobs.runners.local DEBUG 2012-07-20 09:52:05,453 executing:
alignReads.pl --target
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_26.dat
-o alignment --aligner bowtie --single
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_23.dat
--seqType fq
galaxy.jobs DEBUG 2012-07-20 09:52:16,673 finish(): Moved
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/job_working_directory/000/30/alignment/alignment.coordSorted.bam
to /Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_50.dat
as directed by from_work_dir
Again, as far as I can tell, everything worked - but the browser
doesn't think so.
I've run the exact command above on the command-line, and it exits(0)
indicating success.
Also, I've verified that when run through my galaxy instance, the
galaxy-relocated output file is as expected.
Many thanks for your help. I'm still getting my feet wet with
galaxy, reading through all the documentation and searching the
mailing list for additional help.
best regards,
-brian
--
--
Brian J. Haas
Manager, Genome Annotation and Analysis, Research and Development
The Broad Institute
http://broad.mit.edu/~bhaas
tool id=alignreads name=alignReads version=0.0.1
descriptionalignReads: short read alignment tool wrapper/description
requirements
requirement type=packagetrinity/requirement
/requirements
command
alignReads.pl --target $target -o alignment --aligner $aligner_selection.aligner
## Inputs.
#if str($inputs.paired_or_single) == paired:
--left $inputs.left_input --right $inputs.right_input
#if $inputs.left_input.ext == 'fa':
--seqType fa
#else:
--seqType fq
#end if
#if str($inputs.library_type) != None:
--SS_lib_type $inputs.library_type
#end if
--max_dist_between_pairs $inputs.max_dist_between_pairs
#else:
--single $inputs.input
#if str($inputs.input.ext) == 'fa':
--seqType fa
#else:
--seqType fq
#end if
#if str($inputs.library_type) != None:
--SS_lib_type $inputs.library_type
#end if
#end if
## Additional parameters.
##if str($inputs.use_additional) == yes:
## -- $inputs.additional_params
##end if
## direct to output
## $trinity_log 2amp;1
/command
inputs
param format=fasta name=target type=data label=target help=Fasta sequences targeted for short-read alignment /
conditional name=inputs
param name=paired_or_single type=select label=Paired or Single-end data?
option value=pairedPaired/option
option value=singleSingle/option
/param
when value=paired
param format=fasta,fastq name=left_input type=data label=Left/Forward strand reads help=/
param format=fasta,fastq name=right_input type=data label=Right/Reverse strand reads help=/
param name=library_type type=select label=Strand-specific Library Type
option value=NoneNone/option
option value=FRFR/option
option value=RFRF/option
/param
param name=max_dist_between_pairs type=integer value=2000 min=1 label=max_dist_between_pairs help=Maximum length expected between fragment pairs as aligned to the target, including introns where relevant./
/when
when value=single
param format=fasta,fastq name=input type=data label=Single-end reads help=/
param name=library_type type=select label=Strand-specific Library Type
option value=NoneNone/option
option value=FF/option
option value=RR/option
/param
/when
/conditional
conditional name=aligner_selection
param name=aligner type=select label=Select alignment tool to run
option value=bowtiebowtie/option
option value=bwabwa/option
option
Re: [galaxy-dev] Problem installing repositories from local tool shed: JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
Hi Jon, It looks like your tool shed is not returning the information necessary for installing your tool shed repository into your local Galaxy instance. Try putting the following print statement just before line 978 in ~/lib/galaxy/web/controllers/admin_toolshed.py. This will tell you why the information is not json decodable. print raw_text: , raw_text It will be difficult for me to tell you what caused this since I do not have access to either your tool shed or your local Galaxy instance, so hopefully this will provide enough information for you to determine the cause. Thanks! Greg Von Kuster On Jul 20, 2012, at 8:51 AM, Jon Manning wrote: Hi all, I'm new to Galaxy- so hello, and excuse me while I get to grips with all the conventions! I've successfully got my local Galaxy (galaxy-central) install set up and working correctly, as well as a local toolshed. I've also linked the two with a line in tool_sheds_conf.xml. But I'm getting the following problem when I try to install tools from the local toolshed (either ones I tried to make myself, or ones I downloaded from the Galaxy public toolshed). The following appears when I click 'install to local Galaxy': URL: http://www.my.domain/galaxy/admin_toolshed/prepare_for_install?tool_shed_url=http://www.my.domain/toolshed/repository_ids=adb5f5c93f827949changeset_revisions=f5ee1108336b File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/controllers/admin_toolshed.py', line 978 in prepare_for_install repo_information_dict = from_json_string( raw_text ) File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py', line 384 in loads File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py', line 402 in decode File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py', line 420 in raw_decode JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) The following error sometimes (not always, but that may be due to my use of Varnish) pops up in my apache logs when the above appears, could this be linked? File does not exist: /var/www/galaxy_home/galaxy-central/static/scripts/packed/modernizr.js, referer: http://www.my.domain/galaxy/admin Pertinent configuration details: * Galaxy and the toolshed proxied from Apache on another machine. * A Cosign-based authentication system is in place, so 'use_remote_user = True' in both Galaxy and the toolshed. * Galaxy's home dir is at /var/www/galaxy_home for consistency with other things I run. All input appreciated. Thanks, Jon -- Dr Jonathan Manning Bioinformatics Team Centre for Cardiovascular Science University of Edinburgh Queens Medical Research Institute 47 Little France Crescent Edinburgh EH16 4TJ United Kingdom T: +44 131 242 6700 F: +44 131 242 6782 E: jmann...@staffmail.ed.ac.uk The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem installing repositories from local tool shed: JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
Hi Greg, Thanks very much for the reply. I've actually just now worked around this- it was due somehow to our varnish/apache configs. I'll explain our setup and how I fixed this in case anyone else gets the same error. For context our setup is a number of virtual machines: * Varnish caching HTTP reverse proxy on one VM handles and caches requests, pointing them at one of a few apache instances running on a handful of other VMs. * A single mysql server itself running on a dedicated VM. * A high-memory multi-core VM we use for analysis purposes- which now also runs Galaxy. This machine also runs apache for various specialist reasons. I was sending things through varnish in order to use the cosign authentication we have set up there. I'm still doing that for toolshed administration purposes, but for the link from Galaxy itself for installing tools I've changed the line in tool_sheds_conf.xml to point directly at the apache on the Galaxy-running machine instead (which I configured appropriately to pass requests to the toolshed). That seems to have fixed things. Obviously this upsets the toolshed because it's not getting the user from Cosign, but I don't need galaxy to authenticate with the toolshed anyway, so I just set a dummy REMOTE_USER on the proxy request from the galaxy-local apache to keep the 'remote_user' setting in the toolshed config happy. Thanks again, Jon On 20/07/2012 15:57, Greg Von Kuster wrote: Hi Jon, It looks like your tool shed is not returning the information necessary for installing your tool shed repository into your local Galaxy instance. Try putting the following print statement just before line 978 in ~/lib/galaxy/web/controllers/admin_toolshed.py. This will tell you why the information is not json decodable. print raw_text: , raw_text It will be difficult for me to tell you what caused this since I do not have access to either your tool shed or your local Galaxy instance, so hopefully this will provide enough information for you to determine the cause. Thanks! Greg Von Kuster On Jul 20, 2012, at 8:51 AM, Jon Manning wrote: Hi all, I'm new to Galaxy- so hello, and excuse me while I get to grips with all the conventions! I've successfully got my local Galaxy (galaxy-central) install set up and working correctly, as well as a local toolshed. I've also linked the two with a line in tool_sheds_conf.xml. But I'm getting the following problem when I try to install tools from the local toolshed (either ones I tried to make myself, or ones I downloaded from the Galaxy public toolshed). The following appears when I click 'install to local Galaxy': URL: http://www.my.domain/galaxy/admin_toolshed/prepare_for_install?tool_shed_url=http://www.my.domain/toolshed/repository_ids=adb5f5c93f827949changeset_revisions=f5ee1108336b File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py', line 143 in __call__ app_iter = self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/debug/prints.py', line 98 in __call__ environ, self.app) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/wsgilib.py', line 539 in intercept_output app_iter = application(environ, replacement_start_response) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/recursive.py', line 80 in __call__ return self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/middleware/remoteuser.py', line 91 in __call__ return self.app( environ, start_response ) File '/var/www/galaxy_home/galaxy-central/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py', line 632 in __call__ return self.application(environ, start_response) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/base.py', line 160 in __call__ body = method( trans, **kwargs ) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/framework/__init__.py', line 184 in decorator return func( self, trans, *args, **kwargs ) File '/var/www/galaxy_home/galaxy-central/lib/galaxy/web/controllers/admin_toolshed.py', line 978 in prepare_for_install repo_information_dict = from_json_string( raw_text ) File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/__init__.py', line 384 in loads File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py', line 402 in decode File '/var/www/galaxy_home/galaxy-central/eggs/simplejson-2.1.1-py2.6-linux-x86_64-ucs4.egg/simplejson/decoder.py', line 420 in raw_decode JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) The following error sometimes (not always, but that may be due to my use of Varnish) pops up in my apache logs when the above appears, could this be linked? File does not exist:
Re: [galaxy-dev] pipeline execution succeeds but galaxy shows failure
Brian;
I wrote a pipeline (xml attached) that, from what I can gather,
succeeds, but galaxy shows it as an error and doesn't make the output
file accessible as a new data set.
Is it possible the software is writing to standard error? Galaxy doesn't
check status codes, but rather check for stderr and assumes that output
indicates a problem. You can wrap the problematic programs with a little
script to eat up stderr and check that everything is okay:
http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr
Brad
From the server log, I can see that the command line is being
constructed correctly, and it even indicates that it's captured the
output, but in the display of the web browser, it just shows up in the
error state. The script being run exits (0) on success. Any ideas?
Here's what the output section of my xml file looks like:
outputs
data format=bam name=coordSortedBam label=${tool.name}
on ${on_string}: coord-sorted read alignments
from_work_dir=alignment/alignment.coordSorted.bam/
/outputs
and here's what the server log states:
galaxy.jobs.handler INFO 2012-07-20 09:52:05,240 (30) Job dispatched
galaxy.jobs.runners.local DEBUG 2012-07-20 09:52:05,453 executing:
alignReads.pl --target
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_26.dat
-o alignment --aligner bowtie --single
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_23.dat
--seqType fq
galaxy.jobs DEBUG 2012-07-20 09:52:16,673 finish(): Moved
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/job_working_directory/000/30/alignment/alignment.coordSorted.bam
to /Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_50.dat
as directed by from_work_dir
Again, as far as I can tell, everything worked - but the browser
doesn't think so.
I've run the exact command above on the command-line, and it exits(0)
indicating success.
Also, I've verified that when run through my galaxy instance, the
galaxy-relocated output file is as expected.
Many thanks for your help. I'm still getting my feet wet with
galaxy, reading through all the documentation and searching the
mailing list for additional help.
best regards,
-brian
--
--
Brian J. Haas
Manager, Genome Annotation and Analysis, Research and Development
The Broad Institute
http://broad.mit.edu/~bhaas
tool id=alignreads name=alignReads version=0.0.1
descriptionalignReads: short read alignment tool wrapper/description
requirements
requirement type=packagetrinity/requirement
/requirements
command
alignReads.pl --target $target -o alignment --aligner
$aligner_selection.aligner
## Inputs.
#if str($inputs.paired_or_single) == paired:
--left $inputs.left_input --right $inputs.right_input
#if $inputs.left_input.ext == 'fa':
--seqType fa
#else:
--seqType fq
#end if
#if str($inputs.library_type) != None:
--SS_lib_type $inputs.library_type
#end if
--max_dist_between_pairs $inputs.max_dist_between_pairs
#else:
--single $inputs.input
#if str($inputs.input.ext) == 'fa':
--seqType fa
#else:
--seqType fq
#end if
#if str($inputs.library_type) != None:
--SS_lib_type $inputs.library_type
#end if
#end if
## Additional parameters.
##if str($inputs.use_additional) == yes:
## -- $inputs.additional_params
##end if
## direct to output
## $trinity_log 2amp;1
/command
inputs
param format=fasta name=target type=data label=target
help=Fasta sequences targeted for short-read alignment /
conditional name=inputs
param name=paired_or_single type=select label=Paired
or Single-end data?
option value=pairedPaired/option
option value=singleSingle/option
/param
when value=paired
param format=fasta,fastq name=left_input type=data
label=Left/Forward strand reads help=/
param format=fasta,fastq name=right_input type=data
label=Right/Reverse strand reads help=/
param name=library_type type=select
label=Strand-specific Library Type
option value=NoneNone/option
option value=FRFR/option
option value=RFRF/option
/param
param name=max_dist_between_pairs type=integer
value=2000 min=1 label=max_dist_between_pairs help=Maximum length
expected between fragment pairs as aligned to the target, including introns
where relevant./
/when
when value=single
param
Re: [galaxy-dev] pipeline execution succeeds but galaxy shows failure
Thanks Brad and Nicole! This definitely explains it. The stderr
(which almost all my tools generate for monitoring purposes) was
resulting in galaxy thinking the process failed.
All is well and good now. HUGE THANKS!
-brian
On Fri, Jul 20, 2012 at 1:53 PM, Nicole Rockweiler
n.rockwei...@gmail.com wrote:
Hi Brian,
A couple of days ago, smcmanus pushed the following change to the repo:
Tools can now specify their own handling of stderr and stdout regular
expressions as well as exit code ranges.
https://bitbucket.org/galaxy/galaxy-central/issue/325/allow-tool-authors-to-decide-whether-to-use-return-codes-or-stderr-for-detecting-job
It looks like the documentation has yet to be written.
Hope this helps,
Nicole
On Fri, Jul 20, 2012 at 12:46 PM, Brad Chapman chapm...@50mail.com wrote:
Brian;
I wrote a pipeline (xml attached) that, from what I can gather,
succeeds, but galaxy shows it as an error and doesn't make the output
file accessible as a new data set.
Is it possible the software is writing to standard error? Galaxy doesn't
check status codes, but rather check for stderr and assumes that output
indicates a problem. You can wrap the problematic programs with a little
script to eat up stderr and check that everything is okay:
http://wiki.g2.bx.psu.edu/Future/Job%20Failure%20When%20stderr
Brad
From the server log, I can see that the command line is being
constructed correctly, and it even indicates that it's captured the
output, but in the display of the web browser, it just shows up in the
error state. The script being run exits (0) on success. Any ideas?
Here's what the output section of my xml file looks like:
outputs
data format=bam name=coordSortedBam label=${tool.name}
on ${on_string}: coord-sorted read alignments
from_work_dir=alignment/alignment.coordSorted.bam/
/outputs
and here's what the server log states:
galaxy.jobs.handler INFO 2012-07-20 09:52:05,240 (30) Job dispatched
galaxy.jobs.runners.local DEBUG 2012-07-20 09:52:05,453 executing:
alignReads.pl --target
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_26.dat
-o alignment --aligner bowtie --single
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_23.dat
--seqType fq
galaxy.jobs DEBUG 2012-07-20 09:52:16,673 finish(): Moved
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/job_working_directory/000/30/alignment/alignment.coordSorted.bam
to
/Users/bhaas/BioIfx/Galaxy/galaxy-dist/database/files/000/dataset_50.dat
as directed by from_work_dir
Again, as far as I can tell, everything worked - but the browser
doesn't think so.
I've run the exact command above on the command-line, and it exits(0)
indicating success.
Also, I've verified that when run through my galaxy instance, the
galaxy-relocated output file is as expected.
Many thanks for your help. I'm still getting my feet wet with
galaxy, reading through all the documentation and searching the
mailing list for additional help.
best regards,
-brian
--
--
Brian J. Haas
Manager, Genome Annotation and Analysis, Research and Development
The Broad Institute
http://broad.mit.edu/~bhaas
tool id=alignreads name=alignReads version=0.0.1
descriptionalignReads: short read alignment tool
wrapper/description
requirements
requirement type=packagetrinity/requirement
/requirements
command
alignReads.pl --target $target -o alignment --aligner
$aligner_selection.aligner
## Inputs.
#if str($inputs.paired_or_single) == paired:
--left $inputs.left_input --right $inputs.right_input
#if $inputs.left_input.ext == 'fa':
--seqType fa
#else:
--seqType fq
#end if
#if str($inputs.library_type) != None:
--SS_lib_type $inputs.library_type
#end if
--max_dist_between_pairs
$inputs.max_dist_between_pairs
#else:
--single $inputs.input
#if str($inputs.input.ext) == 'fa':
--seqType fa
#else:
--seqType fq
#end if
#if str($inputs.library_type) != None:
--SS_lib_type $inputs.library_type
#end if
#end if
## Additional parameters.
##if str($inputs.use_additional) == yes:
## -- $inputs.additional_params
##end if
## direct to output
## $trinity_log 2amp;1
/command
inputs
param format=fasta name=target type=data
label=target help=Fasta sequences targeted for short-read alignment /
conditional name=inputs
param name=paired_or_single type=select
label=Paired or Single-end data?
[galaxy-dev] configuring job runner for toolshed tools
Is there a way to configure a tool downloaded from the toolshed depot to use a job runner other than the local runner? The tool_id for the toolshed tool isn't honored in universe_wsgi.ini the way the default tools are. Specifically, has anyone configured a toolshed tool to use drmaa instead of localhost? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] configuring job runner for toolshed tools
Hi David, I'm not familiar enough with the job runners to know what the problem may be, However, for tools installed from the tool shed, the tool id is the tool-shed generated guid instead of the id attribute value of the tool config's tool tag. So, for example, if you install the freebayes tool shed repository from the main Galaxy tool shed, your tool panel will display the single freebayes tool, which is the only tool contained in the freebayes repository. When this installed tool is loaded into Galaxy, the tools id attribute value is: toolshed.g2.bx.psu.edu/repos/devteam/freebayes/freebayes/0.0.2 and the tool's old_id attribute value is the value os the id attribute in it's tool config tool tag: freebayes Have your tried configuring the installed tool using the new id (guid) rather than the old id? I'm not sure if this works, but just wanted to make sure you were using the correct id value. If it still doesn't work, let us know and we'll get a fix asap. Thanks! Greg Von Kuster On Jul 20, 2012, at 5:16 PM, David Hoover wrote: Is there a way to configure a tool downloaded from the toolshed depot to use a job runner other than the local runner? The tool_id for the toolshed tool isn't honored in universe_wsgi.ini the way the default tools are. Specifically, has anyone configured a toolshed tool to use drmaa instead of localhost? David Hoover Helix Systems Staff ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
