[galaxy-dev] Deseq2 wrapper question
Hi All, we are making a wrapper for deseq2, and there is a step where control/experimental conditions need to be determined. For example, it would be a column like the following in the input file Type CTC CTC LM LM PT PT and we want the select list to contain CTC LM PT In other words, we want to filter out the 'Type' and have only distinct ones in the select list. We could make it in two steps, generate a intermediate file for this but it would be nice that we could directly retrieve the distinct types to make the list. I took a look at the tool config wiki, but didn't see anything (maybe because it was a quick scan). Does anyone have any tip on this? we'll really appreciate. Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Deseq2 wrapper question
Hi Joachim, Thanks for the info! but I could not register on your instance...and thus could not check your wrapper. :-( Please help! Thanks, Rui On Wed, Sep 11, 2013 at 12:59 AM, Joachim Jacob | VIB | < joachim.ja...@vib.be> wrote: > Hi Rui, > > I should provide a 'sample metadata table', a tabular file with one column > the names of your samples, the other column the associated metadata (e.g. > type). > > This is how I've done it in my DESeq2 wrapper. :-) Feel free to check the > interface at > http://toolshed.bits.vib.be/**view/joachim/deseq2<http://toolshed.bits.vib.be/view/joachim/deseq2> > > > Cheers, > Joachim > > Joachim Jacob > Contact details: > http://www.bits.vib.be/index.**php/about/80-team<http://www.bits.vib.be/index.php/about/80-team> > > > > On 09/11/2013 01:15 AM, ruiwang.sz wrote: > >> Hi All, >> >> we are making a wrapper for deseq2, and there is a step where >> control/experimental conditions >> need to be determined. For example, it would be a column like the >> following in the input file >> >> Type >> CTC >> CTC >> LM >> LM >> PT >> PT >> >> and we want the select list to contain >> >> CTC >> LM >> PT >> >> In other words, we want to filter out the 'Type' and have only distinct >> ones in the select list. We could >> make it in two steps, generate a intermediate file for this but it would >> be nice that we could directly >> retrieve the distinct types to make the list. I took a look at the tool >> config wiki, but didn't see anything >> (maybe because it was a quick scan). Does anyone have any tip on this? >> we'll really appreciate. >> >> Thanks, >> Rui >> >> >> __**_ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >> >> http://galaxyproject.org/**search/mailinglists/<http://galaxyproject.org/search/mailinglists/> >> > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Deseq2 wrapper question
Hey Guys, It's great to receive so many responses! Thank you all for the information... and let me check out each of the masterpiece. :-) Best, Rui On Wed, Sep 11, 2013 at 1:56 AM, Peter Cock wrote: > Wow - that makes at least five Deseq2 wrappers for Galaxy > available or in progress :( > > Bjoern's wrapper on the (test) tool shed, > http://testtoolshed.g2.bx.psu.edu/view/bgruening/deseq2 > > Ross' combined wrapper for edgeR, DESeq2 and voom in one tool: > http://testtoolshed.g2.bx.psu.edu/view/fubar/differential_count_models > > Joachim's wrapper on the Tool Shed > http://toolshed.bits.vib.be/view/joachim/deseq2 > > Vipin's which is soon to be released to the (Test?) Tool Shed, > https://github.com/ratschlab/oqtans_tools/tree/master/DESeq2 > > And Rui's is working on one too. > > This does seem like duplicated effort & a source of confusion > for end users and Galaxy administrators (a problem not unique > to deseq2, but affecting many Galaxy wrappers). > > I appreciate there will be different needs, and one wrapper > may not suit all, but I would prefer if the default behaviour > for Galaxy tool wrapper authors was to collaborate on one > good wrapper rather than writing competing ones. > > When I started work on a new wrapper I tried to announce > this on the mailing list to find out if anyone else was already > tackling the same tool - and that seemed to work quite well. > Of course, the volume of emails on galaxy-dev has grown > quite a lot over the last few years so that may not be as > effective, but the archives should be searchable. > > Perhaps we need to improve communication in some way? > One option would be to encourage greater use of the Test > Tool Shed for works in progress to give them visibility? > You could even have the Tool Shed itself require a search > step before creating a new repository to avoid accidental > duplication of effort? Or maybe a wiki page of wrappers > in progress? > > (And maybe we should split this into a new thread) > > Regards, > > Peter > > > On Wed, Sep 11, 2013 at 8:59 AM, Joachim Jacob | VIB | > wrote: > > Hi Rui, > > > > I should provide a 'sample metadata table', a tabular file with one > column > > the names of your samples, the other column the associated metadata (e.g. > > type). > > > > This is how I've done it in my DESeq2 wrapper. :-) Feel free to check the > > interface at http://toolshed.bits.vib.be/view/joachim/deseq2 > > > > > > Cheers, > > Joachim > > > > Joachim Jacob > > Contact details: http://www.bits.vib.be/index.php/about/80-team > > > > > > > > On 09/11/2013 01:15 AM, ruiwang.sz wrote: > >> > >> Hi All, > >> > >> we are making a wrapper for deseq2, and there is a step where > >> control/experimental conditions > >> need to be determined. For example, it would be a column like the > >> following in the input file > >> > >> Type > >> CTC > >> CTC > >> LM > >> LM > >> PT > >> PT > >> > >> and we want the select list to contain > >> > >> CTC > >> LM > >> PT > >> > >> In other words, we want to filter out the 'Type' and have only distinct > >> ones in the select list. We could > >> make it in two steps, generate a intermediate file for this but it would > >> be nice that we could directly > >> retrieve the distinct types to make the list. I took a look at the tool > >> config wiki, but didn't see anything > >> (maybe because it was a quick scan). Does anyone have any tip on this? > >> we'll really appreciate. > >> > >> Thanks, > >> Rui > >> > >> > >> ___ > >> Please keep all replies on the list by using "reply all" > >> in your mail client. To manage your subscriptions to this > >> and other Galaxy lists, please use the interface at: > >>http://lists.bx.psu.edu/ > >> > >> To search Galaxy mailing lists use the unified search at: > >>http://galaxyproject.org/search/mailinglists/ > > > > > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] puzzle: tool result always "set dataset state to ERROR"
Hi Guys,
I'm doing a very simple test about deseq2...there is a weird situation
always happening:
It looks like the deseq2 tool executed just fine, without any error, and
the result files were created, but after the set_metadata, galaxy always
'set dataset state to ERROR'.
The xml file for this test is:
Determines differentially expressed transcripts from read
alignments
t.sh $input1 $test $out $log
Basically, we have an input file from Partek flow(input1), input2 is one
column
from input1, which has all the conditions, in our test, it has 3
conditions, CTC, LM, PT.
input name "test" is the dropdown list that contain all 3 conditions, and
we choose
one as control condition, in our case it is CTC.
t.sh is very simple, it basically calls R script:
Rscript /home/bioinfo/app/galaxy-dist/tools/Deseq/workflow.R $1 $2 $3 $4
now, in the workflow.R, the related output part is:
for (i in 2:3) {
res <- results(dds, contrasts[i])
## sort the result table by FDR
res <- res[order(res$padj),]
## Output the results
write.table(as.data.frame(res), file=paste0(args[i+1]), sep = "\t")
}
So, it looks like the result files were generated as expected, with the
correct information.
However, it always was set to state ERROR. Am I missing something? Or did
anyone see this before?
Any inputs will be greatly appreciated!
Thanks,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] puzzle: tool result always "set dataset state to ERROR"
Hi Peter,
Thanks for the reply.
I checked the history view. I don't see anything when I clicked the "i"
icon and click the stderr link. However, here is what I see if I click the
little bug icon next to the "i":
Tool execution generated the following error message:
Loading required package: GenomicRanges
Loading required package: methods
Loading required package: BiocGenerics
Loading required package: parallel
Attaching package: ‘BiocGenerics’
The following objects are masked from ‘package:parallel’:
clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
clusterExport, clusterMap, parApply, parCapply, parLapply,
parLapplyLB, parRapply, parSapply, parSapplyLB
The following object is masked from ‘package:stats’:
xtabs
The following objects are masked from ‘package:base’:
anyDuplicated, as.data.frame, cbind, colnames, duplicated, eval,
Filter, Find, get, intersect, lapply, Map, mapply, match, mget,
order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
rbind, Reduce, rep.int, rownames, sapply, setdiff, sort, table,
tapply, union, unique, unlist
Loading required package: IRanges
Loading required package: Biobase
Welcome to Bioconductor
Vignettes contain introductory material; view with
'browseVignettes()'. To cite Bioconductor, see
'citation("Biobase")', and for packages 'citation("pkgname")'.
Loading required package: lattice
Loading required package: Rcpp
Loading required package: RcppArmadillo
estimating size factors
estimating dispersions
gene-wise dispersion estimates
mean-dispersion relationship
final dispersion estimates
fitting generalized linear model
All these are the normal output message from loading the 'Deseq2' package
in R, and run some functions. After I use
suppressMessages(Library("DESeq2"))
to suppress the library loading messages, now, the following lines appear
if I click "i" and then click "stderr"
estimating size factors
estimating dispersions
gene-wise dispersion estimates
mean-dispersion relationship
final dispersion estimates
fitting generalized linear model
I guess I need to suppress them as well -- but how come they are on stderr
not stdout?
Thanks,
Rui
On Sun, Sep 29, 2013 at 3:36 PM, Peter Cock wrote:
> On Sun, Sep 29, 2013 at 8:47 PM, ruiwang.sz wrote:
> > Hi Guys,
> >
> > I'm doing a very simple test about deseq2...there is a weird situation
> > always happening:
> >
> > It looks like the deseq2 tool executed just fine, without any error, and
> the
> > result files were created, but after the set_metadata, galaxy always 'set
> > dataset state to ERROR'.
> >
> > The xml file for this test is:
> >
> >
> > Determines differentially expressed transcripts from read
> > alignments
> >
> >t.sh $input1 $test $out $log
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Basically, we have an input file from Partek flow(input1), input2 is one
> > column
> > from input1, which has all the conditions, in our test, it has 3
> conditions,
> > CTC, LM, PT.
> >
> > input name "test" is the dropdown list that contain all 3 conditions,
> and we
> > choose
> > one as control condition, in our case it is CTC.
> >
> > t.sh is very simple, it basically calls R script:
> >
> > Rscript /home/bioinfo/app/galaxy-dist/tools/Deseq/workflow.R $1 $2 $3 $4
> >
> > now, in the workflow.R, the related output part is:
> >
> > for (i in 2:3) {
> > res <- results(dds, contrasts[i])
> >
> > ## sort the result table by FDR
> > res <- res[order(res$padj),]
> >
> > ## Output the results
> > write.table(as.data.frame(res), file=paste0(args[i+1]), sep = "\t")
> > }
> >
> > So, it looks like the result files were generated as expected, with the
> > correct information.
> > However, it always was set to state ERROR. Am I missing something? Or did
> > anyone see this before?
> >
> > Any inputs will be greatly appreciated!
> >
> > Thanks,
> > Rui
>
> Hi Rui,
>
> Was anything written to stderr, like a warning from R itself?
> You should be able to check via the "i" icon of the dataset
> in the history view.
>
> By default, Galaxy treats any output on stderr as an error,
> see the tag information here:
> http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax
>
> Peter
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] puzzle: tool result always "set dataset state to ERROR"
Hi Peter,
Thanks! I got it working by putting a suppressMessages() around the
function call.
I might have some questions on how to generate output file names on the
fly...let me make some attempts
first. :-)
Best,
Rui
On Sun, Sep 29, 2013 at 10:54 PM, ruiwang.sz wrote:
> Hi Peter,
>
> Thanks for the reply.
>
> I checked the history view. I don't see anything when I clicked the "i"
> icon and click the stderr link. However, here is what I see if I click the
> little bug icon next to the "i":
>
>
> Tool execution generated the following error message:
>
> Loading required package: GenomicRanges
> Loading required package: methods
> Loading required package: BiocGenerics
> Loading required package: parallel
>
> Attaching package: ‘BiocGenerics’
>
> The following objects are masked from ‘package:parallel’:
>
> clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
> clusterExport, clusterMap, parApply, parCapply, parLapply,
> parLapplyLB, parRapply, parSapply, parSapplyLB
>
> The following object is masked from ‘package:stats’:
>
> xtabs
>
> The following objects are masked from ‘package:base’:
>
> anyDuplicated, as.data.frame, cbind, colnames, duplicated, eval,
> Filter, Find, get, intersect, lapply, Map, mapply, match, mget,
> order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
> rbind, Reduce, rep.int, rownames, sapply, setdiff, sort, table,
> tapply, union, unique, unlist
>
> Loading required package: IRanges
> Loading required package: Biobase
> Welcome to Bioconductor
>
> Vignettes contain introductory material; view with
> 'browseVignettes()'. To cite Bioconductor, see
> 'citation("Biobase")', and for packages 'citation("pkgname")'.
>
> Loading required package: lattice
> Loading required package: Rcpp
> Loading required package: RcppArmadillo
> estimating size factors
> estimating dispersions
> gene-wise dispersion estimates
> mean-dispersion relationship
> final dispersion estimates
> fitting generalized linear model
>
> All these are the normal output message from loading the 'Deseq2' package
> in R, and run some functions. After I use
>
> suppressMessages(Library("DESeq2"))
>
> to suppress the library loading messages, now, the following lines appear
> if I click "i" and then click "stderr"
>
> estimating size factors
> estimating dispersions
> gene-wise dispersion estimates
> mean-dispersion relationship
> final dispersion estimates
> fitting generalized linear model
>
> I guess I need to suppress them as well -- but how come they are on stderr
> not stdout?
>
> Thanks,
> Rui
>
>
>
> On Sun, Sep 29, 2013 at 3:36 PM, Peter Cock wrote:
>
>> On Sun, Sep 29, 2013 at 8:47 PM, ruiwang.sz wrote:
>> > Hi Guys,
>> >
>> > I'm doing a very simple test about deseq2...there is a weird situation
>> > always happening:
>> >
>> > It looks like the deseq2 tool executed just fine, without any error,
>> and the
>> > result files were created, but after the set_metadata, galaxy always
>> 'set
>> > dataset state to ERROR'.
>> >
>> > The xml file for this test is:
>> >
>> >
>> > Determines differentially expressed transcripts from read
>> > alignments
>> >
>> >t.sh $input1 $test $out $log
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > Basically, we have an input file from Partek flow(input1), input2 is one
>> > column
>> > from input1, which has all the conditions, in our test, it has 3
>> conditions,
>> > CTC, LM, PT.
>> >
>> > input name "test" is the dropdown list that contain all 3 conditions,
>> and we
>> > choose
>> > one as control condition, in our case it is CTC.
>> >
>> > t.sh is very simple, it basically calls R script:
>> >
>> > Rscript /home/bioinfo/app/galaxy-dist/tools/Deseq/workflow.R $1 $2 $3 $4
>> >
>> > now, in the workflow.R, the related output part is:
>> >
>> > for (i in 2:3) {
>> > res <- results(dds, contrasts[i])
>> >
>> > ## sort the result table by FDR
>> > res <- res[order(res$padj),]
>> >
>&
[galaxy-dev] wigTobigwig error: broken pipe and len file not found
Hi All, I‘m seeing some weird error messages...I googled but didn't see anything useful: So, it is during the wigToBigwig conversion: Dataset generation errors Dataset 47: Wig/BedGraph-to-bigWig on data 43 Tool execution generated the following error message: grep: writing output: Broken pipe grep: writing output: Broken pipe grep: writing output: Broken pipe and many more lines of the same error The tool produced the following additional output: Couldn't open tool-data/shared/ucsc/chrom/mm10.len , No such file or directory but it is there: ls -l tool-data/shared/ucsc/chrom/mm10.len -rw-rw-r-- 1 bioinfoadmin bioinfoadmin 1405 Oct 9 11:33 tool-data/shared/ucsc/chrom/mm10.len >From the Galaxy server log, I found these: galaxy.tools WARNING 2013-11-15 17:46:43,200 Failed to resolve dependency on 'ucsc_tools', ignoring galaxy.jobs.runners.local DEBUG 2013-11-15 17:46:43,255 (216) executing: grep -v "^track" /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_458.dat | wigToBigWig stdin tool-data/shared/ucsc/chrom/mm10.len /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_473.dat -blockSize=256 -itemsPerSlot=1024 -clip 2>&1 || echo "Error running wigToBigWig." >&2 and then, setmetadata set the dataset state to ERROR. I did verify that the result dataset_473.dat has a size 0. However, when I run the command alone: grep -v "^track" /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_458.dat | wigToBigWig stdin tool-data/shared/ucsc/chrom/mm10.len /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_473.dat -blockSize=256 -itemsPerSlot=1024 -clip it successfully generated the dataset_473.dat. also, I look into the wig_to_Bigwig.xml in tools/filters, the command is: grep -v "^track" $input1 | wigToBigWig stdin $chromInfo $out_file1 however, I don't see where this "$chromInfo" was defined, yet it indeed found the the correct value: tool-data/shared/ucsc/chrom/mm10.len. how does that happen? it is very puzzling...I'm not sure if anyone has seen this before, please let me know! Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] wigTobigwig error: broken pipe and len file not found
Hi Nate, Thanks for the note! So may I ask whether this is expected? I mean, the len_file_path points to tool-data/..., which is right inside galaxy-dist, and it is natural. I didn't realize that this path is relative to the individual job working dir. Are there any other similar cases? Best, Rui On Mon, Nov 18, 2013 at 10:40 AM, Nate Coraor wrote: > On Sat, Nov 16, 2013 at 12:27 PM, ruiwang.sz wrote: > > Hi All, > > > > > > I‘m seeing some weird error messages...I googled but didn't see anything > > useful: > > > > > > So, it is during the wigToBigwig conversion: > > > > > > Dataset generation errors > > Dataset 47: Wig/BedGraph-to-bigWig on data 43 > > Tool execution generated the following error message: > > grep: writing output: Broken pipe > > grep: writing output: Broken pipe > > grep: writing output: Broken pipe > > and many more lines of the same error > > > > The tool produced the following additional output: > > Couldn't open tool-data/shared/ucsc/chrom/mm10.len , No such file or > > directory > > > > > > but it is there: > > > > > > ls -l tool-data/shared/ucsc/chrom/mm10.len > > > > -rw-rw-r-- 1 bioinfoadmin bioinfoadmin 1405 Oct 9 11:33 > > tool-data/shared/ucsc/chrom/mm10.len > > Hi Rui, > > Jobs run in a working directory based on the job id under whatever is > configured as job_working_directory in universe_wsgi.ini. Thus, from > that directory, tool-data/shared/ ... does not exist. To fix this, > set len_file_path in universe_wsgi.ini to an absolute path. > > --nate > > > > > > > From the Galaxy server log, I found these: > > > > > > galaxy.tools WARNING 2013-11-15 17:46:43,200 Failed to resolve > dependency on > > 'ucsc_tools', ignoring > > > > galaxy.jobs.runners.local DEBUG 2013-11-15 17:46:43,255 (216) executing: > > grep -v "^track" > > /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_458.dat | > > wigToBigWig stdin tool-data/shared/ucsc/chrom/mm10.len > > /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_473.dat > > -blockSize=256 -itemsPerSlot=1024 -clip 2>&1 || echo "Error running > > wigToBigWig." >&2 > > > > > > and then, setmetadata set the dataset state to ERROR. > > > > > > I did verify that the result dataset_473.dat has a size 0. > > > > > > However, when I run the command alone: > > > > > > grep -v "^track" > > /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_458.dat | > > wigToBigWig stdin tool-data/shared/ucsc/chrom/mm10.len > > /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_473.dat > > -blockSize=256 -itemsPerSlot=1024 -clip > > > > > > it successfully generated the dataset_473.dat. > > > > > > also, I look into the wig_to_Bigwig.xml in tools/filters, the command is: > > > > > > grep -v "^track" $input1 | wigToBigWig stdin $chromInfo $out_file1 > > > > > > however, I don't see where this "$chromInfo" was defined, yet it indeed > > found the > > > > the correct value: tool-data/shared/ucsc/chrom/mm10.len. how does that > > happen? > > > > > > it is very puzzling...I'm not sure if anyone has seen this before, please > > let me know! > > > > > > Thanks, > > > > Rui > > > > > > > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] wigToBigwig error
Hi All, I'm having an error at this: *Dataset 18: Wig/BedGraph-to-bigWig on data 12* Tool execution generated the following error message: grep: writing output: Broken pipe grep: writing output: Broken pipe grep: writing output: Broken pipe grep: writing output: Broken pipe .. put: Broken pipe grep: writing output: Broken pipe grep: writing output: Broken pipe .. grep: writing output: Broken pipe grep: write error Error running wigToBigWig. The tool produced the following additional output: hashMustFindVal: '1' not found I searched and found this link: http://redmine.soe.ucsc.edu/forum/index.php?t=msg&goto=10745&S=2a335135e76cf9b7160c0e9d41353767 which says that there is a naming convention difference. I followed what he did and replaced chrom=1 to chrom=chr1 etc, now it goes further, but still dies with error: line 18020168 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276760 line 18020169 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276770 line 18020170 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276780 line 18020171 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276790 line 18020172 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276800 line 18020173 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276810 line 18020174 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276820 line 18020175 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276830 line 18020176 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276840 line 18020177 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276850 line 18020178 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276860 line 18020179 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276870 line 18020180 of /tmp/t3: chromosome chr1 has 195276750 bases, but item ends at 195276880 Unrecognized line 18020181 of /tmp/t3: variableStep chr10 span=10 Error running wigToBigWig. The command line is: wigToBigWig /tmp/t3 /home/bioinfoadmin/app/galaxy-dist/tool-data/shared/ucsc/chrom/ucsc_gg4.len /home/bioinfoadmin/app/galaxy-dist/database/files/000/dataset_769.dat -clip 2>&1 || echo "Error running wigToBigWig." >&2 Quite puzzled...wondering if anyone has seen it before and could give me a hand. :-) I'll really appreciate! Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] wigToBigwig error
Hi Jennifer, Thanks for the note. A related question, do we have a 'ucsc_tool'? I sometimes saw the warning of failed dependency on ucsc_tool, but I'm not sure what it is. I found that there are many binary/script utilities: https://github.com/adamlabadorf/ucsc_tools should I install all these by running python setup.py install or I could just copy all those executables to my own utility dir? Thanks, Rui On Wed, Jan 15, 2014 at 7:51 AM, Jennifer Jackson wrote: > Hello Rui, > > The problems you are describing have to do with the format of the input > wig dataset. > > It looks as if you have corrected the chromosome names to be identical to > the reference genome build used (required). There are options to overcome > the other issues: > > 1. verify that your data has no browser lines, and has track and > definition lines in the correct format. UCSC is the definitive source for > this info, as both the underlying tool and this format were developed by > them. Links to their information and the general format rules in Galaxy can > be found here > https://wiki.galaxyproject.org/Learn/Datatypes#Wig_and_bigWig > > 2. send only a single wig file to the tool at a time, when using the > Galaxy wrapper. > > 3. use the 'full parameter' option 'Clip chromosome positions:' to > removing overhanging coordinates (known to be produced by several common > tools). This is after confirming that the build is correct - overhanging > coordinates can be a clue that there is a genome mismatch problem. > https://wiki.galaxyproject.org/Support#Reference_genomes > > 4. note that variable step data comes in two formats fixed and variable - > and that variable has two format versions, those with a a span definition > and those without (see examples in wig examples in #1). I have only seen > this tool run successfully, in Galaxy, on the type without "span" included. > If you find that the span variable is problematic after correcting any > other format issue issues, switch to the format without span. > > Good luck, > > Jen > Galaxy team > > > On 1/14/14 9:52 PM, ruiwang.sz wrote: > > Hi All, > > I'm having an error at this: > > *Dataset 18: Wig/BedGraph-to-bigWig on data 12* > > Tool execution generated the following error message: > > grep: writing output: Broken pipe > > grep: writing output: Broken pipe > > grep: writing output: Broken pipe > > grep: writing output: Broken pipe > > .. > > put: Broken pipe > > grep: writing output: Broken pipe > > grep: writing output: Broken pipe > > .. > > grep: writing output: Broken pipe > > grep: write error > > Error running wigToBigWig. > > The tool produced the following additional output: > > hashMustFindVal: '1' not found > > > I searched and found this link: > > > > http://redmine.soe.ucsc.edu/forum/index.php?t=msg&goto=10745&S=2a335135e76cf9b7160c0e9d41353767 > > > which says that there is a naming convention difference. > > > I followed what he did and replaced chrom=1 to chrom=chr1 etc, now it > goes further, but > > still dies with error: > > > line 18020168 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276760 > > line 18020169 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276770 > > line 18020170 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276780 > > line 18020171 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276790 > > line 18020172 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276800 > > line 18020173 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276810 > > line 18020174 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276820 > > line 18020175 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276830 > > line 18020176 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276840 > > line 18020177 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276850 > > line 18020178 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276860 > > line 18020179 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276870 > > line 18020180 of /tmp/t3: chromosome chr1 has 195276750 bases, but item > ends at 195276880 > > Unrecognized line 18020181 of /tmp/t3: > > variableStep chr10 span=10 > > > Error running wigToBigWig. > > > The command line is: wigToBigWig /tmp/t3 > /home/bioinfoadmin/app/galaxy-dist/tool-data/shared/ucsc/
[galaxy-dev] a question regarding galaxy 'workflow'
Hi All, A quick question on the indices. Here is the information provided by the sample file: #Your bowtie_indices.loc file should include an entry per line for each #index set you have stored. The "file" in the path does not actually #exist, but it is the prefix for the actual index files. For example: #hg18canon hg18 hg18 Canonical /depot/data2/galaxy/bowtie/hg18/hg18canon #hg18full hg18 hg18 Full /depot/data2/galaxy/bowtie/hg18/hg18full #/orig/path/hg19hg19 hg19 /depot/data2/galaxy/bowtie/hg19/hg19 #Note that for backwards compatibility with workflows, the unique ID of #an entry must be the path that was in the original loc file, because that #is the value stored in the workflow for that parameter. That is why the #hg19 entry above looks odd. New genomes can be better-looking. What exactly is "path that was in the original loc file"? Where do I find this "original loc file"? Are we using a newer version of loc file now? I got the loc file from the main galaxy server, and I could see things like following: ce5 ce5 C. elegans (WS180): ce5 /galaxy/data/ce5/bowtie_index/ce5 /galaxy/data/ce6/bowtie_index/ce6 ce6 C. elegans (WS190): ce6 /galaxy/data/ce6/bowtie_index/ce6 Why would ce5 and ce6 appear differently? for ce6, seems the "original loc file path" is the same as the "filebasepath"? galGal3canongalGal3 Chicken (Gallus gallus): galGal3 Canonical /galaxy/data/galGal3/galGal3canon/bowtie_index/galGal3canon /galaxy/data/galGal3/bowtie_index/galGal3 galGal3 Chicken (Gallus gallus): galGal3 Full /galaxy/data/galGal3/galGal3full/bowtie_index/galGal3full but as we could see, for this galGal3, the "original loc file path" is different from the "filebasepath". same for mm9: mm9canonmm9 Mouse (Mus musculus): mm9 Canonical /galaxy/data/mm9/mm9canon/bowtie_index/mm9canon mm9female mm9 Mouse (Mus musculus): mm9 Canonical Female /galaxy/data/mm9/mm9female/bowtie_index/mm9female /galaxy/data/mm9/bowtie_index/mm9 mm9 Mouse (Mus musculus): mm9 Full /galaxy/data/mm9/mm9full/bowtie_index/mm9full How did this get determined? My conjecture is that in the older version of galaxy we don't have mm9canon and mm9female, for example, but we had mm9, and the entry was like: mm9 mm9mm9/galaxy/data/mm9/bowtie_index/mm9 but now we are using a newer version of galaxy and mm9 genome was placed in /galaxy/data/mm9/mm9full/bowtie_index/mm9full so we need to modify this loc file to add the mapping so that we don't have to modify the older workflow that uses mm9? is it just because data is organized differently? that's why for all the newer genomes we don't have that? I'd appreciate that if someone could please elaborate a bit. Best Regards, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] universe_wsgi.ini question for Cistrome
Hi Guys, Are there anyone who is using Cistrome? I tried to merge Cistrome's setting into our own galaxy instance. A new entry is # Path to the static library files for assembly, ceaslib, chromLen, conservation, liftOver and MAT-lib # Then in other tool configuration xml file, we can use 'from galaxy import config' then '$config.Configuration().cistrome_static_library_path' # default would be in tool-data/ folder cistrome_static_library_path = /home/bioinfoadmin/app/cistrome-apps/cistrome_library However, I got error when I tried CEAS: Enrichment on chromosome and annotation: * Dataset 23: CEAS: Enrichment on chromosome and annotation on data 16 and data 13 The Galaxy framework encountered the following error while attempting to run the tool: Traceback (most recent call last): File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/jobs/runners/__init__.py", line 121, in prepare_job job_wrapper.prepare() File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/jobs/__init__.py", line 707, in prepare config_filenames = self.tool.build_config_files( param_dict, self.working_directory ) File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/tools/__init__.py", line 2609, in build_config_files f.write( fill_template( template_text, context=param_dict ) ) File "/home/bioinfoadmin/app/galaxy-dist/lib/galaxy/util/template.py", line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File "/home/bioinfoadmin/app/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py", line 1004, in __str__ return getattr(self, mainMethName)() File "cheetah_DynamicallyCompiledCheetahTemplate_1391037616_15_43258.py", line 274, in respond NotFound: cannot find 'cistrome_static_library_path' while searching for '__app__.config.cistrome_static_library_path' Tool execution generated the following error message: failure preparing job I'm quite confused since '__app__.config.cistrome_static_library_path' should be defined? or right now __app__ is dropped as Galaxy evolves? Besides, what does the comment mean by # Then in other tool configuration xml file, we can use 'from galaxy import config' then '$config.Configuration().cistrome_static_library_path' Does that mean that we should not be using __app__.config...but how to use this? I tried to insert 'from galaxy import config' into the xml file but only got error... I'll appreciate any input. Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] upload error
Hi Guys,
Our galaxy instance has been in use for a couple years but since Feb 2014
it started to show us some weird behavior. Recently the upload function
suddenly stopped working properly. We are getting messages like the
following. I'm not sure if this only happens to us. If someone has seen
this before, please give me some hints.
P.S., I searched online and many says that this is because the browser does
not wait till all the data from the server is received and closes the
socket. However it happens both in firefox and chrome. I'm wondering if it
is because galaxy has any change recently but when I do 'hg update stable'
it reported that it is up to date.
Thanks!
Rui
Error messages:
Exception happened during processing of request from
('127.0.0.1'
Exception happened during processing of request from (, 52259'127.0.0.1',
52260)
Traceback (most recent call last):
)
Traceback (most recent call last):
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
Exception happened during processing of request from
(
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
Exception happened during processing of request from'127.0.0.1' ,
('127.0.0.1'52264, 52262
)
Exception happened during processing of request from
('127.0.0.1'
, )
Traceback (most recent call last):
Traceback (most recent call last):
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
Exception happened during processing of request from 52261('127.0.0.1',
)Exception happened during processing of request from
Traceback (most recent call last):
52263('127.0.0.1')
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
Traceback (most recent call last):
, 52265)
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
Traceback (most recent call last):
File
"/home/bioinfoadmin/app/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpserver.py",
line 1068, in process_request_in_thread
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.finish_request(request, client_address)
self.finish_request(request, client_address)
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
self.finish_request(request, client_address)
File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request
127.0.0.1 - - [27/Feb/2014:16:59:36 -0700] "GET
/tool_runner?tool_id=upload1 HTTP/1.1" 200 - "http://localhost:20020/root";
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:27.0) Gecko/20100101
Firefox/27.0"
self.RequestHandlerClass(request, client_address, self)
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.RequestHandlerClass(request, client_address, self)
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.finish()
self.RequestHandlerClass(request, client_address, self)
self.finish()
self.RequestHandlerClass(request, client_address, self)
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib/python2.7/SocketServer.py", line 693, in finish
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__
self.finish()
self.finish()
File "/usr/lib/python2.7/SocketServer.py", line 693, in finis
[galaxy-dev] tool_shed question on multiple tool versions
Hi Guys, Sorry for the duplicate posting...nobody saw my original one seems and I'm still wondering what the proper way is. I searched around the wiki but didn't find much. Originally, I installed macs dependency for galaxy myself. The version is 1.4.2. It worked well. Now, after galaxy upgrading removed it, and I installed it back from tool_shed, things broke. In the galaxy tool panel, it looks the same like before(as expected), but when I click on macs, it will show an option of MACS 'version 1.0.0' and 'version 1.0.1'. Be default it is 1.0.1, which is defined in migrated_tools_conf.xml: toolshed.g2.bx.psu.edu macs devteam ae2ec275332a toolshed.g2.bx.psu.edu/repos/devteam/macs/peakcalling_macs/1.0.1 1.0.1 if we dive in this file toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs/macs_wrapper.xml, we could see that in this dir /home/bioinfoadmin/app/shed_tools/ toolshed.g2.bx.psu.edu/repos/devteam/macs/ae2ec275332a/macs, there is a tool_dependency xml file that has following content: http://toolshed.g2.bx.psu.edu"; /> http://toolshed.g2.bx.psu.edu"; /> These versions are both older than what we installed before(1.4.2 and 3.0.2). could we manually modify that to use our own? wouldn't that be a bad practice? while in the same dir, macs_wrapper.xml has: macs R Seems that this is the tool_shed version of macs(which is identical to the one on main galaxy server, I think?). However this one does not work with our input data. Additionally, when I click to switch from 1.0.1 to 1.0.0 of macs, it immediately reports error. So, what if I want to keep using the one I installed myself? I know that tool_shed is a cleaner way to manage tools, but in our situation the upgrade and elimination actually broke the thing that worked before. I want to ask the proper way to handle this kind of situation. could I simply drop the entry from migrated_tools_conf.xml about macs, and restore the old tools_conf.xml entry? in that way we could continue using the 1.4.2 we have. However next time when we do the upgrade, things will break again I guess? I think that I shouldn't be the only person that has this concern? how to handle multiple version/dependency version for a tool? is there an option to not use the tool_shed but keep own version? Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] how to manage multiple tool versions
Hi Guys, I'm sorry to post multiple times for the same question. Now I removed all the specifics of my problem and I only want someone to point me to the right documentation. Admins, I know you are busy, but don't you want more people to benefit from this platform? When galaxy upgraded it started using the tool_shed version of the tool and killed the tools we installed ourselves. Right now we have both versions. How should we manage this situation properly if we don't want the tool_shed version and want to keep using what we had before? Nate said that MACS came with galaxy before. I didn't know that. From the day we started using MACS it was installed by ourselves. Not sure if that caused the problem. Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] how to manage multiple tool versions
Hi Bjoern, Thanks for the note, and sorry I could not type your name in the original alphabet -- it looks Icelandic. :-) I'll follow your instructions and see how it goes. Best, Rui On Fri, Mar 7, 2014 at 7:17 PM, Björn Grüning wrote: > Hi Rui, > > if you want to use only the local installation and not the toolshed one, > go to the toolshed installation inside your galaxy and remove under "manage > tool dependencies" the dependency on macs2. As soon as you get rid of the > dependency the local installation is used. > > Cheers, > Bjoern > > P.S. There is also a MACS2 version now for testing > > Am 08.03.2014 00:48, schrieb ruiwang.sz: > >> Hi Guys, >> >> I'm sorry to post multiple times for the same question. Now I removed all >> the specifics of my problem and I only want someone to point me to the >> right documentation. Admins, I know you are busy, but don't you want more >> people to benefit from this platform? >> >> When galaxy upgraded it started using the tool_shed version of the tool >> and >> killed the tools we installed ourselves. Right now we have both versions. >> How should we manage this situation properly if we don't want the >> tool_shed >> version and want to keep using what we had before? >> >> Nate said that MACS came with galaxy before. I didn't know that. From the >> day we started using MACS it was installed by ourselves. Not sure if that >> caused the problem. >> >> Thanks, >> Rui >> >> >> >> ___ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >>http://lists.bx.psu.edu/ >> >> To search Galaxy mailing lists use the unified search at: >>http://galaxyproject.org/search/mailinglists/ >> >> ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] upload error
Hi John, Thanks for the note. Please see inline. On Fri, Mar 7, 2014 at 7:18 PM, John Chilton wrote: > Hmm... I am not certain but it sounds like the migration script ripped > something out of your Galaxy tool panel that you did not want it to? This > is unfortunate - but I don't think it would happen again now that the > migration process is complete. Some future update to Galaxy may include > additional migrations scripts but now that macs is out it won't ever be > migrated again. imagine you can just update your tool_conf.xml file to > point to your existing macs wrapper you set up manually and you can either > leave the devteam version of macs in place or uninstall it if you prefer > your own. Is it not this easy? > Yes that's what I thought, but I was worrying that if I remove devteam version of macs from the migrated_tool.conf, I might break something potentially useful for galaxy. Mainly, I try not to manually change things since that would not be maintainable over time. That's why I was asking how to handle this properly. It's great to have your confirmation that it works this way. :-) Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] nginx configuration question
Hi All,
I have been trying to get the visualization through ucsc main working, thus
I installed the nginx on our ubuntu(the one comes with update), then I
configured it according to the wiki
https://wiki.galaxyproject.org/Admin/Config/nginxProxy
Here is the section in my nginx.conf
include /etc/nginx/conf.d/*.conf;
#include /etc/nginx/sites-enabled/*;
upstream galaxy_app {
server localhost:20020;
}
server {
client_max_body_size 10G;
# ... other server stuff ...
location / {
proxy_pass http://galaxy_app;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-For
$proxy_add_x_forwarded_for;
}
}
server {
location /_x_accel_redirect/ {
internal;
alias /;
}
}
A couple notes:
1. I commented out
#include /etc/nginx/sites-enabled/*;
should I?
2. I have two sections of 'server', I am not sure if this is correct, but
the wiki does not talk about it.
I also have the following in universe_wsgi.ini:
nginx_x_accel_redirect_base = /_x_accel_redirect
Now, if I go to http://127.0.0.1, it will redirect to http://127.0.0.1:20020,
our galaxy instance, which is right.
However, when i choose 'display in UCSC main' on a bigwig file, it failed
to show and here is the log:
75.142.102.9 - - [16/May/2014:09:18:20 -0700] "GET
/display_application/360b40e91c77a7e9/ucsc_bigwig/main HTTP/1.0" 302 - "
http://128.125.28.215/root"; "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137 Safari/537.36"
128.114.119.135 - - [16/May/2014:09:18:20 -0700] "GET
/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/param/track
HTTP/1.0" 200 201 "-" "genome.ucsc.edu/net.c"
128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
HTTP/1.0" 200 92543742 "-" "genome.ucsc.edu/net.c"
128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
/_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
/galaxy/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
I don't know how this works -- the dataset I clicked is in fact:
/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
and it is on disk. However, it gave 404:
128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
/_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
seems '_x_accel_redirect' is not substituted by the redirect alias '/'? but
why?
I checked the permission and it is 775 for all the dir on path, and 664 for
dataset_3353.dat
so www-data user has the read access.
Also, where is this dir and file?
/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
I don't see them anywhere on the disk at all!
I'm very confused about this. Could anyone give me a hand? I'll really
appreciate!
Thanks,
Rui
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] nginx configuration question
Hi Guys,
I did a little further investigation. I found that the order of 2 server
blocks matter in my case.
If I put the /_x_accel_redirect/ block first, then the 2nd proxy server
block is not effective.
If in the order I pasted in the previous message, then the
_x_accel_redirect block is not effective.
But why is this? btw I'm using the 1.1.19 version of nginx that comes with
ubuntu.
Could anyone give me a hint? I'll be really grateful.
Thanks,
Rui
On Fri, May 16, 2014 at 12:29 PM, ruiwang.sz wrote:
> Hi All,
>
> I have been trying to get the visualization through ucsc main working,
> thus I installed the nginx on our ubuntu(the one comes with update), then I
> configured it according to the wiki
>
> https://wiki.galaxyproject.org/Admin/Config/nginxProxy
>
> Here is the section in my nginx.conf
>
> include /etc/nginx/conf.d/*.conf;
> #include /etc/nginx/sites-enabled/*;
>
> upstream galaxy_app {
> server localhost:20020;
> }
> server {
> client_max_body_size 10G;
> # ... other server stuff ...
> location / {
>proxy_pass http://galaxy_app;
>proxy_set_header X-Forwarded-Host $host;
>proxy_set_header X-Forwarded-For
> $proxy_add_x_forwarded_for;
>}
>}
>server {
> location /_x_accel_redirect/ {
> internal;
> alias /;
> }
> }
>
> A couple notes:
>
> 1. I commented out
>
> #include /etc/nginx/sites-enabled/*;
>
> should I?
>
> 2. I have two sections of 'server', I am not sure if this is correct, but
> the wiki does not talk about it.
>
> I also have the following in universe_wsgi.ini:
>
> nginx_x_accel_redirect_base = /_x_accel_redirect
>
> Now, if I go to http://127.0.0.1, it will redirect to
> http://127.0.0.1:20020, our galaxy instance, which is right.
> However, when i choose 'display in UCSC main' on a bigwig file, it failed
> to show and here is the log:
>
> 75.142.102.9 - - [16/May/2014:09:18:20 -0700] "GET
> /display_application/360b40e91c77a7e9/ucsc_bigwig/main HTTP/1.0" 302 - "
> http://128.125.28.215/root"; "Mozilla/5.0 (Macintosh; Intel Mac OS X
> 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137
> Safari/537.36"
> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "GET
> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/param/track
> HTTP/1.0" 200 201 "-" "genome.ucsc.edu/net.c"
> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
> HTTP/1.0" 200 92543742 "-" "genome.ucsc.edu/net.c"
> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
> /_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
> /galaxy/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
>
> I don't know how this works -- the dataset I clicked is in fact:
>
> /home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
>
> and it is on disk. However, it gave 404:
>
> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
> /_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
>
> seems '_x_accel_redirect' is not substituted by the redirect alias '/'?
> but why?
>
> I checked the permission and it is 775 for all the dir on path, and 664
> for dataset_3353.dat
>
> so www-data user has the read access.
>
> Also, where is this dir and file?
>
>
> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
>
> I don't see them anywhere on the disk at all!
>
> I'm very confused about this. Could anyone give me a hand? I'll really
> appreciate!
>
> Thanks,
> Rui
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] nginx configuration question
Hi Guys,
I just switch to 1.6.0 of nginx, but the behavior is the same. I also tried
the
apache proxy instructions, but rewrite just didn't work. I searched around,
did
what I could, verified that I did have the required module, etc, but still
didn't
work -- it is so frustrating.
For nginx, at least redirect works, but I don't know why there is only one
server
block in use at a time. How do I solve this? Any help would be greatly
appreciated.
Thanks,
Rui
On Sat, May 17, 2014 at 7:36 PM, ruiwang.sz wrote:
> Hi Guys,
>
> I did a little further investigation. I found that the order of 2 server
> blocks matter in my case.
>
> If I put the /_x_accel_redirect/ block first, then the 2nd proxy server
> block is not effective.
> If in the order I pasted in the previous message, then the
> _x_accel_redirect block is not effective.
>
> But why is this? btw I'm using the 1.1.19 version of nginx that comes with
> ubuntu.
>
> Could anyone give me a hint? I'll be really grateful.
>
> Thanks,
> Rui
>
>
> On Fri, May 16, 2014 at 12:29 PM, ruiwang.sz wrote:
>
>> Hi All,
>>
>> I have been trying to get the visualization through ucsc main working,
>> thus I installed the nginx on our ubuntu(the one comes with update), then I
>> configured it according to the wiki
>>
>> https://wiki.galaxyproject.org/Admin/Config/nginxProxy
>>
>> Here is the section in my nginx.conf
>>
>> include /etc/nginx/conf.d/*.conf;
>> #include /etc/nginx/sites-enabled/*;
>>
>> upstream galaxy_app {
>> server localhost:20020;
>> }
>> server {
>> client_max_body_size 10G;
>> # ... other server stuff ...
>> location / {
>>proxy_pass http://galaxy_app;
>>proxy_set_header X-Forwarded-Host $host;
>>proxy_set_header X-Forwarded-For
>> $proxy_add_x_forwarded_for;
>>}
>>}
>>server {
>> location /_x_accel_redirect/ {
>> internal;
>> alias /;
>> }
>> }
>>
>> A couple notes:
>>
>> 1. I commented out
>>
>> #include /etc/nginx/sites-enabled/*;
>>
>> should I?
>>
>> 2. I have two sections of 'server', I am not sure if this is correct, but
>> the wiki does not talk about it.
>>
>> I also have the following in universe_wsgi.ini:
>>
>> nginx_x_accel_redirect_base = /_x_accel_redirect
>>
>> Now, if I go to http://127.0.0.1, it will redirect to
>> http://127.0.0.1:20020, our galaxy instance, which is right.
>> However, when i choose 'display in UCSC main' on a bigwig file, it failed
>> to show and here is the log:
>>
>> 75.142.102.9 - - [16/May/2014:09:18:20 -0700] "GET
>> /display_application/360b40e91c77a7e9/ucsc_bigwig/main HTTP/1.0" 302 - "
>> http://128.125.28.215/root"; "Mozilla/5.0 (Macintosh; Intel Mac OS X
>> 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137
>> Safari/537.36"
>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "GET
>> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/param/track
>> HTTP/1.0" 200 201 "-" "genome.ucsc.edu/net.c"
>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
>> HTTP/1.0" 200 92543742 "-" "genome.ucsc.edu/net.c"
>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>> /_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
>> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>> /galaxy/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
>> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
>>
>> I don't know how this works -- the dataset I clicked is in fact:
>>
>> /home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
>>
>> and it is on disk. However, it gave 404:
>>
>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>> /_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
>> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
>>
>> seems '_x_accel_redirect' is n
Re: [galaxy-dev] nginx configuration question
Just to answer myself...
Those 2 location blocks should be in the same server block!
Hope this could help someone else...
:)
Rui
On Sat, May 17, 2014 at 8:54 PM, ruiwang.sz wrote:
> Hi Guys,
>
> I just switch to 1.6.0 of nginx, but the behavior is the same. I also
> tried the
> apache proxy instructions, but rewrite just didn't work. I searched
> around, did
> what I could, verified that I did have the required module, etc, but still
> didn't
> work -- it is so frustrating.
>
> For nginx, at least redirect works, but I don't know why there is only one
> server
> block in use at a time. How do I solve this? Any help would be greatly
> appreciated.
>
> Thanks,
> Rui
>
>
> On Sat, May 17, 2014 at 7:36 PM, ruiwang.sz wrote:
>
>> Hi Guys,
>>
>> I did a little further investigation. I found that the order of 2 server
>> blocks matter in my case.
>>
>> If I put the /_x_accel_redirect/ block first, then the 2nd proxy server
>> block is not effective.
>> If in the order I pasted in the previous message, then the
>> _x_accel_redirect block is not effective.
>>
>> But why is this? btw I'm using the 1.1.19 version of nginx that comes
>> with ubuntu.
>>
>> Could anyone give me a hint? I'll be really grateful.
>>
>> Thanks,
>> Rui
>>
>>
>> On Fri, May 16, 2014 at 12:29 PM, ruiwang.sz wrote:
>>
>>> Hi All,
>>>
>>> I have been trying to get the visualization through ucsc main working,
>>> thus I installed the nginx on our ubuntu(the one comes with update), then I
>>> configured it according to the wiki
>>>
>>> https://wiki.galaxyproject.org/Admin/Config/nginxProxy
>>>
>>> Here is the section in my nginx.conf
>>>
>>> include /etc/nginx/conf.d/*.conf;
>>> #include /etc/nginx/sites-enabled/*;
>>>
>>> upstream galaxy_app {
>>> server localhost:20020;
>>> }
>>> server {
>>> client_max_body_size 10G;
>>> # ... other server stuff ...
>>> location / {
>>>proxy_pass http://galaxy_app;
>>>proxy_set_header X-Forwarded-Host $host;
>>>proxy_set_header X-Forwarded-For
>>> $proxy_add_x_forwarded_for;
>>>}
>>>}
>>>server {
>>> location /_x_accel_redirect/ {
>>> internal;
>>> alias /;
>>> }
>>> }
>>>
>>> A couple notes:
>>>
>>> 1. I commented out
>>>
>>> #include /etc/nginx/sites-enabled/*;
>>>
>>> should I?
>>>
>>> 2. I have two sections of 'server', I am not sure if this is correct,
>>> but the wiki does not talk about it.
>>>
>>> I also have the following in universe_wsgi.ini:
>>>
>>> nginx_x_accel_redirect_base = /_x_accel_redirect
>>>
>>> Now, if I go to http://127.0.0.1, it will redirect to
>>> http://127.0.0.1:20020, our galaxy instance, which is right.
>>> However, when i choose 'display in UCSC main' on a bigwig file, it
>>> failed to show and here is the log:
>>>
>>> 75.142.102.9 - - [16/May/2014:09:18:20 -0700] "GET
>>> /display_application/360b40e91c77a7e9/ucsc_bigwig/main HTTP/1.0" 302 - "
>>> http://128.125.28.215/root"; "Mozilla/5.0 (Macintosh; Intel Mac OS X
>>> 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.137
>>> Safari/537.36"
>>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "GET
>>> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/param/track
>>> HTTP/1.0" 200 201 "-" "genome.ucsc.edu/net.c"
>>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>>> /display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e91c77a7e9.bigwig
>>> HTTP/1.0" 200 92543742 "-" "genome.ucsc.edu/net.c"
>>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>>> /_x_accel_redirect/home/bioinfoadmin/app/galaxy-dist/database/files/003/dataset_3353.dat
>>> HTTP/1.0" 404 - "-" "genome.ucsc.edu/net.c"
>>> 128.114.119.135 - - [16/May/2014:09:18:20 -0700] "HEAD
>>> /galaxy/display_application/360b40e91c77a7e9/ucsc_bigwig/main/6e1547da5bd290c9/data/galaxy_360b40e
[galaxy-dev] error: flexbar dependency is not installed by the toolshed
Hi Guys, I tried to install flexbar from toolshed, but upon running, error said it could not find it. I checked and found toolshed.g2.bx.psu.edu/repos/jtilman/flexbar/flexbar/2.4 in the config file, but this path does not exist. Is it that I need to install the dependency on my own? I had the impression that toolshed will do it for me. Please correct me if I'm wrong. Thanks, Rui ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
