Re: [galaxy-dev] Unable to add R package dependency to tool_dependencies.xml
Hi, Bruno. As Björn points out, cross-toolshed dependencies aren't currently supported but exporting (eg) package_r_3_1_1 as a capsule (make sure to include ALL dependencies) from one toolshed and importing that capsule into your own local toolshed brings all the sub-dependent packages so they exist locally. One trick: if the toolshed URL is NOT specified in a package reference in tool_dependencies.xml, the local toolshed will be sourced as the default. Not sure if this makes much sense, but it works for me developing on my laptop's toolshed with lots of packages that came through capsules from the main and other toolsheds. On Sat, Nov 1, 2014 at 10:49 AM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Bruno, it is currently not possible to specify dependencies from other Tool Sheds. So you can only depend on packages that are in the same Tool Shed as your tool. Is this your main issue? Greg has some nice blog posts about the Tool Shed and how to set up a local test Tool Shed with other packages, like the one from foobar. http://gregvonkuster.org/ Cheers, Bjoern I'm attempting to create a tool dependency definition file for ExomeCNV, which notably depends on R (I'm trying to use the package_r_3_1_1 repository). However, I can't seem to set up the dependency properly. See attached for my tool_dependencies.xml. Also, see here http://cl.ly/YK3C for the error message when I attempt to upload the tool_dependencies.xml file to the Tool Shed. One of the possible issues is that the toolshed repository tag attribute for specifying an external Tool Shed isn't being considered. This is being done in a local Tool Shed installation that doesn't include the package_r_3_1_1 repository. Therefore, I must refer to the Test Tool Shed. This is inspired by a tool definition https://github.com/bgruening/galaxytools/blob/78f35bc97160bdfe4783c9ac2e017e9fe55e041c/packages/package_deseq2_1_0_17/tool_dependencies.xml written by Björn and the repository tag description https://wiki.galaxyproject.org/ToolDependenciesTagSets. Best regards, Bruno ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] strange issue with .RData files
Rdata is binary and serialises R objects so I sure hope you don't have to
peek inside - probably needs most of an R environment - like rpy or
something.
A binary header signature magic number would be ideal - I checked a few
saved rdata files lying around here and all seemed to start with the
following bytes - all were variable after the 12th:
1f 8b 08 00 00 00 00 00 00 03 d4 fd
Maybe someone else can confirm that as a reliable binary hex header
signature for rdata?
I couldn't find anything in the R docs - probably take a good deep dive
into the guts of the save/load function source to be sure.
On Thu, Oct 30, 2014 at 8:54 PM, Lukasse, Pieter pieter.luka...@wur.nl
wrote:
Hi John,
Yes, I think this should work as I have seen it work for another binary
type I made before. See below:
class FileSet( Binary ):
FileSet containing N files
file_ext = prims.fileset.zip
blurb = (zipped) FileSet containing multiple files
def sniff( self, filename ):
# If the zip file contains multiple files then return true, false
otherwise:
zf = zipfile.ZipFile(filename)
if (len(zf.infolist())1):
return True
else :
return False
# the if is just for backwards compatibility...could remove this at some
point
if hasattr(Binary, 'register_sniffable_binary_format'):
Binary.register_sniffable_binary_format('FileSet',
'prims.fileset.zip', FileSet)
Now the question I have is: what would be a good logic to use in the sniff
method? I need something that uniquely distinguishes this zipped file from
other zip files, right? In the previous example above I found a solution by
checking whether the zip file has multiple files inside and return true if
this is the case. Now with RData, does it mean I have to try to parse the
binary contents inside and come with a good heuristic/rule ? Just wondering
if someone already has thought about such a rule, specifically for RData.
Thanks,
Pieter.
-Original Message-
From: John Chilton [mailto:jmchil...@gmail.com]
Sent: donderdag 23 oktober 2014 3:02
To: Lukasse, Pieter
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] strange issue with .RData files
Hey Pieter,
Sorry I am swamped right now so I don't have time to dig into this in
detail - but I have encountered this before with datatypes that are
compressed - zipped, gzipped, etc Galaxy will attempt to decompress
them in order to figure out what they are. I believe this is what is
happening to your data. If you register the type as a sniffable binary it
looks like it should skip the decompression though
- unless I am reading this logic wrong in tools/data_source/upload.py
https://gist.github.com/jmchilton/54b5d7485fcd16eec984.
E.g. like bam datatypes:
class Bam( Binary ):
Binary.register_sniffable_binary_format(bam, bam, Bam)
Have you registered a sniffable binary datatype for RData?
-John
On Wed, Oct 22, 2014 at 9:38 AM, Lukasse, Pieter pieter.luka...@wur.nl
wrote:
Hi,
When I upload any .RData file to my Galaxy server it seems to be
unpacking/changing it. The resulting file in my history is different
and around 2x larger than the uploaded file. The tool that needs to
use it also aborts with an error due to this erroneous file.
What are the workarounds?
Thanks,
Pieter Lukasse
Wageningen UR, Plant Research International
Department of Bioinformatics (Bioscience)
Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB,
Wageningen, the Netherlands
T: +31-317481122;
M: +31-628189540;
skype: pieter.lukasse.wur
http://www.pri.wur.nl
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this and other
Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Can't change working directory during tool dependency installation
You might need to specify multiple command lines in a single shell action - separated by ; like: cd foo; perl foo bah; more something; cleanup Otherwise you might just find yourself back in the temp installation directory after each separate action Ross Lazarus Head, Computational Biology, Baker IDI, Melbourne, Australia Pubs: http://scholar.google.com/citations?hl=enuser=UCUuEM4J On Sat, Oct 25, 2014 at 4:10 PM, Bruno Grande bgra...@sfu.ca wrote: I originally posted this question https://biostar.usegalaxy.org/p/9401/ on the Galaxy Biostars, but later realized that galaxy-dev is probably more appropriate. So, here it goes. I'm trying to automate the installation of a tool's dependency with a tool dependency definition repository. During the package installation process, after moving the contents of a downloaded archive to the installation directory (*i.e.* $INSTALL_DIR), I simply want to change to that target directory away from the initial temporary working directory. However, I can't seem to change directory, either with a change_directory action or a cd shell_command action. Running pwd before and after attempting to change directory confirms this (see attached for installation log). I even made sure that $INSTALL_DIR is different from my initial working directory. Best regards, Bruno ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] tool for STAR RNA-seq aligner
Hi David. I've not needed that workflow so haven't a solution for you and no, it doesn't do anything with chimeric output - won't be hard to add I suspect. There's no python wrapper - just shell script in the command segment. It's not in an IUC main tool shed repository because it lacks a data manager - manual star indexes are a bit of a pain but less pain than writing a data manager :( so I haven't yet. Might be run best through the API. On shared memory: Pity. it works a treat for us. I didn't see anything on the google group - do you recall where you learned about this deprecation ? On Thu, Sep 25, 2014 at 10:41 PM, David Hoover hoove...@helix.nih.gov wrote: Ross, About the index files: It is way easier to have pre-built index files. However, when running a 2-pass STAR run, a user will need to generate their own reference index files based on the output SJ.tab.out file created in the first pass. Is this incorporated into your tool? About shared memory: I am under the impression that the latest version of STAR has deprecated this feature. I am unclear how this would help unless a single large-memory machine was dedicated to running all STAR jobs. Is this the case? Also, does the tool merge the SAM/BAM file with the output chimeric SAM file? David Hoover On 9/24/2014 7:03 PM, Ross wrote: Hi All, That (fubar in testtoolshed) star wrapper was derived from one originally written by Jeremy Goecks. I modified it for multiple inputs and added a few tweaks and it has been in production use in our group for about 6 months so I'm pretty sure it works reasonably well in our hands at least. I would really appreciate any available help getting it to a proven useful state - suggestions and code welcomed. I have not moved it to the main toolshed because aside from some encouragement, I've had no feedback to suggest it's working - or not. It is extremely fast - we regularly see 200-300M reads per minute in the logs! We regularly run a whole experiment worth (eg 12 - 24) fastq files simultaneously with the shared memory option working on our cluster - see the readme. Star index files made with a gene model (requires valid gff3) are huge - 20-30GB for hg19 - hence the need for shared memory if you run multiple jobs. That will eventually become a serious problem if you really want to allow users to make their own - we definitely do not. You need to be very careful about matching the gene model gff3 file to the reference and I had enough trouble getting it right for the few major genomes we use to make me think that I do not want users trying to do that generating 25GB of rubbish every time they get it wrong. There are challenges to do with needing different indexes for different length reads but we are seeing fairly consistent 60bp single ended reads for most of the incoming RNA seq experiments. A data manager would be a boon if anyone cares to write one... On Thu, Sep 25, 2014 at 6:55 AM, Curtis Hendrickson (Campus) curt...@uab.edu wrote: Bjorn We'd be interested in this tool, as well. Any idea how close to functional it is? I see it's only on TEST toolshed, and not on production, at this point. I don't see any related Trello card when searching on star Regards, Curtis Galaxy Admin @ University of Alabama at Birmingham -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto: galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Björn Grüning Sent: Wednesday, September 24, 2014 3:15 PM To: galaxy-dev@lists.bx.psu.edu; hoove...@helix.nih.gov David Hoover Subject: Re: [galaxy-dev] tool for STAR RNA-seq aligner Hi David, yes there is inital code in the https://testtoolshed.g2.bx.psu.edu/. I think Ross has done some work on it. The main problem with Star is that is needs special indices (and a lot of it) and it would be great to offer data managers for it. Cheers, Bjoern Am 24.09.2014 um 22:05 schrieb David Hoover: Hi, I am developing a tool for STAR (https://code.google.com/p/rna-star/), and I realize I may be reinventing another wheel. Has anyone else created a tool for STAR? There's nothing else in the toolsheds for it yet. David David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu
Re: [galaxy-dev] tool for STAR RNA-seq aligner
Hi David, -1 on the composite datatype for star reference genomes idea Let's use the existing reference genome and index file infrastructure - star is not sufficiently different from bwa or bowtie to warrant anything different IMHO - it works for us but generating the index files manually is a pain - best to write a data manager. Ross Lazarus Head, Computational Biology, Baker IDI, Melbourne, Australia Pubs: http://scholar.google.com/citations?hl=enuser=UCUuEM4J On Thu, Sep 25, 2014 at 6:39 AM, David Hoover hoove...@helix.nih.gov wrote: Why didn't I see these before? Hmm, I thought I had searched both toolsheds... I was kind of hoping someone had tackled this in a different way. It would be nice if there was a composite datatype for the reference genome. It is important for users to generate their own personal genome references, rather than rely on shared, admin-installed indices. And you're correct, we'd need at least 5-10 separate genome references for each organism, depending on read length and annotation GTF. Back to wheel reinvention. BTW, can you tell me which standardly installed tools use composite datatypes? It's always easier to build thing from comparison, rather than from scratch. David David Hoover, PhD Helix Systems Staff On Sep 24, 2014, at 4:14 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi David, yes there is inital code in the https://testtoolshed.g2.bx.psu.edu/. I think Ross has done some work on it. The main problem with Star is that is needs special indices (and a lot of it) and it would be great to offer data managers for it. Cheers, Bjoern Am 24.09.2014 um 22:05 schrieb David Hoover: Hi, I am developing a tool for STAR (https://code.google.com/p/rna-star/), and I realize I may be reinventing another wheel. Has anyone else created a tool for STAR? There's nothing else in the toolsheds for it yet. David David Hoover, PhD Helix Systems Staff SCB/DCSS/CIT/NIH 301-435-2986 http://helix.nih.gov ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dependencies within dependencies
Hi Christophe - there's a trello card for this at https://trello.com/c/ClgcV0Sn - upvote and add your experiences please? On Wed, Sep 17, 2014 at 8:01 AM, Christophe Antoniewski christophe.antoniew...@snv.jussieu.fr wrote: I am trying to move the package_r_3_0_3 repository owned by devteam from the main toolshed to our local toolshed. This does not work as some dependencies required for package_r_3_0_3 are not exported (export capsule) and not included in the archive. For instance, package_freetype_2_5_2 and package_libpng_1_6_7 are present in the capsule and thus moved to the local toolshed, but not package_readline_6_2, package_cairo_1_12_14 and package_pixman_0_32_4. Is it a bug (I would expect a sort of recursivity in the export process) or did I miss some logics behind this ? And do we have to manually export-import the missing dependencies to the local toolshed ? Cheers Chris Christophe Antoniewski Drosophila Genetics and Epigenetics Laboratoire de Biologie du Développement 9, Quai St Bernard, Boîte courrier 24 75252 Paris Cedex 05 christophe.antoniew...@upmc.fr http://bio-dev.snv.jussieu.fr/ Tel +33 1 44 27 34 39 Fax +33 1 44 27 34 45 Mobile +33 6 68 60 51 50 http://drosophile.org http://www.sciencesenmarche.org/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error Setting Up Local Galaxy Server
Hi, Kevin. Yes. that looks like an interesting error message which might have been caused because Galaxy currently requires a version of Python such as 2.7.8 (2.6+ will work fine) but it will not run under the newest Python v3 releases (eg Python 3.4.1) (You may also have encountered a bug in that error detection code making the error message a little confusing) Which version of Python did you install into the VM? You can check with python --version A preconfigured VM used for some recent Galaxy training day workshops might be worth trying - https://wiki.galaxyproject.org/Events/GCC2014/TrainingDay/VMs ? I hope this helps - and yes, this email list is a good place to ask questions about installation. On Sun, Jul 27, 2014 at 12:04 PM, Kevin chang...@byramhills.net wrote: To Whom It May Concern, First, my apologizes if this email is sent to the incorrect location for Galaxy suppose. I am only trying to find a solution to a problem I have setting up a local Galaxy instance to use for my project. Because my computer runs Windows 8 and cannot set up a local Galaxy instance, I am using Oracle VM Virtualbox to run an instance with Linux. I have already downloaded the files for Galaxy and the proper versions of Python. However, when ever I try to enter $ sh run.sh in terminal, the following error message appears: Traceback (most recent call last): File ./scripts/check_python.py, line 16, in check_python assert sys.version_info[:2] = ( 2, 6 ) and sys.version_info[:2] = ( 2, 7 ) AssertionError During handling of the above exception, another exception occurred: Traceback (most recent call last): File ./scripts/check_python.py, line 24, in module check_python() File ./scripts/check_python.py, line 18, in check_python print sys.stderr, msg TypeError: unsupported operand type(s) for : 'builtin_function_or_method' and '_io.TextIOWrapper' During handling of the above exception, another exception occurred: Traceback (most recent call last): File ./scripts/check_python.py, line 25, in module except StandardError: NameError: name 'StandardError' is not defined I do not know what to do with this error message, and require help getting around it so that I can start an instance on my computer. Any advice on how to deal with this problem would be much appreciated. Thank you for taking the time to read this email, Kevin Chang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to configure galaxy with a cluster
and still no cluster options in megablast item. How can I see cluster options in the page, for example, the page will let me choose to use local server or a cluster. Users can't control destinations for tool execution through any interface at present AFAIK - tool destinations are automated although you can have complex plugins which might help. One obvious if slightly painful way to make them user selectable is to (eg) in tool_conf.xml, include a suitably cloned version megablast for each of the two destinations - each with a new id (eg) megablast_local is always sent the local queue and the megablast_cluster tool always goes to the cluster in your job conf. Ugly but possible - I hope this helps. On Wed, Jul 16, 2014 at 5:43 PM, 王渭巍 wangw...@inspur.com wrote: Thanks a lot, Thomas! It really helps, I added tools section followed your suggestion... here is my job_conf.xml ( I am using Torque, I have 3 servers. One for galaxy server, two for cluster computing. ) ?xml version=1.0? job_conf plugins plugin id=pbs type=runner load=galaxy.jobs.runners.pbs:PBSJobRunner/ /plugins destinations default=pbs_default destination id=pbs_default runner=pbs/ /destination destination id=long_jobs runner=pbs param id=Resource_Listwalltime=72:00:00,nodes=1:ppn=8/param param id=-p128/param /destination /destinations tools tool id=megablast_wrapper destination=long_jobs/ /tools /job_conf and still no cluster options in megablast item. How can I see cluster options in the page, for example, the page will let me choose to use local server or a cluster. Ben *From:* Thomas Bellembois thomas.bellemb...@ens-lyon.fr *Date:* 2014-07-15 17:41 *To:* galaxy-dev@lists.bx.psu.edu *Subject:* Re: [galaxy-dev] How to configure galaxy with a cluster Hello Ben, you can configure your Galaxy instance to use your cluster in the job_conf.xml file: https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster You can set up your instance to use your cluster by default for all jobs or only for specific jobs. Here is a part of my job_conf.xml for example: plugins !-- LOCAL JOBS -- plugin id=local type=runner load=galaxy.jobs.runners.local:LocalJobRunner workers=4/ !-- SUN GRID ENGINE -- plugin id=sge type=runner load=galaxy.jobs.runners.drmaa:DRMAAJobRunner/ /plugins handlers default=handlers handler id=handler0 tags=handlers/ handler id=handler1 tags=handlers/ /handlers destinations default=sge_default destination id=local runner=local/ destination id=sge_default runner=sge param id=nativeSpecification-r yes -b n -cwd -S /bin/bash -V -pe galaxy 1/param /destination destination id=sge_big runner=sge param id=nativeSpecification-r yes -b n -cwd -S /bin/bash -V -pe galaxy 12/param /destination /destinations tools tool id=upload1 destination=local/ tool id=toolshed.g2.bx.psu.edu/repos/bhaas/trinityrnaseq/trinityrnaseq/0.0.1 destination=sge_big/ tool id=mira_assembler destination=sge_big/ tool id=megablast_wrapper destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastp_wrapper/0.1.00 destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_tblastn_wrapper/0.1.00 destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastx_wrapper/0.1.00 destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_blastn_wrapper/0.1.00 destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_tblastx_wrapper/0.1.00 destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_rpstblastn_wrapper/0.1.00 destination=sge_big/ tool id= toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_rpsblast_wrapper/0.1.00 destination=sge_big/ /tools Moreover you Galaxy user and Galaxy server must be allowed to submit jobs to your scheduler. Hope it helps, Thomas -- Thomas Bellembois, Network and System Administrator ENS-Lyon/IGFL - FRANCE +33 4 26 73 13 67 .The IGFL is a member of the portail-labos project (https://portail-labos.ens-lyon.fr/site/). .IGFL IT documentation: http://itdoc.igfl.ens-lyon.fr/itdoc On 15/07/2014 11:22, 王渭巍 wrote: Hello, everyone, I am new here, and I am trying to configure galaxy with a cluster. I have 3 servers. One for galaxy server, two for cluster computing. And I followed Running Galaxy Tools on a Cluster done the configuration. Now I have 2 questions: 1. How to vertify whether I have configured the galaxy cluster successfully? For example, I don't see there is any cluster options in NGS mapping (bowtie2) , but I see cluster options in a web galaxy.
Re: [galaxy-dev] Galaxy Tool Wrapper/DTD/Schema Needed
Your question was answered in the negative on biostar a few days back: https://biostar.usegalaxy.org/p/8101/ Contributions welcomed. On Sat, Jun 21, 2014 at 8:25 AM, Jones, Tyler William tyler.jo...@coyotes.usd.edu wrote: Hi, I am working for USD Bioinformatics and trying to integrate Galaxy tools in an application. I have been looking at this site: https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax but was wondering if there was anyway to get the actual DTD or Schema for your tool definitions. Is this possible? Thanks, Tyler Jones ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Difficulties wrapping r script with r_wrapper.sh
This syntax
*write.table(data1, file={$out_data}, *
might not do what you expect. The $ has to be outside the braces, but
they're not needed there anyway - something like this might work better:
*write.table(data1, file=$out_data,..*
When things don't work, it's always useful to check the contents written
into that configfile script by setting job cleanup to never and taking a
look at the temporary folder noted in the galaxy log.
On Tue, Jun 24, 2014 at 2:35 AM, Jeremy Liu jeremy@yale.edu wrote:
Hi all,
I have been trying to wrap a test r script into my local galaxy install
using r_wrapper.sh but have been running with problems. When I run the
tool, I get the following error in the side bar. For reference, I've pasted
the xml file below.
Error:
Error in file(file, rt) : cannot open the connection Calls: read.table
- file Error during wrapup: object 'l' not found Lost warning messages
Execution halted
The xml file:
tool id=r_script_test name=Test R Script
descriptionfor proof of concept/description
command interpreter=bashr_wrapper.sh $script_file/command
inputs
param name=in_data type=data format=tabular label=Test source
file/
/inputs
outputs
data name=out_data format=tabular /
/outputs
configfiles
configfile name=script_file
## Setup R error handling to go to stderr
options(show.error.messages=F,
error = function() { cat(geterrmessage(), file=stderr());
q( no, l, F)})
## Read in tab file
data1 = read.table({$in_data}, sep=\t)
## Output tab file
write.table(data1, file={$out_data}, quote=FALSE, sep=\t, eol =
\n, row.names=FALSE, col.names=FALSE)
## Close the session
dev.off()
/configfile
/configfiles
help
This tools tests a R script to do simple file I/O
/help
/tool
I have r_wrapper.sh copied in the same directory as the xml file. Any help
would be appreciated!
Thanks!
Jeremy Liu
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Non-admin tool install/import
On Tue, Jun 10, 2014 at 7:37 AM, Karthik Gururaj gururaj.kart...@gmail.com wrote: Thanks - will take a look at the Tool Shed and see if there is anything there we can use in safe manner. Else, the old fashioned way of asking users to test out on their tools on a standalone Galaxy system and then requesting the administrator to pull in the relevant XML files. Sorry this is a bit confusing... Just to be quite clear: The Tool Factory is available from the main Tool Shed, but Tool Factory =/= Tool Shed !! The Tool Factory is just another Galaxy tool administrators can use to run scripts interactively in Galaxy. It installs automatically from the main Tool Shed and optionally generates new Galaxy tools from working scripts. New generated tools are in a tgz archive ready to be uploaded into Tool Shed repositories. The Tool Shed is a specialised web server that supports version control and management of Galaxy tool source code and automated installation into Galaxy instances. You could run a local Tool Shed and yes, your admins could use it to install properly configured user provided tools. The Tool Factory allows your scripting-capable users of creating new Galaxy tools from scripts if they run it as administrators of their own laptop/development instances, or your administrators could use it to run scripts directly on your private instance and optionally generate new safe tools (by uploading the archives to your local tool shed then installing those new tools into your local Galaxy) for ordinary users to use in their workflows. Thanks, Karthik On Sun, Jun 8, 2014 at 6:25 PM, Ross ross.laza...@gmail.com wrote: Hi, Karthik and John. Some details on the tool factory for anyone interested. Executive summary: it may be helpful in this context but only for trusted administrators. TL;DR: Firstly, it will refuse to run for anyone other than a local Galaxy administrator. This is because it exposes unrestricted scripting so should only be installed if you can trust your administrative users not to run cd /; rm -rf *. I'd advise installing ONLY on your own private instance and NEVER on a public Galaxy. Secondly, it has two modes of operation - script running and tool generation. When executed without the option to generate a tool archive, it will run a pasted (perl, python, R, bash) script creating an output in the history. This history output is re-doable in the usual Galaxy way including allowing the script to be edited and rerun, so it's possible to (eg) get a script working interactively - galaxy as an IDE anyone ? :) Once a script runs on some test data, the tool factory will optionally generate a complete tool shed compatible gzip which can be uploaded to any tool shed as a new or updated repository. The generated tool includes the supplied test data as a proper Galaxy functional test. Once a tool is in a toolshed, it is just another Galaxy tool, ready to be installed to any Galaxy like any other tool - but will require restarting of multiple web processes as John mentions. If the script is safe, the tool is safe - there are no specific security risks for tool factory generated tools other than the script itself. Finally, currently it takes only one input and generates one output which is a substantial restriction - but of course the generated tool source code is easy to edit if you need more complex I/O. It has a really neat option to create a simple but useful HTML display with links and thumbnails to arbitrary (eg) pdf or other output files from a script - the tool form includes examples in all 4 scripting languages ready to cut and paste, including one which generates 50 random images and presents them in a grid as an HTML page for the user. On Mon, Jun 9, 2014 at 10:32 AM, John Chilton jmchil...@gmail.com wrote: Galaxy doesn't really support this use case and it will be major effort to get it work this way I suspect. Pieces to look at include: The Galaxy Tool Factory (it has the ability to create reusable tools from scripts): http://www.ncbi.nlm.nih.gov/pubmed/23024011 You may be able to modify it in such a way that each tool is tagged with who created and then use ToolBox filters to limit added tools to a given user: https://wiki.galaxyproject.org/UserDefinedToolboxFilters I think the latest version of Galaxy has improved support for adding tools without requiring restarts (using message queues). I don't know if this will automatically work with the tool factory or not. I suspect fighting Galaxy at every step on this will frustrate you and the users - and you are exposing all of your users data to every user you give this privilege to. Is this a shared cluster or is dedicated to Galaxy? If it is shared - it might be better for advanced users to just get importing and exporting data to user directories really well. In my previous position at MSI we created a set of tools that allowed Galaxy to SCP files as the user
Re: [galaxy-dev] ClustalW jobs aren't working
Hi Malcolm, That error makes me think you might be running an outdated version of galaxy code - the toolshed code has undergone extensive revision over the last few months? I just tested that repository on a freshly updated galaxy-central clone and it installed without drama, so I wonder what: hg tip shows? I just tested using : (vgalaxy)rlazarus@rlazarus-UX31A:~/galaxy$ hg tip changeset: 13756:84a00e4f7d06 tag: tip user:Dannon Baker dannonba...@me.com date:Fri Jun 06 17:12:30 2014 -0400 summary: Clarify language in DeleteIntermediateDataset PJA. If your clone is not up to date, I'd recommend completely removing the failed installation (through the admin menu - check the box for complete removal), shut down galaxy, backup your database, do the usual hg pull -u dance and any necessary database upgrade steps then try a clean install? Thanks for reporting this - if it persists on recent Galaxy code we'll need to do some deeper investigation. On Fri, Jun 6, 2014 at 11:32 PM, Malcolm Tobias mtob...@wustl.edu wrote: Ross, Thanks for the reply! Unfortunately I am the local Galaxy admin ;-) I had tried installing the clustalw tool from the toolshed, but that failed with an error (more on that later). I disabled the local tool: [galaxy@login002 galaxy-dist]$ diff tool_conf.xml tool_conf.xml.bkup 226a227,229 section name=Multiple Alignments id=clustal tool file=rgenetics/rgClustalw.xml / /section bounced galaxy in case that's necessary, then retried installing from the toolshed. Shortly after clicking install, I get this message: Internal Server Error Galaxy was unable to sucessfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. The logs appear to be complaining about 'prior_installation_required' which I'm assuming means the package_clustalw_2_1 dependency. I was able to install that, and I can verify by looking at the local toolshed: [galaxy@login002 ~]$ ls galaxy-toolshed/ toolshed.g2.bx.psu.edu/repos/devteam/ bowtie_wrappers package_clustalw_2_1 package_vcftools_0_1_11 bwa_wrappers package_fastx_toolkit_0_0_13 Again, I'll post the logs from when the install fails in case that helps. Any suggestions are much appreciated. Cheers, Malcolm 10.28.56.101 - - [06/Jun/2014:08:20:38 -0500] GET /admin_toolshed/prepare_for_install?tool_shed_url= http://toolshed.g2.bx.psu.edu/repository_ids=0e5d027cf47ecae0changeset_revisions=7cc64024fe92 HTTP/1.1 500 - http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?repository_id=0e5d027cf47ecae0changeset_revision=7cc64024fe92; Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0 Error - type 'exceptions.UnboundLocalError': local variable 'prior_installation_required' referenced before assignment URL: http://galaxy.chpc.wustl.edu/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=0e5d027cf47ecae0changeset_revisions=7cc64024fe92 File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 896 in prepare_for_install common_install_util.get_dependencies_for_repository( trans, tool_shed_url, repo_info_dict, includes_tool_dependencies ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 84 in get_dependencies_for_repository installed_rd, missing_rd = get_installed_and_missing_repository_dependencies_for_new_install( trans, repo_info_tuple ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 190 in get_installed_and_missing_repository_dependencies_for_new_install tool_shed, name, owner, changeset_revision, prior_installation_required = suc.parse_repository_dependency_tuple( rd_tup ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/shed_util_common.py', line 1132 in parse_repository_dependency_tuple prior_installation_required = str
Re: [galaxy-dev] Are there examples of tools that run processes in the background?
Hi Melissa, The question in the title is easy to answer - most large galaxy instances run (most) jobs on a connected cluster which is effectively 'background' I guess so yes - all of them on cluster backed Galaxy instances might be a reasonable response. The questions in the text are more complex, but (eg) AFAIK there's no technical reason why some batch process owned by the Galaxy user shouldn't issue a sh run.sh --stop-daemon at any time to stop Galaxy. Exactly when that might be a good idea is not so clear to me so I'm probably missing something here - the Galaxy process doesn't consume a lot of resources and is typically left running permanently so users can take advantage of the persistence and reproducibility. Tools are AFAIK not generally written to launch their own background processes because they are managed by the Galaxy job runner which interfaces with (eg) SGE or Torque to run each separate tool invocation and clean up when it has completed. I'm sure you could run a transient Galaxy to batch up some workflows or other processes and I'm sure a VM might be part of that and the API allows a lot of automation. There's a new Galaxy docker image project which might be even more efficient and helpful - see http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019289.html On Sat, Jun 7, 2014 at 10:18 AM, Melissa Cline cl...@soe.ucsc.edu wrote: Hi folks, We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications). Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time. Or is there a better approach I didn't consider? Assuming the answer is no, then are there any tools in the toolshed that launch background processes? If so, I'd like to look at them. And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down. Is there a way to configure a process to be run when it's time to quit Galaxy? Thanks! Melissa ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ClustalW jobs aren't working
Malcolm, It looks like you have downloaded the wrapper and put it in your local tools folder. We strongly recommend that you remove the tool_conf.xml entry and get the local galaxy administrator to install the devteam clustalw repository from the main toolshed. That should solve your problem - please let us know if not. You can browse that repo here http://toolshed.g2.bx.psu.edu/view/devteam/clustalw and install it from the admin menu; search and browse tool sheds; main tool shed;... On Fri, Jun 6, 2014 at 3:24 AM, Malcolm Tobias mtob...@wustl.edu wrote: All, When running clustalW jobs, the only output we're getting is an error message that appears to show the tool isn't receiving the proper run-time parameters: Unknown option: I Value FILE=infile_copy invalid for option N (number expected) Unknown option: F Unknown option: I Unknown option: L Unknown option: E ... This appears to come from the rgClustalw.py wrapper: cl = ['clustalw2 -INFILE=%s -OUTFILE=%s -OUTORDER=%s -TYPE=%s -OUTPUT=%s ' % (self.iname,self.opts.output,self.opts.out_order,self.opts.dnarna,self.opts. outform)] Strangely, the log files appear to show the correct paramters: galaxy.tools WARNING 2014-06-05 10:37:43,484 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.local DEBUG 2014-06-05 10:37:43,596 (779) executing: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i /home/galaxy/galaxy-dist/database/files/001/dataset_1243.dat -o /home/galaxy/galaxy-dist/database/files/001/dataset_1244.dat -s ALIGNED -l /home/galaxy/galaxy-dist/database/files/001/dataset_1245.dat -t Clustal_run -d DNA -f CLUSTAL I'll post the full output from the logs below, but I was hoping someone might be able to offer some suggestions on how to debug this further. Thanks! Malcolm 10.28.56.101 - - [05/Jun/2014:10:37:42 -0500] POST /tool_runner/index HTTP/1.1 200 - http://galaxy.chpc.wustl.edu/tool_runner?tool_id=clustalw; Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0 galaxy.jobs DEBUG 2014-06-05 10:37:43,025 (779) Working directory for job is: /home/galaxy/galaxy-dist/database/job_working_directory/000/779 galaxy.jobs.handler DEBUG 2014-06-05 10:37:43,057 (779) Dispatching to local runner galaxy.jobs DEBUG 2014-06-05 10:37:43,205 (779) Persisting job destination (destination id: local) galaxy.jobs.handler INFO 2014-06-05 10:37:43,258 (779) Job dispatched 10.28.56.101 - - [05/Jun/2014:10:37:42 -0500] GET /history HTTP/1.1 200 - http://galaxy.chpc.wustl.edu/tool_runner/index; Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0 galaxy.tools DEBUG 2014-06-05 10:37:43,466 Building dependency shell command for dependency 'clustalw2' galaxy.tools WARNING 2014-06-05 10:37:43,484 Failed to resolve dependency on 'clustalw2', ignoring galaxy.jobs.runners.local DEBUG 2014-06-05 10:37:43,596 (779) executing: python /home/galaxy/galaxy-dist/tools/rgenetics/rgClustalw.py -i /home/galaxy/galaxy-dist/database/files/001/dataset_1243.dat -o /home/galaxy/galaxy-dist/database/files/001/dataset_1244.dat -s ALIGNED -l /home/galaxy/galaxy-dist/database/files/001/dataset_1245.dat -t Clustal_run -d DNA -f CLUSTAL 10.28.56.101 - - [05/Jun/2014:10:37:43 -0500] GET /api/histories/1ac3b66e11b8fe50 HTTP/1.1 200 - http://galaxy.chpc.wustl.edu/history; Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0 galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,732 Exception caught attempting to generate viewport for dataset '972' Traceback (most recent call last): File /home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py, line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_769.dat' galaxy.jobs DEBUG 2014-06-05 10:37:43,736 (779) Persisting job destination (destination id: local) galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,776 Exception caught attempting to generate viewport for dataset '985' Traceback (most recent call last): File /home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py, line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_782.dat' galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,830 Exception caught attempting to generate viewport for dataset '986' Traceback (most recent call last): File /home/galaxy/galaxy-dist/lib/galaxy/datatypes/interval.py, line 159, in get_estimated_display_viewport fh = open( dataset.file_name ) IOError: [Errno 2] No such file or directory: '/home/galaxy/galaxy-dist/database/files/000/dataset_783.dat' galaxy.datatypes.interval ERROR 2014-06-05 10:37:43,876 Exception caught attempting to generate viewport for dataset '987' Traceback (most recent
Re: [galaxy-dev] uploading data libraries options
Hi, Mark, Perhaps you need to adjusted universe_wsgi.ini where it says: # Add an option to the admin library upload tool allowing admins to paste # filesystem paths to files and directories in a box, and these paths will be # added to a library. Set to True to enable. Please note the security # implication that this will give Galaxy Admins access to anything your Galaxy # user has access to. allow_library_path_paste = True A restart will be needed. If you already have this set, please provide some more details like the hg head for your installation? On Tue, Apr 22, 2014 at 12:09 PM, mark.r...@syngenta.com wrote: Hello when attempting to create data libraries as admin I am not presented with the same options listed in the documentation, namely Instead I only get “Upload Files” and “Import datasets from your current history”. I want to use “Upload files from filesystem paths”. What should I do? Thanks Mark -- *This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited.* ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Functional Tests and ftype
Hi, Saket - 436 tests in 19.153 seconds means something odd (or fast) with your setup? What errors do you see? Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J On Thu, Apr 17, 2014 at 10:43 PM, Saket Choudhary sake...@gmail.com wrote: Just to add, that is the case with all the tests: $ sh run_functional_tests.sh Ran 436 tests in 19.153s FAILED (errors=434) All tests fail locally and hence I decided to rely on travis. https://wiki.galaxyproject.org/Admin/RunningTests#Functional_Tests says it might be due to conflicting version of paste in my PYTHONPATH. But it happens even if I run these under a virtualenv. Saket On 17 April 2014 18:03, Saket Choudhary sake...@gmail.com wrote: On 17 April 2014 15:34, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, Apr 17, 2014 at 12:59 AM, Saket Choudhary sake...@gmail.com wrote: I happened to copy Peter's .travis.yml for running tests for some of my packages. Background information: http://blastedbio.blogspot.co.uk/2013/09/using-travis-ci-for-testing-galaxy-tools.html However the process has not been smooth at all. See : https://travis-ci.org/saketkc/galaxy_tools/jobs/23166678#L908 AssertionError: Attempting to set field 'input' to value '['condel_input.tabular']' in form 'tool_form' threw exception: cannot find value/label condel_input.tabular in list control I am already passing it the correct ftype : https://github.com/saketkc/galaxy_tools/blob/travis-test/condel_web/condel_web.xml#L22 Is there something I am missing? This is similar to (but different) to the error message I recently reported as being unclear when a datatype was not defined: http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019090.html In your case, this is an input file which seems to be missing, test-data/condel_input.tabular - but I see the line copying the tool's test-data into the main Galaxy test-data folder is there in the .travis.yml file. I am puzzled too. The key question I would ask you is: do your tests work locally? If so, are they installed via a (local) Tool Shed, or manually along the same lines of the attempted TravisCI tool installation? Unfortunately No. TwillAssertionError: code is 500 != 200 I am running Galaxy under virtualenv. Unfortunately I get the same error irrespective of whther my input/output files exits inside test-data/ directory. I forgot to link them to test-data path earlier, but I get the same error even otherwise. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] . Output file to history (Sebastian Luna Valero)
Hi, Jun and Sebastian, This is not trivial, but it is already being done with some tools using the method described at https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Single_history_output_Html_file_with_links_to_any_number_of_output_files_and_images if that can solve your problem. The Galaxy Html composite datatype can be used to expose arbitrary outputs without creating a separate history item for each one if that's what you want. The tool executable or the wrapper must write valid and complete Html content to the path Galaxy provides as the Html file for the tool and the sanitize_all_html switch must be turned off in universe_wsgi.ini (which is a potential security problem for public sites!) for the page to be pretty, but since the script itself is generating all the output it is possible to write code to create a nicely laid out page of links and images for the user. Composite datatypes (like Html) are documented in the wiki and contributions to the wiki are always welcomed - but at present, it's an advanced and not well documented possibility. If there really is demand for this functionality to be exposed, I have been thinking about an autoHtml datatype that would do what the tool factory does - look inside the job working directory at the end of the job and arrange every file it finds there into a simple Html page of links. If https://trello.com/c/vNQLZnSk gets enough upvotes, we'll put it into the development pipeline? On Sun, Mar 23, 2014 at 4:43 AM, Jun Fan j@qmul.ac.uk wrote: Hi Sebastian, I am not an expert in galaxy developing. So the following content is just my personal opinion. I do not think it is possible to meet you requirement. I reckon Galaxy searches the certain folder, defined in the universal_wsgi.ini by a parameter called #collect_outputs_from = new_file_path,job_working_directory. On top of that, a job id is required, which is the reason in the dynamic multiple output job id is part of the file name. If your script.py cannot follow the rules, Galaxy for sure cannot recognize the outputs. Normally in this kind of case, I write an additional script to execute the script.py with its original parameter and do the file manipulation afterwards. Of course this is not ideal, but I do nothing about python. It may be possible to write some complicated python code within the command tag of your wrapper file to do this. I am here also waiting for experts' opinions/comments. Best regards! Jun On 3/21/14 9:28 AM, Sebastian Luna Valero wrote: Dear All, I am trying to add a new tool in Galaxy and I have the following problem. Let me explain a simplified example. Let's imagine that my script works from CLI as follows: python script.py --input input-file --pattern output-pattern After processing input-file, the script writes several output files and their name is given according to output-pattern. The number of output files depends on the content of input-file. The output files are written to the current working directory where script.py is located. In the simplest scenario, I get only one output file in the working directory. My problem is that I would like to see the output files in Galaxy's history without modifying the CLI interface. To solve this problem, I have looked at the wiki page: https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#N umber_of_Output_datasets_cannot_be_determined_until_tool_run and the email here: http://dev.list.galaxyproject.org/Multiple-output-files-do-not-appear- in-history-td4660470.html However, I think that mine is a different scenario. I do not want to add new output parameters in the CLI. I would like Galaxy to bring these output files to my history without modifying the CLI options, is that possible? Many thanks in advance for your help! Best regards, Sebastian. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Nightly testing status on the (Test) Tool Shed
if a tool failed we all really benefit from seeing as much as we can - as securely as we can. Can we safely expose them somewhere protected? On Thu, Feb 20, 2014 at 8:59 PM, Peter Cock p.j.a.c...@googlemail.comwrote: , On Thu, Feb 20, 2014 at 2:56 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, On Wed, Feb 19, 2014 at 9:33 PM, Greg Von Kuster g...@bx.psu.edu wrote: In any case, we have a fix for out move_directory_files which Dave is now committing. So your tests should pass with tonight's run - Let's hope! That explanation makes perfect sense to me - but I can see it took some digging to solve it! There is actually one MIRA binary (mira) and several aliases (e.g. mirabait) which are really symlinks back to mira. Magic. For anyone interested, the fix was this commit: https://bitbucket.org/galaxy/galaxy-central/commits/a89bbdfd4a71671d575d9a0a3a9f8abeba3f7b21 My next query would be: why wasn't the failing action move_directory_files being caught as a failed install? Scanning over the code, some of the actions have explicit error handling but make_directory, move_directory_files, move_file and others do not. I guess any exception is propagated up... lib/tool_shed/galaxy_install/tool_dependencies/fabric_util.py Your installation recipe for mira attempts to download a binary and if that fails, it echoes an error, but still performs set_environment actions.. Ah - I can see that now, I need a fall back action tag which either tries to compile MIRA or raises an explicit error. Thanks! Here is the entire log of the installtion process that helped us uncover the problem - notice that upon failure of the initial binary installation, the process proceeds with install and compile recipe for tool dependency MIRA. Would it be a security risk to automatically expose these logs? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] space in text tool results in two arguments
Hi Ketan - great news - all the best with your project...
On Tue, Feb 18, 2014 at 3:58 PM, Ketan Maheshwari ke...@mcs.anl.gov wrote:
Ross,
That's correct. It seems I tested on a cached copy of tool. Works now!
Thanks,
Ketan
On Mon, Feb 17, 2014 at 10:55 PM, Ross ross.laza...@gmail.com wrote:
Hi Ketan.
We routinely recommend quotes as you've shown for this reason and it
certainly works for me at least !
Did you reload the tool xml after editing (via the admin menu or via a
galaxy restart) before testing? If not, you tested a cached copy of the
tool!
Ross Lazarus MBBS MPH;
Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444
http://scholar.google.com/citations?hl=enuser=UCUuEM4J
On Tue, Feb 18, 2014 at 3:40 PM, Ketan Maheshwari
ketancmaheshw...@gmail.com wrote:
Hi Ross,
I did try to use double quotes and curly braces: ${outloc} but it
does not seem to address the issue.
Thanks,
Ketan
On Mon, Feb 17, 2014 at 10:37 PM, Ross ross.laza...@gmail.com wrote:
Hi Ketan.
Please try quotation marks to enclose any parameter containing spaces
in the tool command template - eg something like:
python myscript.py $text_with_spaces $param2 $param3
Please confirm that this solves the problem?
On Tue, Feb 18, 2014 at 12:47 PM, Ketan Maheshwari
ketancmaheshw...@gmail.com wrote:
Hi,
My tool in galaxy accepts text argument which can have zero or more
spaces depending on user requirements. When user inputs one word it is
parsed fine but in the case when user inputs more words separated by space
it results in multiple separate arguments which messes with the way I am
treating commandline args in my script. Is it possible to tell Galaxy to
treat textbox as a single argument irrespective of spaces in the values
provided?
Thanks for any suggestions.
Best,
--
Ketan
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
--
Ketan
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] SRA datatype
Hi, Todd, The search bar at https://wiki.galaxyproject.org/ is always a good place to find stuff like https://wiki.galaxyproject.org/Admin/Datatypes/Adding%20Datatypes?highlight=%28converters%29 I hope this gets you started On Sat, Feb 8, 2014 at 12:22 PM, Todd Oakley todd.oak...@lifesci.ucsb.eduwrote: Hello, I am seeking advice on how to add the SRA datatype to Galaxy. We want to download data from NCBI that is in SRA format, and can no longer have the NCBI site convert it to fastq. I am not experienced with adding file types, to rudimentary advice would be most appreciated. Thanks, Todd Oakley, UCSB ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error running Clustalw and HyPhy on Cloudman
Hi Karen, Glad to hear it worked for you. It the tool auto-installs correctly, it downloads and unpacks the binary under [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ so the binary you installed manually should not ever be needed by the toolshed version - try moving it off your path - the tool should continue to work correctly? Apologies - I should have suggested removing any manual installations and their corresponding tool_conf.xml entries before using the automated installation - the tool shed installation doesn't prevent you from having 2 versions of the same tool in your tool menu - until those manually installed ones (which may depend on clustalw2 being on the path) are removed? On Mon, Dec 2, 2013 at 8:36 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, Thanks for your email. I have clustalw2 working now - I installed the clustal repository (as suggested) from toolshed and then installed the clustalw2 binaries in our mnt/galaxy/tools/bin. Thanks again! Karen On Wed, Nov 27, 2013 at 12:46 AM, Ross ross.laza...@gmail.com wrote: Hi, Karen - how did you install clustalw? If not already done, can you please try installing the clustal repository owned by fubar from the test toolshed through your Galaxy admin interface - search/browse toolsheds? Please let me know how you go and if there's a problem I'll try to fix it for you. I think there's a clustal omega tool too but it's not one of mine. As far as I know my clustal repository should install cleanly on a CloudMan instance. Sorry but we'll have to wait to from someone who knows about HyPhy. Hello - I am running to errors when trying to run Clustalw and HyPhy on our cloudman instance. (1) Clustalw: The program completes without error and produces an empty file and a log file that states /bin/sh: 1: clustalw2: not found Searching around a bit I am not seeing clustalw2 previously installed. I am happy to do it if need be, however, I wanted to check in and see if I am overlooking the directory or an easy fix. If I do need to install clustalw, I would prefer to use clustalw-omega, but was not sure if I needed to script a brand new wrapper or if I could use the previous one that came with galaxy image. (2) HyPhy: The cloud the link was initially broken. I changed it to the correct path: sudo ln -sfn /mnt/galaxy/tools/hyphy/default /mnt/galaxy/galaxy-app/tool-data/HYPHY I am using a fasta alignment file of 6 protein sequences (566 AA each) as a test case (*also fails with nucleotide alignments). It runs briefly and then issues an error statement Single Alignment Analyses Segmentation fault (core dumped). Thank you in advance, Karen ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error running Clustalw and HyPhy on Cloudman
yes. In universe_wsgi_ini you have a definition for tool_dependency_dir - that's why I wrote: [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ as the path - the funny numbers are the repository revision number - that's where the toolshed installs the binary and the env.sh file needed for the tool to find it. On Mon, Dec 2, 2013 at 9:08 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, I did not detect any type of auto-install (?) After installing from toolshed I did not find the binary under the /mnt/galaxy/shed_tools/ toolshed.g2.bx.psu.edu/repos/ (or /mnt/galaxy/tools or /mnt/galaxy/tools/bin for that matter). Am I looking in the wrong place? Karen On Sun, Dec 1, 2013 at 2:00 PM, Ross ross.laza...@gmail.com wrote: Hi Karen, Glad to hear it worked for you. It the tool auto-installs correctly, it downloads and unpacks the binary under [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ so the binary you installed manually should not ever be needed by the toolshed version - try moving it off your path - the tool should continue to work correctly? Apologies - I should have suggested removing any manual installations and their corresponding tool_conf.xml entries before using the automated installation - the tool shed installation doesn't prevent you from having 2 versions of the same tool in your tool menu - until those manually installed ones (which may depend on clustalw2 being on the path) are removed? On Mon, Dec 2, 2013 at 8:36 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, Thanks for your email. I have clustalw2 working now - I installed the clustal repository (as suggested) from toolshed and then installed the clustalw2 binaries in our mnt/galaxy/tools/bin. Thanks again! Karen On Wed, Nov 27, 2013 at 12:46 AM, Ross ross.laza...@gmail.com wrote: Hi, Karen - how did you install clustalw? If not already done, can you please try installing the clustal repository owned by fubar from the test toolshed through your Galaxy admin interface - search/browse toolsheds? Please let me know how you go and if there's a problem I'll try to fix it for you. I think there's a clustal omega tool too but it's not one of mine. As far as I know my clustal repository should install cleanly on a CloudMan instance. Sorry but we'll have to wait to from someone who knows about HyPhy. Hello - I am running to errors when trying to run Clustalw and HyPhy on our cloudman instance. (1) Clustalw: The program completes without error and produces an empty file and a log file that states /bin/sh: 1: clustalw2: not found Searching around a bit I am not seeing clustalw2 previously installed. I am happy to do it if need be, however, I wanted to check in and see if I am overlooking the directory or an easy fix. If I do need to install clustalw, I would prefer to use clustalw-omega, but was not sure if I needed to script a brand new wrapper or if I could use the previous one that came with galaxy image. (2) HyPhy: The cloud the link was initially broken. I changed it to the correct path: sudo ln -sfn /mnt/galaxy/tools/hyphy/default /mnt/galaxy/galaxy-app/tool-data/HYPHY I am using a fasta alignment file of 6 protein sequences (566 AA each) as a test case (*also fails with nucleotide alignments). It runs briefly and then issues an error statement Single Alignment Analyses Segmentation fault (core dumped). Thank you in advance, Karen -- Karen H. Miga, PhD Postdoctoral Scholar University of California, Santa Cruz (831)459-5232 -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Error running Clustalw and HyPhy on Cloudman
Just to confirm, you used automated installation via the admin menu - search and browse toolsheds - test toolshed - etc ? If you installed manually by downloading and unpacking the archive under your tools/ directory, then yes, there is no automation and you need to install the binary on your path. I believe the documentation explains this - if it's not clear please suggest some revisions ? On Mon, Dec 2, 2013 at 9:18 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, That is listed as tool_dependency_dir = /mnt/galaxy/tools in my universe_wsgi.ini file, and that did not contain a clustalw directory either before or after the toolshed install. Karen On Sun, Dec 1, 2013 at 2:10 PM, Ross ross.laza...@gmail.com wrote: yes. In universe_wsgi_ini you have a definition for tool_dependency_dir - that's why I wrote: [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ as the path - the funny numbers are the repository revision number - that's where the toolshed installs the binary and the env.sh file needed for the tool to find it. On Mon, Dec 2, 2013 at 9:08 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, I did not detect any type of auto-install (?) After installing from toolshed I did not find the binary under the /mnt/galaxy/shed_tools/ toolshed.g2.bx.psu.edu/repos/ (or /mnt/galaxy/tools or /mnt/galaxy/tools/bin for that matter). Am I looking in the wrong place? Karen On Sun, Dec 1, 2013 at 2:00 PM, Ross ross.laza...@gmail.com wrote: Hi Karen, Glad to hear it worked for you. It the tool auto-installs correctly, it downloads and unpacks the binary under [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ so the binary you installed manually should not ever be needed by the toolshed version - try moving it off your path - the tool should continue to work correctly? Apologies - I should have suggested removing any manual installations and their corresponding tool_conf.xml entries before using the automated installation - the tool shed installation doesn't prevent you from having 2 versions of the same tool in your tool menu - until those manually installed ones (which may depend on clustalw2 being on the path) are removed? On Mon, Dec 2, 2013 at 8:36 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, Thanks for your email. I have clustalw2 working now - I installed the clustal repository (as suggested) from toolshed and then installed the clustalw2 binaries in our mnt/galaxy/tools/bin. Thanks again! Karen On Wed, Nov 27, 2013 at 12:46 AM, Ross ross.laza...@gmail.com wrote: Hi, Karen - how did you install clustalw? If not already done, can you please try installing the clustal repository owned by fubar from the test toolshed through your Galaxy admin interface - search/browse toolsheds? Please let me know how you go and if there's a problem I'll try to fix it for you. I think there's a clustal omega tool too but it's not one of mine. As far as I know my clustal repository should install cleanly on a CloudMan instance. Sorry but we'll have to wait to from someone who knows about HyPhy. Hello - I am running to errors when trying to run Clustalw and HyPhy on our cloudman instance. (1) Clustalw: The program completes without error and produces an empty file and a log file that states /bin/sh: 1: clustalw2: not found Searching around a bit I am not seeing clustalw2 previously installed. I am happy to do it if need be, however, I wanted to check in and see if I am overlooking the directory or an easy fix. If I do need to install clustalw, I would prefer to use clustalw-omega, but was not sure if I needed to script a brand new wrapper or if I could use the previous one that came with galaxy image. (2) HyPhy: The cloud the link was initially broken. I changed it to the correct path: sudo ln -sfn /mnt/galaxy/tools/hyphy/default /mnt/galaxy/galaxy-app/tool-data/HYPHY I am using a fasta alignment file of 6 protein sequences (566 AA each) as a test case (*also fails with nucleotide alignments). It runs briefly and then issues an error statement Single Alignment Analyses Segmentation fault (core dumped). Thank you in advance, Karen -- Karen H. Miga, PhD Postdoctoral Scholar University of California, Santa Cruz (831)459-5232 -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J -- Karen H. Miga, PhD Postdoctoral Scholar University of California, Santa Cruz (831)459-5232 -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http
Re: [galaxy-dev] Error running Clustalw and HyPhy on Cloudman
ah. you are right - I had not updated that older repository - I think I was waiting for confirmation that the testtoolshed version worked - will do shortly - thanks for pointing that out! Try removing that one completely and then installing the test toolshed version please? That should work until I get the main one updated in a few hours. On Mon, Dec 2, 2013 at 9:28 AM, Karen Miga khm...@soe.ucsc.edu wrote: Yes, I do all of my installs from the toolshed through the admin menu, however I did not use the test toolshed... I used the main. Ah ha - maybe that was the issue? Karen On Sun, Dec 1, 2013 at 2:23 PM, Ross ross.laza...@gmail.com wrote: Just to confirm, you used automated installation via the admin menu - search and browse toolsheds - test toolshed - etc ? If you installed manually by downloading and unpacking the archive under your tools/ directory, then yes, there is no automation and you need to install the binary on your path. I believe the documentation explains this - if it's not clear please suggest some revisions ? On Mon, Dec 2, 2013 at 9:18 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, That is listed as tool_dependency_dir = /mnt/galaxy/tools in my universe_wsgi.ini file, and that did not contain a clustalw directory either before or after the toolshed install. Karen On Sun, Dec 1, 2013 at 2:10 PM, Ross ross.laza...@gmail.com wrote: yes. In universe_wsgi_ini you have a definition for tool_dependency_dir - that's why I wrote: [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ as the path - the funny numbers are the repository revision number - that's where the toolshed installs the binary and the env.sh file needed for the tool to find it. On Mon, Dec 2, 2013 at 9:08 AM, Karen Miga khm...@soe.ucsc.edu wrote: Hi Ross, I did not detect any type of auto-install (?) After installing from toolshed I did not find the binary under the /mnt/galaxy/shed_tools/ toolshed.g2.bx.psu.edu/repos/ (or /mnt/galaxy/tools or /mnt/galaxy/tools/bin for that matter). Am I looking in the wrong place? Karen On Sun, Dec 1, 2013 at 2:00 PM, Ross ross.laza...@gmail.com wrote: Hi Karen, Glad to hear it worked for you. It the tool auto-installs correctly, it downloads and unpacks the binary under [tool_dependency_directory]/clustalw/2.1/fubar/clustalw/76de6971a0de/ so the binary you installed manually should not ever be needed by the toolshed version - try moving it off your path - the tool should continue to work correctly? Apologies - I should have suggested removing any manual installations and their corresponding tool_conf.xml entries before using the automated installation - the tool shed installation doesn't prevent you from having 2 versions of the same tool in your tool menu - until those manually installed ones (which may depend on clustalw2 being on the path) are removed? On Mon, Dec 2, 2013 at 8:36 AM, Karen Miga khm...@soe.ucsc.eduwrote: Hi Ross, Thanks for your email. I have clustalw2 working now - I installed the clustal repository (as suggested) from toolshed and then installed the clustalw2 binaries in our mnt/galaxy/tools/bin. Thanks again! Karen On Wed, Nov 27, 2013 at 12:46 AM, Ross ross.laza...@gmail.comwrote: Hi, Karen - how did you install clustalw? If not already done, can you please try installing the clustal repository owned by fubar from the test toolshed through your Galaxy admin interface - search/browse toolsheds? Please let me know how you go and if there's a problem I'll try to fix it for you. I think there's a clustal omega tool too but it's not one of mine. As far as I know my clustal repository should install cleanly on a CloudMan instance. Sorry but we'll have to wait to from someone who knows about HyPhy. Hello - I am running to errors when trying to run Clustalw and HyPhy on our cloudman instance. (1) Clustalw: The program completes without error and produces an empty file and a log file that states /bin/sh: 1: clustalw2: not found Searching around a bit I am not seeing clustalw2 previously installed. I am happy to do it if need be, however, I wanted to check in and see if I am overlooking the directory or an easy fix. If I do need to install clustalw, I would prefer to use clustalw-omega, but was not sure if I needed to script a brand new wrapper or if I could use the previous one that came with galaxy image. (2) HyPhy: The cloud the link was initially broken. I changed it to the correct path: sudo ln -sfn /mnt/galaxy/tools/hyphy/default /mnt/galaxy/galaxy-app/tool-data/HYPHY I am using a fasta alignment file of 6 protein sequences (566 AA each) as a test case (*also fails with nucleotide alignments). It runs briefly and then issues an error statement Single Alignment Analyses Segmentation fault (core dumped). Thank you in advance, Karen -- Karen H. Miga, PhD Postdoctoral Scholar University
Re: [galaxy-dev] Error running Clustalw and HyPhy on Cloudman
Hi, Karen - how did you install clustalw? If not already done, can you please try installing the clustal repository owned by fubar from the test toolshed through your Galaxy admin interface - search/browse toolsheds? Please let me know how you go and if there's a problem I'll try to fix it for you. I think there's a clustal omega tool too but it's not one of mine. As far as I know my clustal repository should install cleanly on a CloudMan instance. Sorry but we'll have to wait to from someone who knows about HyPhy. Hello - I am running to errors when trying to run Clustalw and HyPhy on our cloudman instance. (1) Clustalw: The program completes without error and produces an empty file and a log file that states /bin/sh: 1: clustalw2: not found Searching around a bit I am not seeing clustalw2 previously installed. I am happy to do it if need be, however, I wanted to check in and see if I am overlooking the directory or an easy fix. If I do need to install clustalw, I would prefer to use clustalw-omega, but was not sure if I needed to script a brand new wrapper or if I could use the previous one that came with galaxy image. (2) HyPhy: The cloud the link was initially broken. I changed it to the correct path: sudo ln -sfn /mnt/galaxy/tools/hyphy/default /mnt/galaxy/galaxy-app/tool-data/HYPHY I am using a fasta alignment file of 6 protein sequences (566 AA each) as a test case (*also fails with nucleotide alignments). It runs briefly and then issues an error statement Single Alignment Analyses Segmentation fault (core dumped). Thank you in advance, Karen ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] pass regular expression
Hi, Jun. The inbuilt parameter sanitizer can be controlled - see http://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax#A.3Csanitizer.3E_tag_set The valid sub-tag allows precise control and you can specify eg string.printable if you want ANY printable character in your parameter Here's an example from the Tool Factory: param name=tool_name type=text value=My dynamic script label=New tool ID and title for outputs size=60 help=This will become the toolshed repository name so please choose thoughtfully to avoid namespace clashes with$ sanitizer invalid_char= valid initial=string.letters,string.digits/ /sanitizer /param I hope this helps? On Sat, Oct 26, 2013 at 11:43 AM, Jun Fan j@qmul.ac.uk wrote: Hi all, ** ** I am trying to develop a tool which allows the user to use regular expression to only keep the lines matching the given pattern in the output. There are many special characters, e.g. \ in \d. In the perl script the wrapper invokes, the regular expression is printed as Xd+ for the regular expression \d+. My question is how to pass these special characters including \, (, ), ^ etc. correctly into the perl script. ** ** Best regards! Jun ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Missing test results on (Test) Tool Shed
=/Search/keyword/Snord104' Snord104/a -Mir1983 -5.89550024150745 0.993185099223749 -11.4458119994178 1.52754786535047e-08 6.44160462853232e-07 9.74926029381244 101 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir1983' Mir1983/a -Mir322 -3.29661750880005 8.21534154356388 -11.4153616003567 1.58076187203247e-08 6.44160462853232e-07 10.0084716002011 7074 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir322'Mir322/a -Mir200a 6.19156065085543 1.79813092499896 11.3221723123067 1.75622912046568e-08 6.9098531946598e-07 9.66229453831667 264 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir200a' Mir200a/a -Mir215 -3.04587333807051 5.75442336214621 -11.1481336257529 2.14182153707674e-08 8.08245865886245e-07 9.75326755116029 1182 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir215'Mir215/a -Dnm3os -3.36334357719079 5.86074322417943 -11.0922610835813 2.28395969947309e-08 8.08245865886245e-07 9.68949616901383 1401 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Dnm3os'Dnm3os/a -Mir182 4.90399541739044 7.1511683493624 11.0744681203078 2.33130367310143e-08 8.08245865886245e-07 9.65884218207857 7189 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir182'Mir182/a -Mir181a-2 -3.04829832099813 6.94146510070354 -11.0721276255975 2.33760855164295e-08 8.08245865886245e-07 9.64401697815694 2817 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir181a-2' Mir181a-2/a -Mir1948 7.19552540631629 4.5513492833967 11.0054920626234 2.52493600829575e-08 8.47338819254543e-07 9.34179361673467 2404 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir1948' Mir1948/a -Mir214 -3.28087400431203 5.47844506177362 -10.7682572190636 3.3325545851092e-08 1.0864127947456e-06 9.3185039394887 1048 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir214'Mir214/a -Mir153 -5.9638030672045 1.43863148956561 -10.7270821099311 3.49874201497232e-08 1.09398957489501e-06 9.03556928822473 140 a href=' http://www.genecards.org/index.php?path=/Search/keyword/Mir153'Mir153/a Categories Statistics On Sep 26, 2013, at 6:47 AM, Ross ross.laza...@gmail.com wrote: +1 I think it's been a problem with some of my repos for a while on test so possible not related to blast_datatypes - I sent this privately about 2 weeks ago: Ross ross.laza...@gmail.com Sep 13 (13 days ago) to Dave, Greg Hey Dave - any thoughts on how to fix http://testtoolshed.g2.bx.psu.edu/view/fubar/differential_count_models so the test results appear? AFAIK the test works but there's no test results section in the manage repository sections I can see. On Thu, Sep 26, 2013 at 8:40 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Tue, Sep 24, 2013 at 1:44 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, The one on the main tool shed is due to an issue Greg and I are in the process of resolving. As soon as we've tested the fix, I'll schedule a re-test of that repository and update you on the status. --Dave B. Looks to be spreading, or linked to the blast_datatypes dependency? Listed under Latest revision: failing tool tests http://toolshed.g2.bx.psu.edu/view/peterjc/blast2go/0ac3ef59ea93 - missing test results http://toolshed.g2.bx.psu.edu/view/peterjc/blastxml_to_top_descr/09a68a90d552 - missing test results http://toolshed.g2.bx.psu.edu/view/peterjc/seq_primer_clip/8c02a91a8680 - failed as expected (missing Biopython dependency, see separate thread about the ALTAS dependency problem) http://toolshed.g2.bx.psu.edu/view/peterjc/venn_list/6aae6bc0802d - failing as expected (missing R dependency) And over on the Test Tool Shed (which was offline but recently came back up), listed under Latest revision: failing tool tests: http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go/8c462f7b2c8d - missing test results http://testtoolshed.g2.bx.psu.edu/view/peterjc/predictnls/aabc2852571e - missing test results Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J
Re: [galaxy-dev] Deseq2 wrapper question
Vipin, Search works pretty well on the toolshed - I'd recommend a search for (eg) 'star' before writing yet another rnastar toolshed wrapper and dependency package? There's an installable fubar package_rnastar and an rnastar wrapper based on Jeremy's code mentioned below - but working AFAIK - at least it installs and works fine here and on the GVL. On Thu, Sep 12, 2013 at 1:49 AM, Jeremy Goecks jeremy.goe...@emory.eduwrote: STAR aligner I have a partial STAR wrapper that may be useful: https://bitbucket.org/jgoecks/jeremys-code/src/d7096c132657420f08b540c6954b947f648aac80/galaxy/tool-wrappers/star.xml?at=default Whether you use this code or not, I do have one request: please put the output in BAM rather than SAM format in order to save disk space. Best, J. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Local Instance and current website are not the same
Hi Tevfik, Adam is right, but even easier assuming it goes smoothly, you should be able to install an updated version 3.3 of JJ's previous 3.2 wrapper via the Galaxy admin interface - test toolshed; SNP Analysis; snpeff_3_3_wrapperhttp://testtoolshed.g2.bx.psu.edu/repository/browse_repositories_in_category?sort=nameoperation=view_or_manage_repositoryid=1cea2cd3f63ea2a9 owned by fubar. Please let me know if it works for you? On Thu, Aug 29, 2013 at 4:42 PM, Adam Brenner aebre...@uci.edu wrote: Hamid, Read the information here: http://snpeff.sourceforge.net/SnpEff_manual.html#galaxy -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Wed, Aug 28, 2013 at 2:16 PM, Tevfik Hamdi Kitapci tkita...@usc.edu wrote: Hi, I installed a local instance of galaxy browser on my local machine using the steps in: http://wiki.galaxyproject.org/Admin/Get%20Galaxy I tried both the default (hg update default) and stable versions (hg update stable) but the current website has different features on the left panel then the local instance that I downloaded. Specifically I want to use the SnpEff feature under the NGS toolbox Beta on the left panel of the website. But I can't find this feature on the local instance is there anyway to install a local instance of the current version of the website? Thanks Best Regards Hamdi Kitapci ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Manhattan/qq plots
Hi, Thomas. There are no resources available to support that now 3 year old rgenetics code so I'm sad but not surprised to hear of this new problem. The grant ended long ago and I no longer work with SNP so have no excuse to keep fixing or working on it. Is someone using this code willing to take it over, fix these kinds of bugs and add automated installation to bring it up to speed ? I'll help all I can but without a volunteer, the code should be considered deprecated and will soon disappear from the distribution. On Tue, Aug 27, 2013 at 2:14 AM, Bjoern Gruening bjoern.gruen...@gmail.comwrote: Hi Thomas, I think you are running a to new version of gplot http://blog.rstudio.org/2012/09/07/ggplot2-0-9-2/ opts is now deprecated and likely causes that error. Can you downgrade the gplot version? We are working on a R dependencies mechanism but we are not done yet, sorry. Hope that helps, Bjoern Hey guys, we are running a local galaxy instance (10200:fd4113962c32)with R version 3.0.1 and want to use Manhattan/qq, but we are missing the pval manhatan.pdf, only pval qqplot.pdf is generated and we are getting the following error: Nonzero exit code = 1 [1] ### 11008 values read from /opt/galaxy/galaxy-dist/database/files/002/dataset_2408.dat read - now running plots 'opts' is deprecated. Use 'theme' instead. (Deprecated; last used in version 0.9.1) Setting the plot title with opts(title=...) is deprecated. Use labs(title=...) or ggtitle(...) instead. (Deprecated; last used in version 0.9.1) Scale for 'x' is already present. Adding another scale for 'x', which will replace the existing scale. Scale for 'y' is already present. Adding another scale for 'y', which will replace the existing scale. [1] ## qqplot on pval done [1] ## manhattan on pval starting 1 2 3 Error in structure(list(call = match.call(), aesthetics = aesthetics, : argument values is missing, with no default Calls: rgqqMan ... scale_colour_manual - manual_scale - discrete_scale - structure Execution halted At the galaxy-main ist working without any problems using the same input file. All the necessary packages should be installed. Any ideas? Thanks for your help. Thomas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] DESeq for Galaxy
to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] DESeq for Galaxy
Sridhar, Kindly try rerunning the failed job to confirm that it still fails now that the tool installation should have well and truly finished - it does take a while. If it still fails the same way, please send me a copy of the installation log for the htseq dependency found in: [your tool_dependency_directory]/htseq/0.5.4p3/fubar/htseq_bams_to_count_matrix/f69b55c71ae5/INSTALLATION.log On Wed, Aug 21, 2013 at 4:38 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi Ross, i Installed through admin interface . for pysam and freetype it showed new... and rest installed.. Thanks On Wed, Aug 21, 2013 at 11:47 AM, Ross ross.laza...@gmail.com wrote: Sridhar, The error message you provide suggests that the HTSeq python module is not available when Galaxy tries to run the tool. Please confirm that you installed that tool through the admin interface from the test toolshed and that the admin interface for toolshed installed tools shows the installed repository in your Galaxy instance has the htseq dependency successfully installed like the end of the page below : - Repository Actions Installed tool shed repository 'htseq_bams_to_count_matrix' Tool shed:testtoolshed.g2.bx.psu.edu Name:htseq_bams_to_count_matrix Description: Revision:f69b55c71ae5 Owner:fubar Location:/mnt/galaxy/shed_tools/ testtoolshed.g2.bx.psu.edu/repos/fubar/htseq_bams_to_count_matrix/f69b55c71ae5/htseq_bams_to_count_matrix Deleted:False Dependencies of this repository Installed repository dependencies NameRevisionOwner Installation statuspackage_numpy_1_7http://115.146.84.239/admin_toolshed/manage_repository?id=148934b1ed5706b5 af9633757cf0blankenbergInstalledpackage_pysam_0_7_5http://115.146.84.239/admin_toolshed/manage_repository?id=96c0d701aac155a0 a4e35f23093ffubarInstalledpackage_freetype_2_4http://115.146.84.239/admin_toolshed/manage_repository?id=dee0124e4d3ab063 4e54e357ac25bgrueningInstalledpackage_matplotlib_litehttp://115.146.84.239/admin_toolshed/manage_repository?id=c32939704028ccc3 8df6bbf48c3afubarInstalled Installed tool dependencies* - click the name to browse the dependency installation directory* NameVersionTypeInstallation statusfreetypehttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=148934b1ed5706b5 2.4.11packagematplotliblitehttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=dee0124e4d3ab063 1.2.1packagenumpyhttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=94e61e06dbd29e52 1.7.1packagepysamhttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=c32939704028ccc3 0.7.5packagehtseqhttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=96c0d701aac155a0 0.5.4p3package On Wed, Aug 21, 2013 at 4:05 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi All, I installed the toolshed htseq_bams_to_count_matrix repository , i gave the gene model gtf file(ex: tss.gtf file) Gene model (GFF) file to count reads over from your current history: in this section and all bam files. it gives error as empty format: tabular, database: hg19 Traceback (most recent call last): File /data/galaxy/shed_tools/ testtoolshed.g2.bx.psu.edu/repos/fubar/htseq_bams_to_count_matrix/f69b55c71ae5/htseq_bams_to_count_matrix/htseq_bams_to_count_matrix/htseqsams2mx.py, line 45, in module import HTSeq.scripts.count as htcount Thanks On Wed, Aug 21, 2013 at 10:44 AM, Mahtab Mirmomeni m.mirmom...@student.unimelb.edu.au wrote: You can use htseq_bams_to_count_matrix repository by fubar in the test toolshed. On Wed, Aug 21, 2013 at 2:37 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi All, Thanks for the suggestions. I installed the Toolshed ( Differential_Count (version 0.21) ). It asks for the input matrix - rows are contigs, columns are counts for each sample. Any idea on how to generate this matrix file?? Thanks On Tue, Aug 20, 2013 at 12:53 PM, Mahtab Mirmomeni m.mirmom...@student.unimelb.edu.au wrote: Hi Sridhar, I'm using differential_count_model repository by fubar in the test toolshed which has both EdgeR and Deseq2 and it's working for me. Thanks Mahtab On Tue, Aug 20, 2013 at 5:03 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Sridhar, we are developing a DESeq2 tool at the moment, if you want to join get in touch with me. Its developed here: https://github.com/bgruening/galaxytools/tree/master/deseq2 I will upload our latest progress during the day. Cheers, Bjoern Dear users, Could anybody suggest the valid DESeq toolshed for RNA sequencing??? Thanks Sridhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your
Re: [galaxy-dev] DESeq for Galaxy
Sridhar, Tool shed installations always create those log files, even when they fail. I am unable to offer any further assistance without seeing the installation log for the dependency. What does: find [your tool dependency directory] -name INSTALLATION.log show? If necessary, try navigating through the tool page in your admin interface. The htseq dependency will contain the installation log I need to see. On Wed, Aug 21, 2013 at 5:06 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi Ross, I tried installing again it shows no item.. Also i couldn't find any .log file or the ( /htseq/0.5.4p3/fubar/htseq_bams_to_count_matrix) in the tool dependency directory. Thanks On Wed, Aug 21, 2013 at 12:13 PM, Ross ross.laza...@gmail.com wrote: Sridhar, Kindly try rerunning the failed job to confirm that it still fails now that the tool installation should have well and truly finished - it does take a while. If it still fails the same way, please send me a copy of the installation log for the htseq dependency found in: [your tool_dependency_directory]/htseq/0.5.4p3/fubar/htseq_bams_to_count_matrix/f69b55c71ae5/INSTALLATION.log On Wed, Aug 21, 2013 at 4:38 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi Ross, i Installed through admin interface . for pysam and freetype it showed new... and rest installed.. Thanks On Wed, Aug 21, 2013 at 11:47 AM, Ross ross.laza...@gmail.com wrote: Sridhar, The error message you provide suggests that the HTSeq python module is not available when Galaxy tries to run the tool. Please confirm that you installed that tool through the admin interface from the test toolshed and that the admin interface for toolshed installed tools shows the installed repository in your Galaxy instance has the htseq dependency successfully installed like the end of the page below : - Repository Actions Installed tool shed repository 'htseq_bams_to_count_matrix' Tool shed:testtoolshed.g2.bx.psu.edu Name:htseq_bams_to_count_matrix Description: Revision:f69b55c71ae5 Owner:fubar Location:/mnt/galaxy/shed_tools/ testtoolshed.g2.bx.psu.edu/repos/fubar/htseq_bams_to_count_matrix/f69b55c71ae5/htseq_bams_to_count_matrix Deleted:False Dependencies of this repository Installed repository dependencies NameRevisionOwner Installation statuspackage_numpy_1_7http://115.146.84.239/admin_toolshed/manage_repository?id=148934b1ed5706b5 af9633757cf0blankenbergInstalledpackage_pysam_0_7_5http://115.146.84.239/admin_toolshed/manage_repository?id=96c0d701aac155a0 a4e35f23093ffubarInstalledpackage_freetype_2_4http://115.146.84.239/admin_toolshed/manage_repository?id=dee0124e4d3ab063 4e54e357ac25bgrueningInstalledpackage_matplotlib_litehttp://115.146.84.239/admin_toolshed/manage_repository?id=c32939704028ccc3 8df6bbf48c3afubarInstalled Installed tool dependencies* - click the name to browse the dependency installation directory* NameVersionTypeInstallation statusfreetypehttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=148934b1ed5706b5 2.4.11packagematplotliblitehttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=dee0124e4d3ab063 1.2.1packagenumpyhttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=94e61e06dbd29e52 1.7.1packagepysamhttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=c32939704028ccc3 0.7.5packagehtseqhttp://115.146.84.239/admin_toolshed/browse_tool_dependency?repository_id=94e61e06dbd29e52id=96c0d701aac155a0 0.5.4p3package On Wed, Aug 21, 2013 at 4:05 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi All, I installed the toolshed htseq_bams_to_count_matrix repository , i gave the gene model gtf file(ex: tss.gtf file) Gene model (GFF) file to count reads over from your current history: in this section and all bam files. it gives error as empty format: tabular, database: hg19 Traceback (most recent call last): File /data/galaxy/shed_tools/ testtoolshed.g2.bx.psu.edu/repos/fubar/htseq_bams_to_count_matrix/f69b55c71ae5/htseq_bams_to_count_matrix/htseq_bams_to_count_matrix/htseqsams2mx.py, line 45, in module import HTSeq.scripts.count as htcount Thanks On Wed, Aug 21, 2013 at 10:44 AM, Mahtab Mirmomeni m.mirmom...@student.unimelb.edu.au wrote: You can use htseq_bams_to_count_matrix repository by fubar in the test toolshed. On Wed, Aug 21, 2013 at 2:37 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi All, Thanks for the suggestions. I installed the Toolshed ( Differential_Count (version 0.21) ). It asks for the input matrix - rows are contigs, columns are counts for each sample. Any idea on how to generate this matrix file?? Thanks On Tue, Aug 20, 2013 at 12:53 PM, Mahtab Mirmomeni m.mirmom...@student.unimelb.edu.au wrote: Hi Sridhar, I'm using differential_count_model
Re: [galaxy-dev] DESeq for Galaxy
Sridhar, If you read the tool form help text for the differential count tool, you will have noticed that right at the top, it says: Requires a count matrix as a tabular file. These are best made using the companion HTSeq based counter Galaxy wrapper and your fave gene model to generate inputs. My hope was that anyone reading that hint might realise that companion HTSeq based counter Galaxy wrapper meant they should go take a look in the same place for something with htseq in the name from the same author. It seems that I need some assistance to make the wording even more clear - suggestions will be appreciated. On Wed, Aug 21, 2013 at 2:37 PM, Sridhar Srinivasan sridhar2bioi...@gmail.com wrote: Hi All, Thanks for the suggestions. I installed the Toolshed ( Differential_Count (version 0.21) ). It asks for the input matrix - rows are contigs, columns are counts for each sample. Any idea on how to generate this matrix file?? Thanks On Tue, Aug 20, 2013 at 12:53 PM, Mahtab Mirmomeni m.mirmom...@student.unimelb.edu.au wrote: Hi Sridhar, I'm using differential_count_model repository by fubar in the test toolshed which has both EdgeR and Deseq2 and it's working for me. Thanks Mahtab On Tue, Aug 20, 2013 at 5:03 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Sridhar, we are developing a DESeq2 tool at the moment, if you want to join get in touch with me. Its developed here: https://github.com/bgruening/galaxytools/tree/master/deseq2 I will upload our latest progress during the day. Cheers, Bjoern Dear users, Could anybody suggest the valid DESeq toolshed for RNA sequencing??? Thanks Sridhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] edgeR Tool Shed
Hi, Richard, I see you are looking in the main toolshed. I cant speak for fcarmia's one but it doesn't do any dependency installation. The 'digital_dge' one was deprecated - I've now deleted it. The differential count models package in the statistics section of the test toolshed - http://testtoolshed.g2.bx.psu.edu/view/fubar/differential_count_models is ready for testing so if you have time, please try it. It installs all dependencies including R and bioconductor automatically. Installation through the galaxy admin interface should work automatically but be warned - it takes a very long time to install all the dependencies including compiling R and installing Bioconductor. You will need to generate count matrices from bam files - there's an htseq_bams_to_count_matrix tool to do that which also installs all dependencies - also in the statistics section. On Thu, Aug 15, 2013 at 1:17 AM, Richard Kuo izen...@gmail.com wrote: Hello, I have some questions about the edgeR (fcaramia) and digital_dge (fubar) repositories in tool shed. What is the difference between the two respositories in terms of being able to run edgeR? What is the proper way to install R so that the tool wrappers can use edgeR? What versions of R and edgeR do each of the repositories work with? Similaraly why is it that the repositories do not have information about which versions of R and edgeR are compatible? I have tried both repositories and have installed R and loaded the edgeR library but the tools do not seem to work. There are no errors coming up but the outputs are empty. The input file works with edgeR ran the traditional way through R (without Galaxy). Thank you, Richard ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] How to display tool instead of html WWFSMD page at start-up?
Neil, good news: the initial center panel renders static/welcome.html so easy to change. bad news: AFAIK no way to show a (dynamic) default tool form instead - others may know more.. On Mon, Jul 29, 2013 at 5:03 PM, neil.burd...@csiro.au wrote: Hi, When Galaxy is started the centre column displays the “Hello world! It’s running ...” WWFSMD? Html Page. How can this be changed so that a tool i.e. get data tool is shown up on start-up with all its text boxes etc rather than the default html file? Is this possible? ** ** Thanks Neil** ** ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Fwd: Dynamically Generated Tools
Ah. Forgive me for not paying attention to that requirement. Tools usually run as the Galaxy user so only local admins can do that at present because allowing a non-admin user to install a newly generated tool, especially on a public site opens up some interesting security challenges. I think it would be hard to sell as a new feature unless you've figured out how to prevent potential mischief from miscreant users? On Thu, Jul 18, 2013 at 11:33 PM, Michael E. Cotterell mepcotter...@gmail.com wrote: This doesn't seem to fit into the use case that my colleagues have described to me. Our tool, used within Galaxy, creates/builds/generates new tools for Web service operations. Once a user uses our tool to generate a Web service tool, it should be visible and usable to them. Currently, in order to accomplish this, we have to manually modify the tool_config.xml file, however, I don't think we should release a tool that does that. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 10:26 PM, Ross wrote: AFAIK it's not yet possible, but a tool shed API to expose subversion hooks for repository creation and update is a logical development which is already on the radar AFAIK. For the bigger picture of complexity for the user, I think some key clarifications required include: which 'user' is experiencing complexity; how big a barrier it is to generating tools; how readily can we make it go away; how much will it help? Tool builders (even 'users' of the TF) might reasonably be expected to have a capacity for substantial complexity that the end user of the generated tools should not need to possess - after all, once a generated tool is installed in Galaxy, the Galaxy user can't tell the difference and sees no more complexity from the generated tool than a hand written one. On absent local tool sheds: it's trivial to run a toolshed on a dev laptop and not hard to run in production - but tool makers can always use the public test and main TS if they want to distribute new tools so I'm not sure how big a barrier a missing local toolshed really is. On Thu, Jul 18, 2013 at 12:08 PM, Michael E. Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) wrote: I didn't mean to imply it was unclear. It definitely made sense. I completely agree with you. However, is there for a way (maybe via the toolshed api) for my tool to push these tools to a TS repo? Also, doesn't that add some more complexity for the user? Not everyone runs a local tool shed. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto: mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 10:05 PM, Ross wrote: Sorry if my post was rambling and unclear. Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance. On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto: mepcotter...@gmail.com) wrote: Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto: mepcotter...@gmail.com) (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds
Re: [galaxy-dev] Functional Test Errors for Tool Shed Tools
Hi Nilesh,
A test_data folder and contents seem to be missing from your (eg) dev
repository - the docs at http://wiki.galaxyproject.org/Tool%20Shed include
http://wiki.galaxyproject.org/TestingInstalledTools which starts with:
Tool shed repositories that contain tools with defined functional tests can
be tested when they've been installed into your local Galaxy instance using
Galaxy's functional test framework. For tests to work, the repository must
include a directory named test-data somewhere in its file system hierarchy,
and all functional test input and output datasets must be included in this
directory.
Maybe take a look at the contents of some of the devteam or Bjorn's
repositories for examples? Note that there are still some toolshed test
framework limitations like repeat groups...but simple test cases should
work once you provide the test i/o files in the repository test-data
subdirectory so they get installed when the repo is installed.
I hope this helps?
On Fri, Jul 19, 2013 at 7:44 AM, Nilesh Kavthekar nakavthe...@gmail.comwrote:
Hi everyone,
I'm running into a bit of trouble with running functional tests on tool
wrappers for rseqc that I am writing (testing according to the wiki
pagehttp://wiki.galaxyproject.org/TestingInstalledTools).
The tools are listed in shed_tool_conf.xml. Though I'm working with ~15
tool wrappers, I'm attaching the error for read_quality.py, which is
recurring for each of the other tools I'm writing. I'm attaching the link
to the xml wrapper for
read_qualityhttps://gist.github.com/nkav/abae3a22c1e4c881c2d8to show you
how I write the tests.
Here is the specific error message (this was the same message returned for
the 12 other rseqc tools I had wrapped):
==
ERROR: test_tool_00 (
functional.test_toolbox.TestForTool_testtoolshed.g2.bx.psu.edu/repos/nilesh/rseqc_dev2/read_quality/1.0.0
)
Read Quality (
testtoolshed.g2.bx.psu.edu/repos/nilesh/rseqc_dev2/read_quality/1.0.0 )
Test-1
--
Traceback (most recent call last):
File /home/nilesh/galaxy-dist/test/functional/test_toolbox.py, line
171, in test_tool
self.do_it( td, shed_tool_id=shed_tool_id )
File /home/nilesh/galaxy-dist/test/functional/test_toolbox.py, line
43, in do_it
shed_tool_id=shed_tool_id )
File /home/nilesh/galaxy-dist/test/base/twilltestcase.py, line 204, in
upload_file
filename = self.get_filename( filename, shed_tool_id=shed_tool_id )
File /home/nilesh/galaxy-dist/test/base/twilltestcase.py, line 179, in
get_filename
return os.path.abspath( os.path.join( file_dir, filename ) )
File /usr/lib/python2.7/posixpath.py, line 77, in join
elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'
begin captured stdout -
form 'login' contains the following controls ( note the values )
- end captured stdout --
I'm using the latest version of Galaxy and the RSeQC package, and the tool
dependencies for rseqc install correctly after downloading from the test
tool shed. Am I forgetting to do something here? Thanks so much for your
help.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost. Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? I think that is a better way to do it. The API can be used to install your tools. But it need to be configured beforehand from the user, administrator. 2. What's the best way to add the generated tools to Galaxy? My predecessors modified the tool_config.xml directly and required users to restart Galaxy. Now, If these tools are in a tool shed repo, I know that I can use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in a tool shed repo, however, if that's not the recommended course of action, then I'd like some advice on what to do? There is a feature request, that you can specify in your toolshed-repository the preferred location in the tool panel. For now I would ignore it and just install the tools somewhere and hope that feature request is quiet high in Gregs ToDo list :) Cheers, Bjoern ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy load problem
Hi, Ken, You don't reveal much about your local configuration making it hard to offer useful advice - and of course Postgres is a different ball of wax. FWIW I noticed much lower idle cpu loads when I switched from the default setup to running 2 handlers on our lightly loaded system here. eg http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance YMMV On Thu, Jul 18, 2013 at 12:56 AM, Auerbach, Kenneth R. kauerb...@research.bwh.harvard.edu wrote: Hello, I'm running Galaxy on a Linux server and have been getting warning messages about heavy loads on the CPU due to Galaxy. This is a dedicated Galaxy server and is currently used very lightly. There seem to be six idle postgres processes running, but only two jobs running. Any idea why the load is so high? Thank you. Ken. The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
Sorry if my post was rambling and unclear. Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance. On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell mepcotter...@gmail.com wrote: Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de (mailto: bjoern.gruen...@pharmazie.uni-freiburg.de) wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost. Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? I think that is a better way to do it. The API can be used to install your tools. But it need to be configured beforehand from the user, administrator. 2
[galaxy-dev] Fwd: Dynamically Generated Tools
AFAIK it's not yet possible, but a tool shed API to expose subversion hooks for repository creation and update is a logical development which is already on the radar AFAIK. For the bigger picture of complexity for the user, I think some key clarifications required include: which 'user' is experiencing complexity; how big a barrier it is to generating tools; how readily can we make it go away; how much will it help? Tool builders (even 'users' of the TF) might reasonably be expected to have a capacity for substantial complexity that the end user of the generated tools should not need to possess - after all, once a generated tool is installed in Galaxy, the Galaxy user can't tell the difference and sees no more complexity from the generated tool than a hand written one. On absent local tool sheds: it's trivial to run a toolshed on a dev laptop and not hard to run in production - but tool makers can always use the public test and main TS if they want to distribute new tools so I'm not sure how big a barrier a missing local toolshed really is. On Thu, Jul 18, 2013 at 12:08 PM, Michael E. Cotterell mepcotter...@gmail.com wrote: I didn't mean to imply it was unclear. It definitely made sense. I completely agree with you. However, is there for a way (maybe via the toolshed api) for my tool to push these tools to a TS repo? Also, doesn't that add some more complexity for the user? Not everyone runs a local tool shed. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 10:05 PM, Ross wrote: Sorry if my post was rambling and unclear. Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance. On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) wrote: Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto: mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de (mailto: bjoern.gruen...@pharmazie.uni-freiburg.de) (mailto: bjoern.gruen...@pharmazie.uni-freiburg.de) wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want
Re: [galaxy-dev] Using Jar files in Galaxy
Hi, Sridar, Why use the FastQC directory -tool-data/shared/jars would make it easy to simply copy the way the current FastQC tool does this - the source for FastQC xml and python wrappers are easy to find in the central repository on bitbucket? However, to do it properly so it's easy to autoinstall from a tool shed, please take a look at the test fastqc tool http://testtoolshed.g2.bx.psu.edu/view/fubar/fastqc repository which includes a tool_dependency.xml that autoinstalls the jar and sets up the path for a properly managed dependency - avoiding the ugly solution found in the existing fastqc tool. That's the way of the future for all tool wrappers, so a good thing to start using. Plenty of docs on the wiki and examples in the test toolshed. On Fri, Jul 12, 2013 at 2:34 PM, sridhar srinivasan sridhar2bioi...@gmail.com wrote: Dear Developers, I am using java jar files for softwares like fastqc, GATK. For running these tools i install the softare and set path in bash_profile file. My doubt is i placed the jar files in galaxy dist folder ( galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc ) is this enough to call the jar files from galaxy?? After placing the jar file i tried running the tool using the interface it gives ERROR as # FastQC cl = /illumina/data/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc --outdir=/illumina/data/galaxy/galaxy-dist/database/job_working_directory/000/216/dataset_546_files --f=bam /illumina/data/galaxy/galaxy-dist/database/job_working_directory/000/216/dataset_546_files/Bowtie2_on_data_7_and_data_6__aligned_reads Exception in thread main java.lang.NoClassDefFoundError: uk/ac/babraham/FastQC/FastQCApplication Caused by: java.lang.ClassNotFoundException: uk.ac.babraham.FastQC.FastQCApplication at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: uk.ac.babraham.FastQC.FastQCApplication. Program will exit. I got same error for GATK also # ERROR -- # ERROR A USER ERROR has occurred (version 2.5-2-gf57256b): # ERROR The invalid arguments or inputs must be corrected before the GATK can proceed # ERROR Please do not post this error to the GATK forum # ERROR # ERROR See the documentation (rerun with -h) for this tool to view allowable command-line arguments. # ERROR Visit our website and forum for extensive documentation and answers to # ERROR commonly asked questions http://www.broadinstitute.org/gatk # ERROR # ERROR MESSAGE: The fasta file you specified (/tmp/tmp-gatk-a3rr8t) does not exist. # ERROR - Could you please suggest on the same.. Thanks Sridhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Error: Failure Preparing Job
Unicode isn't handled nicely yet AFAIK so UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 23: ordinal not in range(128) might mean a http://www.fileformat.info/info/unicode/char/2013/index.htm - overtyping these in an ascii text editor will remove them but this is a wart for sure IMHO. You can follow our progress at fixing this on the trello card at https://trello.com/c/couxOih6 On Mon, Jul 8, 2013 at 6:28 PM, Venice Margarette Juanillas venicemargarettejuanil...@gmail.com wrote: Hi all, I'm working on a local Galaxy instance and I want to integrate TASSEL's GLM pipeline into it. But when I run the tool, it prompts: tool error An error occurred running this job:* failure preparing job* This is how the log looks like: galaxy.jobs.handler INFO 2013-07-08 16:00:47,648 (73) Job dispatched galaxy.jobs.runners.local DEBUG 2013-07-08 16:00:47,735 Local runner: starting job 73 galaxy.jobs.runners.local ERROR 2013-07-08 16:00:47,939 failure running job 73 Traceback (most recent call last): File /storage/irri-gsl-galaxy/lib/galaxy/jobs/runners/local.py, line 72, in run_job job_wrapper.prepare() File /storage/irri-gsl-galaxy/lib/galaxy/jobs/__init__.py, line 184, in prepare self.command_line = self.tool.build_command_line( param_dict ) File /storage/irri-gsl-galaxy/lib/galaxy/tools/__init__.py, line 2435, in build_command_line command_line = fill_template( self.command, context=param_dict ) File /storage/irri-gsl-galaxy/lib/galaxy/util/template.py, line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 23: ordinal not in range(128) galaxy.datatypes.metadata DEBUG 2013-07-08 16:00:48,206 Cleaning up external metadata files 127.0.0.1 - - [08/Jul/2013:16:00:50 +0800] GET /api/histories/846c4efa2e1f970a HTTP/1.1 200 - http://localhost:8081/history?status=doneshow_deleted=Falsefilename=Nonedataset_id=b89568d7c4cc60f1; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:18.0) Gecko/20100101 Firefox/18.0 I figured the problem is that UnicodeEncodeError but I don't have an idea how to fix this or where to begin. I do hope you can help me. Thanks and Best Regards, Venice Margarette B. Juanillas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Regarding galaxy wrappers: lumi, DESeq edgeR
(Sachit, please don't cross-post to the dev and user lists?) There's a beta version of an edgeR/deseq/voom wrapper and a companion htseq based count matrix maker in the test toolshed statistics section owned by me - fubar. Definitely not quite ready for production but installable and I'd appreciate feedback from anyone brave enough to try them out. Best installed on a test instance you plan on destroying because the R 3.0 dependency is about to swap over to Bjorns better one - should be done later this week when I get some time. Be warned it takes a long time to auto-install - 10-15 minutes or so on my laptop. On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Regarding galaxy wrappers: lumi, DESeq edgeR
Hi Sachit, It's packaged up so it installs from the test toolshed directly via the admin interface of your test instance, but you'll need a good reliable network as there is a lot that gets downloaded and installed. Like other toolshed tools, it would be possible but not advisable to download and unpack the archive at http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_test and install manually but your instance will need R 3.0.1 with bioconductor/edgeR/deseq/limma installed - if it's your first try, it will probably take some time and effort to read all the relevant docs to do that or the R/BioC mailing lists can provide assistance if you need it. The edgeR tool requires a count matrix (reads per sample per contig) as input which the htseq tool http://testtoolshed.g2.bx.psu.edu/view/fubar/htseq_bams_to_count_matrix will generate from any number of aligned bam/sam files and a GTF file with the appropriate gene model. Again, automatic installation from the toolshed is highly recommended, otherwise you'll need to manually install the tool and the dependencies. Note that edgeR requires replicate samples in each comparison group so the absolute minimum is 4 bam files as input to the count matrix maker. p values from analyses without replicates are not interpretable in any biologically meaningful way and are meaningless in terms of the usual frequentist interpretation in my opinion. On Mon, Jul 8, 2013 at 4:28 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Sorry about that Ross. I would try your wrapper of edgeR and would happily report you the bug. Can you make me the link of Galaxy Test Toolshed? Thanks, Sachit On Mon, Jul 8, 2013 at 11:51 AM, Ross ross.laza...@gmail.com wrote: (Sachit, please don't cross-post to the dev and user lists?) There's a beta version of an edgeR/deseq/voom wrapper and a companion htseq based count matrix maker in the test toolshed statistics section owned by me - fubar. Definitely not quite ready for production but installable and I'd appreciate feedback from anyone brave enough to try them out. Best installed on a test instance you plan on destroying because the R 3.0 dependency is about to swap over to Bjorns better one - should be done later this week when I get some time. Be warned it takes a long time to auto-install - 10-15 minutes or so on my laptop. On Mon, Jul 8, 2013 at 3:36 PM, Sachit Adhikari sachit.techner...@gmail.com wrote: Hi, These are the R libraries. Have anyone written a wrapper for these libraries lumi, DESeq edgeR? Thanks, Sachit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy Server Processes Dying?
Hi Dan, That's old code. Updating will probably help. Logging level just takes disk space, but just in case you haven't followed http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance leaving debug = True uncommented used to fill all available server process RAM eventually AFAIK. If not already done, try # Debug enables access to various config options useful for development and # debugging: use_lint, use_profile, use_printdebug and use_interactive. It # also causes the files used by PBS/SGE (submission script, output, and error) # to remain on disk after the job is complete. Debug mode is disabled if # commented, but is uncommented by default in the sample config. # debug = True Hope this helps? On Mon, Jul 8, 2013 at 4:26 PM, Dan Sullivan dansulli...@gmail.com wrote: Hi, Galaxy Developers, I have what I'm hoping is a fairly simple inquiry for the Galaxy community; basically, our production Galaxy server processes appear to be dying off over time. Our production Galaxy instance implements apache web scaling features so I have a number of server processes, for example my apache Apache configuration has: BalancerMember http://127.0.0.1:8080 BalancerMember http://127.0.0.1:8081 BalancerMember http://127.0.0.1:8082 BalancerMember http://127.0.0.1:8083 BalancerMember http://127.0.0.1:8084 BalancerMember http://127.0.0.1:8085 Nothing unconventional as I understand it. Similarly, my galaxy config has matching [server:ws3], [server:ws2] configuration blocks for each of these processes. When I restart Galaxy, everything is all fine and good. I'll see a server listening on each one of these ports (if I do something like lsof -i TCP -P, for example). What appears to be happening, is that for whatever reason, these server processes seem to die off over time (i.e eventually nothing is listening on ports 8080-8085). This process can take days, and at the time when no servers are available, Apache will begin throwing 503 service unavailable errors. I am fairly confident this process is gradual, for example I just checked now and the Galaxy was still available, however one server had died (the one on TCP port 8082). I do do have a single separate job manager and two job handlers; at this point I believe this problem to be related to the servers only (i.e. the job manager and job handlers do not app! ear to be crashing). Now, I believe that late last week I might have 'caught' the last server process dying, just by coincidence, although I am not 100% certain. Here is the Traceback as it occurred: galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:47:12,011 (6822/39485.sc01) PBS job state changed from Q to R galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,565 (6822/39485.sc01) PBS job state changed from R to C galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:36,566 (6822/39485.sc01) PBS job has completed successfully galaxy.jobs DEBUG 2013-07-02 08:54:36,685 Tool did not define exit code or stdio handling; checking stderr for success galaxy.datatypes.metadata DEBUG 2013-07-02 08:54:36,812 loading metadata from file for: HistoryDatasetAssociation 6046 galaxy.jobs DEBUG 2013-07-02 08:54:38,153 job 6822 ended galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:49,130 (6812/39473.sc01) PBS job state changed from R to E galaxy.jobs.runners.pbs DEBUG 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job state changed from E to C galaxy.jobs.runners.pbs ERROR 2013-07-02 08:54:52,267 (6812/39473.sc01) PBS job failed: Unknown error: -11 galaxy.jobs.runners ERROR 2013-07-02 08:54:52,267 (unknown) Unhandled exception calling fail_job Traceback (most recent call last): File /group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/__init__.py, line 58, in run_next method(arg) File /group/galaxy/galaxy-dist/lib/galaxy/jobs/runners/pbs.py, line 560, in fail_job if pbs_job_state.stop_job: AttributeError: 'AsynchronousJobState' object has no attribute 'stop_job' Now, I have some questions regarding this issue; 1) It appears to me that although this is a sub-optimal solution, restarting Galaxy solves this problem (i.e. server processes will be listening after restarting Galaxy). Is it possible, or safe, or sane to just restart a single server on a singe port? Ideally I would actually like to fix the problem that is causing my server processes to crash, although I figured it wouldn't hurt to ask this question regardless. 2) Similar to the question above, is it possible to configure Galaxy in a way that server processes re-spawn in a self-service manner (i.e. is this a feature of Galaxy, for example, because server processes dying regularly is either a known issue or expected and tolerable (but undesired) behaivor)? 3) To me, the error messages above aren't very meaningful, other than the Traceback appears to be PBS-related. Would anybody be able comment on the problem above (i.e. have you
Re: [galaxy-dev] Empty 'no peek' Returned Instead of HTML Output?
Did you try what was suggested below in my reply 5 days ago? Ross ross.laza...@gmail.com Jun 29 (5 days ago) to *azuzolo*, galaxy-dev Hi, Amanda, If your previous version was very old, it may have lacked a parameter added to universe_wsgi.ini some time ago to control universal sanitization of HTML as a security precaution on public sites. If it's not already there, could you try adding sanitize_all_html = False to your universe_wsgi.ini and restart Galaxy to see if that fixes the problem please? If not, please let us know and we'll try to help figure out what's going on. On Wed, Jul 3, 2013 at 8:23 PM, Amanda Zuzolo azuz...@masonlive.gmu.eduwrote: I recently updated my local testing Galaxy install to the latest stable version. When I went to test the working of HTML output in some scripts that worked in previous installs of Galaxy, the latest stable version gave me an empty return with a message of 'no peek' rather than outputting the HTML file with the embedded Javascript functionality. Additionally, the HTML output from previous runs of the same tool no longer shows the Javascript file/functionality when opened from the eye button in Galaxy, but still works fine from the HTML file opened out of the file directory. Any assistance on this issue would be greatly appreciated. -- Amanda Zuzolo Bioengineering Major, George Mason University Metabiome Informatics Group, Environmental Biocomplexity ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Empty 'no peek' Returned Instead of HTML Output?
Hi, Amanda, If your previous version was very old, it may have lacked a parameter added to universe_wsgi.ini some time ago to control universal sanitization of HTML as a security precaution on public sites. If it's not already there, could you try adding sanitize_all_html = False to your universe_wsgi.ini and restart Galaxy to see if that fixes the problem please? If not, please let us know and we'll try to help figure out what's going on. On Sat, Jun 29, 2013 at 5:16 AM, Amanda Zuzolo azuz...@masonlive.gmu.eduwrote: I recently updated my local testing Galaxy install to the latest stable version. When I went to test the working of HTML output in some scripts that worked in previous installs of Galaxy, the latest stable version gave me an empty return with a message of 'no peek' rather than outputting the HTML file with the embedded Javascript functionality. Additionally, the HTML output from previous runs of the same tool no longer shows the Javascript file/functionality when opened from the eye button in Galaxy, but still works fine from the HTML file opened out of the file directory. Any assistance on this issue would be greatly appreciated. -- Amanda Zuzolo Bioengineering Major, George Mason University Metabiome Informatics Group, Environmental Biocomplexity ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Missing test results on (Test) Tool Shed
I'm not seeing anything other than the test environment report in the failing tests report for http://testtoolshed.g2.bx.psu.edu/view/fubar/htseq_bams_to_count_matrix - if you have time to take a look Dave? I'm having a hard time figuring out what I've done wrong - it builds, installs and tests fine on my test machine but I've probably forgotten some dependency that I already have installed on my laptop... The result for http://testtoolshed.g2.bx.psu.edu/view/fubar/edger_testmakes me think that one or more of the test input files is missing but it's hard to tell. I wonder if overall we need a more or less complete output data spew for failing tests to make the task of tracking them down easier? On Thu, Jun 27, 2013 at 8:58 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Wed, Jun 26, 2013 at 10:01 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, The empty failed tests folder issue has been resolved in 10171:eaa5112fefcd, and the test tool shed has been updated to that revision. --Dave B. Great - currently all the missing test failures have been resolved :) The bad news is there seems to be a new sniffer bug. http://testtoolshed.g2.bx.psu.edu/view/peterjc/blast2go/f3a61c2cf309 Can see test failures (tool configuration problem). http://testtoolshed.g2.bx.psu.edu/view/peterjc/effectivet3/392279f2e120 One test pass, on test failure. The failure is a bit odd and is likely a new sniffer related bug. http://testtoolshed.g2.bx.psu.edu/view/peterjc/get_orfs_or_cdss/26ce6e046040 Tests pass. http://testtoolshed.g2.bx.psu.edu/view/peterjc/mira_assembler/da604cc07658 Can see test failure, good. http://testtoolshed.g2.bx.psu.edu/view/peterjc/nlstradamus/2be36fa7565e One pass, one failure - same new failure as effectiveT3 above. http://testtoolshed.g2.bx.psu.edu/view/peterjc/seq_rename/44b2e489e26f Can see a failure, also seems to be sniffer related (but using a different datatype). http://testtoolshed.g2.bx.psu.edu/view/peterjc/tmhmm_and_signalp/4cd848c5590b As before, no test results as there are marked as skip - yet this is still listed under Latest revision: failing tool tests. The filter for this needs to exclude repositories marked as skip testing. http://testtoolshed.g2.bx.psu.edu/view/peterjc/venn_list/51fe47a5a803 Can see test failure, good (missing undeclared dependency) Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visulaization tool
Shashi, Hans made a brave guess based on your vague question, but you'll probably get better answers if you do us all a favour and make an effort to provide unambiguous detail about exactly what you want to achieve. If there is a working Galaxy tool that does what you need but using a different visualisation program (whatever you mean by that vague term), substituting your executable will probably be relatively trivial. eg do you consider the Html object produced by the fastqc or the image from the scatterplot tool to be visualisation program outputs? If so, clone a working tool. On Wed, Jun 19, 2013 at 5:37 PM, shashi shekhar me_shash...@yahoo.co.inwrote: Hi, I want to add the the complete separate tool in galaxy . how will i do it ..i don't want to use existing visulaization tool. Regards shashi -- *From:* Hans-Rudolf Hotz h...@fmi.ch *To:* shashi shekhar me_shash...@yahoo.co.in *Cc:* galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu *Sent:* Wednesday, 19 June 2013 12:29 PM *Subject:* Re: [galaxy-dev] visulaization tool Hi Shashi Have a look at: http://wiki.galaxyproject.org/Learn/Visualization http://wiki.galaxyproject.org/Visualization%20Setup Regards, Hans-Rudolf On 06/19/2013 08:07 AM, shashi shekhar wrote: Hi, i want to add new visualization tool in galaxy . can i get the required documents ? can u send me the details how can i add the visualization tool in galaxy . Regards shashi ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visulaization tool
Shashi, Thanks for the additional information. AFAIK: If it's command line driven and spews out pdf's, adapt any similar tool that spews out pdfs like fastqc. Galaxy tool execution is isolated from the user and from Galaxy itself, so communication is just not possible if it's interactive - unless you hack the tool into Galaxy itself or something. There is no documentation for that. On Wed, Jun 19, 2013 at 9:01 PM, shashi shekhar me_shash...@yahoo.co.inwrote: Hi Ross, thanks for the reply . actually i want to use TreeView as visulaization tool. TreeView X is an open source and multi-platform program to display phylogenetic trees. It can read and display NEXUS and Newick format tree files (such as those output by PAUP*, ClustalX, TREE-PUZZLE, and other programs). It allows one to order the branches of the trees, and to export the trees in SVG format. i have to add treeviewx in galaxy.. Regards shashi -- *From:* Ross ross.laza...@gmail.com *To:* shashi shekhar me_shash...@yahoo.co.in *Cc:* Hans-Rudolf Hotz h...@fmi.ch; galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu *Sent:* Wednesday, 19 June 2013 3:28 PM *Subject:* Re: [galaxy-dev] visulaization tool Shashi, Hans made a brave guess based on your vague question, but you'll probably get better answers if you do us all a favour and make an effort to provide unambiguous detail about exactly what you want to achieve. If there is a working Galaxy tool that does what you need but using a different visualisation program (whatever you mean by that vague term), substituting your executable will probably be relatively trivial. eg do you consider the Html object produced by the fastqc or the image from the scatterplot tool to be visualisation program outputs? If so, clone a working tool. On Wed, Jun 19, 2013 at 5:37 PM, shashi shekhar me_shash...@yahoo.co.inwrote: Hi, I want to add the the complete separate tool in galaxy . how will i do it ..i don't want to use existing visulaization tool. Regards shashi -- *From:* Hans-Rudolf Hotz h...@fmi.ch *To:* shashi shekhar me_shash...@yahoo.co.in *Cc:* galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu *Sent:* Wednesday, 19 June 2013 12:29 PM *Subject:* Re: [galaxy-dev] visulaization tool Hi Shashi Have a look at: http://wiki.galaxyproject.org/Learn/Visualization http://wiki.galaxyproject.org/Visualization%20Setup Regards, Hans-Rudolf On 06/19/2013 08:07 AM, shashi shekhar wrote: Hi, i want to add new visualization tool in galaxy . can i get the required documents ? can u send me the details how can i add the visualization tool in galaxy . Regards shashi ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] visulaization tool
Shashi, Beats me. Cloning the visualisation code would be a good start I'd guess. It will not be trivial. Others may have better ideas. On Wed, Jun 19, 2013 at 9:56 PM, shashi shekhar me_shash...@yahoo.co.inwrote: Hi Ross, Actually i have to add the visualization which will be interactive . it will take dataset from galaxy .it will be in the same way as tracker works . in stead of tracker i want to add my own visualization tool. from wehere should i start to replace tracker with my own visulization tool. Regards shashi -- *From:* Ross ross.laza...@gmail.com *To:* shashi shekhar me_shash...@yahoo.co.in *Cc:* Hans-Rudolf Hotz h...@fmi.ch; galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu *Sent:* Wednesday, 19 June 2013 5:02 PM *Subject:* Re: [galaxy-dev] visulaization tool Shashi, Thanks for the additional information. AFAIK: If it's command line driven and spews out pdf's, adapt any similar tool that spews out pdfs like fastqc. Galaxy tool execution is isolated from the user and from Galaxy itself, so communication is just not possible if it's interactive - unless you hack the tool into Galaxy itself or something. There is no documentation for that. On Wed, Jun 19, 2013 at 9:01 PM, shashi shekhar me_shash...@yahoo.co.inwrote: Hi Ross, thanks for the reply . actually i want to use TreeView as visulaization tool. TreeView X is an open source and multi-platform program to display phylogenetic trees. It can read and display NEXUS and Newick format tree files (such as those output by PAUP*, ClustalX, TREE-PUZZLE, and other programs). It allows one to order the branches of the trees, and to export the trees in SVG format. i have to add treeviewx in galaxy.. Regards shashi -- *From:* Ross ross.laza...@gmail.com *To:* shashi shekhar me_shash...@yahoo.co.in *Cc:* Hans-Rudolf Hotz h...@fmi.ch; galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu *Sent:* Wednesday, 19 June 2013 3:28 PM *Subject:* Re: [galaxy-dev] visulaization tool Shashi, Hans made a brave guess based on your vague question, but you'll probably get better answers if you do us all a favour and make an effort to provide unambiguous detail about exactly what you want to achieve. If there is a working Galaxy tool that does what you need but using a different visualisation program (whatever you mean by that vague term), substituting your executable will probably be relatively trivial. eg do you consider the Html object produced by the fastqc or the image from the scatterplot tool to be visualisation program outputs? If so, clone a working tool. On Wed, Jun 19, 2013 at 5:37 PM, shashi shekhar me_shash...@yahoo.co.inwrote: Hi, I want to add the the complete separate tool in galaxy . how will i do it ..i don't want to use existing visulaization tool. Regards shashi -- *From:* Hans-Rudolf Hotz h...@fmi.ch *To:* shashi shekhar me_shash...@yahoo.co.in *Cc:* galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu *Sent:* Wednesday, 19 June 2013 12:29 PM *Subject:* Re: [galaxy-dev] visulaization tool Hi Shashi Have a look at: http://wiki.galaxyproject.org/Learn/Visualization http://wiki.galaxyproject.org/Visualization%20Setup Regards, Hans-Rudolf On 06/19/2013 08:07 AM, shashi shekhar wrote: Hi, i want to add new visualization tool in galaxy . can i get the required documents ? can u send me the details how can i add the visualization tool in galaxy . Regards shashi ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] dynamically load and run R scripts
Hi Jens, the toolfactory (install from the admin menu - it's in the main toolshed) allows admin users to paste and run arbitrary R scripts. Yes, there are serious security issues exposing unrestricted scripting! So the toolfactory only works for admin users and should never be installed on public/production instance but you can generate a new Galaxy tool from your script once it runs correctly, in a toolshed compatible format. Once installed from a toolshed, the generated tools are ordinary Galaxy tools with normal Galaxy security. For gory details see https://bitbucket.org/fubar/galaxytoolfactory On Mon, Jun 17, 2013 at 11:32 PM, Keilwagen, Jens jens.keilwa...@jki.bund.de wrote: Hi guys, I was searching for a possiblity to dynamically load and run R scripts and I found http://dev.list.galaxyproject.org/dynamic-loading-of-tools-tt4138600.html I know that it might be a security issue, but we only have a local instance here. Optimally an R script can be loaded into the history. Subsequently, this script can be execute via tool that takes this script and some parameters and returns the result. Is anything like that out there? Could anyone give me a reference? Thanks a lot, Jens ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Adding a Tool for PANAMA
Hi Saket, I haven't looked at this but I noticed the following advice on their web site: If you want to call PANAMA directly from python, just take a look at the PANAMA function inpanama.core.run. Do you have a plan for representing the required incoming expression and snp matrices in Galaxy? AFAIK they need to be from the same subjects - so probably need some thought to make it optimal for a biologist user On Tue, Jun 4, 2013 at 8:39 PM, Saket Choudhary sake...@gmail.com wrote: Thanks Ross and Peter. That worked. I think I will keep the Python wrapper though, gives me more control :) Saket On 3 June 2013 16:48, Ross ross.laza...@gmail.com wrote: Hi Saket If you want to redirect the executable's stdout to a new Galaxy dataset you've defined as $output, something like: gt; '$output' is a workable idiom for a command line AFAIK - you need to escape the redirection '' character or you'll confuse the xml parser. It's often possible to hack a complex command line (or use configfile to write a script!) but as Peter says, it all boils down to personal preference - they all work. Use whatever you find most convenient but remember you may need to maintain it :) On Mon, Jun 3, 2013 at 9:05 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Mon, Jun 3, 2013 at 11:53 AM, Saket Choudhary sake...@gmail.com wrote: As part of a warmup, I added a tool for perming eQTL analysis using PANAMA(http://ml.sheffield.ac.uk/qtl/panama/). By default PANAMA writes to stderr and creates a PANAM_results.csv file in the same directory as the input files. I wrote ... XML : https://gist.github.com/saketkc/5697388 Apparently the above XML runs too, but I dont get back a CSV. Essentially I need to move the PANAMA_results.csv file to output1. Can this be done without using a wrapper at all , just using the XML files ? Is there a way to nest the commands in the XML so that I could do a move command after panama command , but how would the paths be specified ? Hi Saket, You could try this (two commands in one shell line), but catching error conditions would be a problem: commandpanama $expression_data $snp_data; mv PANAMA_results.csv $output1/command In cases like this where the underlying tool is too inflexible to accept an output filename, I would personally use a wrapper script which can handle moving things to the desired location, and raising a clear error if the file was not created. Peter -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BWA installation
Hi, מירב Sounds like you might want to try reindexing and make sure you have plenty of disk space - I have a .rbwt file from the last time I indexed using BWA version: 0.6.2-r126 and it works fine for me. On Sun, Jun 2, 2013 at 4:02 PM, מירב . mer...@gmail.com wrote: Thanks fo your answer, But when I am tring to use the bwa and to do Map with BWA for Illuminahttp://127.0.0.1:8080/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/bwa_wrapper/1.2.3 I'm getting this error: The alignment failed. Error aligning sequence. [bwa_aln] 17bp reads: max_diff = 2 [bwa_aln] 38bp reads: max_diff = 3 [bwa_aln] 64bp reads: max_diff = 4 [bwa_aln] 93bp reads: max_diff = 5 [bwa_aln] 124bp reads: max_diff = 6 [bwa_aln] 157bp reads: max_diff = 7 [bwa_aln] 190bp reads: max_diff = 8 [bwa_aln] 225bp reads: max_diff = 9 [bwt_restore_bwt] fail to open file '/home/user/extra_galaxy_project/bwa/data/hg18full/hg18full.fa.rbwt'. Abort! Aborted (core dumped) It seems like the .rbwt file is missing and cause the error?! 2013/5/30 Carl Eberhard carlfeberh...@gmail.com Hello, מירב It looks like the indexing system/format in 'bwa index' has changed and we haven't updated the wiki to reflect the changes. The reverse index (which includes the .r* files) are now part of the forward index. You should be fine to proceed. I'll update the wiki to reflect this. From https://github.com/lh3/bwa/blob/master/NEWS : Release 0.5.10 and 0.6.0 (12 November, 2011) The 0.6.0 release comes with two major changes. Firstly, the index data structure has been changed to support genomes longer than 4GB. The forward and reverse backward genome is now integrated in one index. On Thu, May 30, 2013 at 8:03 AM, מירב . mer...@gmail.com wrote: Hello all, I am trying to install BWA , I used the command: bwa index -a bwtsw /global_data/hg18full.fa and I got only 5 files insted of 8 The missing files are - rbwt - rpac - .rsa I didn't recieve any error message while running. thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Adding a Tool for PANAMA
Hi Saket If you want to redirect the executable's stdout to a new Galaxy dataset you've defined as $output, something like: gt; '$output' is a workable idiom for a command line AFAIK - you need to escape the redirection '' character or you'll confuse the xml parser. It's often possible to hack a complex command line (or use configfile to write a script!) but as Peter says, it all boils down to personal preference - they all work. Use whatever you find most convenient but remember you may need to maintain it :) On Mon, Jun 3, 2013 at 9:05 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Mon, Jun 3, 2013 at 11:53 AM, Saket Choudhary sake...@gmail.com wrote: As part of a warmup, I added a tool for perming eQTL analysis using PANAMA(http://ml.sheffield.ac.uk/qtl/panama/). By default PANAMA writes to stderr and creates a PANAM_results.csv file in the same directory as the input files. I wrote ... XML : https://gist.github.com/saketkc/5697388 Apparently the above XML runs too, but I dont get back a CSV. Essentially I need to move the PANAMA_results.csv file to output1. Can this be done without using a wrapper at all , just using the XML files ? Is there a way to nest the commands in the XML so that I could do a move command after panama command , but how would the paths be specified ? Hi Saket, You could try this (two commands in one shell line), but catching error conditions would be a problem: commandpanama $expression_data $snp_data; mv PANAMA_results.csv $output1/command In cases like this where the underlying tool is too inflexible to accept an output filename, I would personally use a wrapper script which can handle moving things to the desired location, and raising a clear error if the file was not created. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] [GSoC 2013] Introduction
Hi, Saket. Welcome aboard - help is always available here or irc if you're in the right timezone. https://chgr.mc.vanderbilt.edu/plato/https://chgr.mc.vanderbilt.edu/plato/steps/3 is a brand new eQTL option from Marylyn Ritchie's group I stumbled across at http://gettinggeneticsdone.blogspot.com.au/ today - the comments about plink are apt - it can be a pain to work with in my experience. On Fri, May 31, 2013 at 4:35 PM, Saket Choudhary sake...@gmail.com wrote: Hi Everyone! I have been selected to work on the eQTL pipeline and visualisation for the Galaxy Project as part of the Google Summer of Code 2013 ( http://www.google-melange.com/gsoc/homepage/google/gsoc2013) I am pursuing my fourth year of undergraduate studies at Indian Institute of Technology Bombay(http://www.iitb.ac.in/). My areas of interest are broadly Bioinformatics and of lately I have been working on NGS problems. I have created a blog here : http://galaxy-gsoc2013.blogspot.com/ . The first post should be up soon. My proposal is here : http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/saketkc/51001. I would love to know your suggestions/comments. I am relatively new to Galaxy, and some of my questions in the process might be elementary, so kindly bear with it :) Thanks ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] installing galaxy
Hi, Merav. A fully functional Galaxy server relies on a lot of third party software and the error message you see is probably because you do not have samtools available - the upload process needs it to make an index for your bam file. So the solution may be to install the samtools package - eg for ubuntu something like: sudo apt-get install samtools After that, if you execute samtools at a console, you should see something like: rlazarus@iaas1-int:~$ samtools Program: samtools (Tools for alignments in the SAM format) Version: 0.1.18 (r982:295) Usage: samtools command [options] Command: viewSAM-BAM conversion sortsort alignment file mpileup multi-way pileup Please take a look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies On Wed, May 29, 2013 at 8:00 PM, מירב . mer...@gmail.com wrote: hello, I am installing galaxy locally. I tried to upload a .bam file and I got this message: Traceback (most recent call last): File /home/user/galaxy_project/tools/data_source/upload.py, line 386, in module __main__() File /home/user/galaxy_project/tools/data_source/upload.py, line 375, in __main__ add_file( dataset, registry, json_file, output_path ) File /home/user/galaxy_project/tools/data_source/upload.py, line 314, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /usr/lib/python2.7/subprocess.py, line 679, in __init__ errread, errwrite) File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This file uploaded successfully, to the galaxy on the net. what can it be? thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Fwd: installing galaxy
-- Forwarded message -- From: מירב . mer...@gmail.com Date: Wed, May 29, 2013 at 8:40 PM Subject: Re: [galaxy-dev] installing galaxy To: Ross ross.laza...@gmail.com It worked!! Thank you very mach [?] Merav 2013/5/29 Ross ross.laza...@gmail.com Hi, Merav. A fully functional Galaxy server relies on a lot of third party software and the error message you see is probably because you do not have samtools available - the upload process needs it to make an index for your bam file. So the solution may be to install the samtools package - eg for ubuntu something like: sudo apt-get install samtools After that, if you execute samtools at a console, you should see something like: rlazarus@iaas1-int:~$ samtools Program: samtools (Tools for alignments in the SAM format) Version: 0.1.18 (r982:295) Usage: samtools command [options] Command: viewSAM-BAM conversion sortsort alignment file mpileup multi-way pileup Please take a look at http://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies On Wed, May 29, 2013 at 8:00 PM, מירב . mer...@gmail.com wrote: hello, I am installing galaxy locally. I tried to upload a .bam file and I got this message: Traceback (most recent call last): File /home/user/galaxy_project/tools/data_source/upload.py, line 386, in module __main__() File /home/user/galaxy_project/tools/data_source/upload.py, line 375, in __main__ add_file( dataset, registry, json_file, output_path ) File /home/user/galaxy_project/tools/data_source/upload.py, line 314, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 113, in dataset_content_needs_grooming version = self._get_samtools_version() File /home/user/galaxy_project/lib/galaxy/datatypes/binary.py, line 97, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File /usr/lib/python2.7/subprocess.py, line 679, in __init__ errread, errwrite) File /usr/lib/python2.7/subprocess.py, line 1259, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This file uploaded successfully, to the galaxy on the net. what can it be? thanks for your help merav ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J 328.png___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Tool Shed test framework breaking where run_functional_tests.sh works
Buildbot can be triggered here http://buildbot.g2.bx.psu.edu/builders/central-py2.7-linux-x86_64-postgres manually I think. On Fri, May 10, 2013 at 10:56 PM, Greg Von Kuster g...@bx.psu.edu wrote: Is there a more optimal time to initiate this testing process to better accommodate those in other time zones? If anyone is waiting for the nightly test results, we should tweak cron so no waiting is necessary (if possible). On May 10, 2013, at 8:53 AM, Dave Bouvier wrote: That is correct, the tests are run nightly, some time after midnight eastern US time, unless I manually initiate the testing process. --Dave B. On 5/10/13 05:36:41.000, Peter Cock wrote: On Thu, May 9, 2013 at 6:59 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, As of my latest commit, the automated test results should now match that of run_functional_tests.sh. --Dave B. Thanks Dave, Presumably once the Test Tool Shed is updated, all the old tests will be re-run, and so I should see the revised results for this tool then? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Command Line Tool for Creating New Users
Neil, AFAIK, python will not load a module unless the relevant script file is marked as executable by the current user? Try something like: chmod ugo+x scripts/db_shell.py and see if that fixes your problem? On Tue, Apr 30, 2013 at 11:50 AM, neil.burd...@csiro.au wrote: Hi, I'm trying to use the tool, however, when I run it I get: python create_galaxy_users.py Traceback (most recent call last): File create_galaxy_users.py, line 2, in module from scripts.db_shell import * ImportError: No module named scripts.db_shell I've set my PYTHONPATH to be ~/galaxy-dist/lib. What else do I need to do? I'm running the script from ~/galaxy-dist/ directory, but I get the same error if I copy it into the scripts directory. I've checked and I can see that the scripts.db_shell.py script exists If I comment out the line then the next error I get is: python create_galaxy_users.py Traceback (most recent call last): File create_galaxy_users.py, line 7, in module from galaxy.security import GalaxyRBACAgent File /home/galaxy/galaxy-dist/lib/galaxy/security/__init__.py, line 9, in module from galaxy.model.orm import * File /home/galaxy/galaxy-dist/lib/galaxy/model/__init__.py, line 15, in module from galaxy.security import RBACAgent, get_permitted_actions ImportError: cannot import name RBACAgent Any help much appreciated Neil -- Message: 1 Date: Sat, 27 Apr 2013 14:07:49 -0500 From: John Chilton chil...@msi.umn.edu To: Adam Brenner aebre...@uci.edu Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Command Line Tool for Creating New Users Message-ID: canwbokctvxo-dfmamr77fu6bbdhgr1stg7mph_lp+fe2nmc...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 I put together a script to do this. It is described in the galaxy-dev thread titled: [galaxy-dev] user creation using API. It doesn't actually use the API, it uses the db_shell.py script. Hope this helps. -John ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Command Line Tool for Creating New Users
Funny, it works for me. One last thought: You do still have the distributed __init__.py readable by your userid in /scripts so the module loader knows it's a python package directory? On Tue, Apr 30, 2013 at 12:05 PM, neil.burd...@csiro.au wrote: Thanks for the info Ross ** ** But I still get the same error ** ** “ImportError: No module named scripts.db_shell” ** ** After chmod 777 scripts/db_shell.py ** ** When you get “ImportError: No module named galaxy” for example it’s because the PYTHONPATH isn’t set so I guess there’s some variable I need to set for the scripts directory??? ** ** Thanks anyway ** ** Neil ** ** *From:* Ross [mailto:ross.laza...@gmail.com] *Sent:* Tuesday, 30 April 2013 12:00 PM *To:* Burdett, Neil (ICT Centre, Herston - RBWH) *Cc:* galaxy-dev@lists.bx.psu.edu *Subject:* Re: [galaxy-dev] Command Line Tool for Creating New Users ** ** Neil, AFAIK, python will not load a module unless the relevant script file is marked as executable by the current user? Try something like: chmod ugo+x scripts/db_shell.py and see if that fixes your problem? ** ** ** ** On Tue, Apr 30, 2013 at 11:50 AM, neil.burd...@csiro.au wrote: Hi, I'm trying to use the tool, however, when I run it I get: python create_galaxy_users.py Traceback (most recent call last): File create_galaxy_users.py, line 2, in module from scripts.db_shell import * ImportError: No module named scripts.db_shell I've set my PYTHONPATH to be ~/galaxy-dist/lib. What else do I need to do? I'm running the script from ~/galaxy-dist/ directory, but I get the same error if I copy it into the scripts directory. I've checked and I can see that the scripts.db_shell.py script exists If I comment out the line then the next error I get is: python create_galaxy_users.py Traceback (most recent call last): File create_galaxy_users.py, line 7, in module from galaxy.security import GalaxyRBACAgent File /home/galaxy/galaxy-dist/lib/galaxy/security/__init__.py, line 9, in module from galaxy.model.orm import * File /home/galaxy/galaxy-dist/lib/galaxy/model/__init__.py, line 15, in module from galaxy.security import RBACAgent, get_permitted_actions ImportError: cannot import name RBACAgent Any help much appreciated Neil -- Message: 1 Date: Sat, 27 Apr 2013 14:07:49 -0500 From: John Chilton chil...@msi.umn.edu To: Adam Brenner aebre...@uci.edu Cc: galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Command Line Tool for Creating New Users Message-ID: canwbokctvxo-dfmamr77fu6bbdhgr1stg7mph_lp+fe2nmc...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 I put together a script to do this. It is described in the galaxy-dev thread titled: [galaxy-dev] user creation using API. It doesn't actually use the API, it uses the db_shell.py script. Hope this helps. -John -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Include some XML in configfiles tag
Hi, David.
does enclosing your XML in a CDATA section help? It should just get written
out without being interpreted, and it works for me in the toolfactory - the
contents still get written out but without the CDATA section.
configfiles
configfile name=loadTestxml
![CDATA[
*!DOCTYPE crystal SYSTEM /usr/share/sgml/crystal.dtd
!--- Load a pos file called babla.pos --
crystal
version value=0.0.1/
posload name=${posfile}/
/crystal*
]]*
* /configfile
/configfiles
On Fri, Mar 8, 2013 at 4:23 PM, David Angot dav...@intersect.org.au wrote:
Hello,
I have a tool that takes an XML file as input.
So I thought it would be a good idea to put it in the configfiles tag,
to replace some of the variables.
Quick example :
inputs
param name=posfile type=data label=Posfile help=Posfile you want
to input to POSGEN format=pos /
/inputs
configfiles
configfile name=loadTestxml
*!DOCTYPE crystal SYSTEM /usr/share/sgml/crystal.dtd
!--- Load a pos file called babla.pos --
crystal
version value=0.0.1/
posload name=${posfile}/
/crystal
* /configfile
/configfiles
But when I load the tool configuration, it seems that the XML in
configfile is interpreted, and/or I don't get anything in the input file.
Is there a way to do what I want ?
Cheers,
--
*David *
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Built-in reference genome
Hi, Charles, builds.txt is what tells the Galaxy framework about available genomes. Various binary index files created from those genomes are separately managed for specific tools since some folks won't need some application specific sets of indexes for any given genome - so you need to edit a different file! The BWA for illumina tool for example gets BWA index files from tool-data/bwa_index.loc so that's where the reference to your ecoli.fa and related BWA index files needs to be added - the format is described in the comments you'll find when you edit that file. Tabs for delimiters again and another restart should fix your problem for that tool? On Wed, Mar 6, 2013 at 11:46 AM, Charles Murphy murphy.charl...@gmail.comwrote: Hi, I' am trying to add built-in reference genomes for my local galaxy installation. I know this topic has been covered multiple times: I have looked through the discussion threads and the wiki (e.g. NGS Local Setup), but I' am still unsuccessful. Previously I have just loaded a reference genome through the history, but in the interest of saving time and space while I experiment with Galaxy I wish to use the built-in feature. Here is what I have tried: 1. Indexed ecoli.fa using bwa and put all files (fasta and index files) under /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ The directory looks like: -rwxrwxrwx@ 1 charlesmurphy staff 4706043 Mar 5 14:54 ecoli.fa -rwxrwxrwx 1 charlesmurphy staff 12 Mar 5 16:38 ecoli.fa.amb -rwxrwxrwx 1 charlesmurphy staff 111 Mar 5 16:38 ecoli.fa.ann -rwxrwxrwx 1 charlesmurphy staff 4639752 Mar 5 16:38 ecoli.fa.bwt -rwxrwxrwx 1 charlesmurphy staff 1159920 Mar 5 16:38 ecoli.fa.pac -rwxrwxrwx 1 charlesmurphy staff 2319888 Mar 5 16:38 ecoli.fa.sa 2. Downloaded and installed the bwa wrapper files from the Galaxy main toolshed, and put bwa in $PATH. 3. Added the following line to the bwa_index.loc (using tabs as the delimiter): ecoli ecoli ecoli /Users/charlesmurphy/Desktop/Research/data/genome/ecoli/bwa/ecoli.fa 4. Added the following line to builds.txt (using tabs as the delimiter): ecoli ecoli 5. Restarted Galaxy. The ecoli genome does not show up as a reference genome for BWA (Map with BWA for Illumina). Any help would be appreciated. Thanks, Charlie ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] all the data files in .loc files needed to download ?
Hi, shenwiyn I'd also add: Fourth, replace distributed default sqlite with postgresql to your list if it's for any serious use. Sqlite is ok for testing. On Wed, Feb 27, 2013 at 9:37 PM, shenwiyn shenw...@gmail.com wrote: ** **Hello Carl, Thank you very much for your help.I have some thoughts about installing my local galaxy : First,download the latest source of galaxy from https://bitbucket.org/galaxy/galaxy-dist/ ,then install it in my local computer. Second,install dependencies tools mentioned in http://wiki.galaxyproject.org/Admin/Data%20Integration successfully. Third ,install the needed data and configure the .loc files from the following help of wiki page http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup. Finally ,our installation of local galaxy overs and it works in the mass.Is it right ? And I also have some confusions: First ,How large the total needed data is ?Is all the data in datacache.g2.bx.psu.edu/indexes needed ? Second, can all of these needed data be downloaded (via rsync) by the help of http://wiki.galaxyproject.org/Admin/Data%20Integration? These are the most important questions I concerned about now.I am so thankful for some of you advice. Thank you very much. -- shenwiyn *From:* Carl Eberhard carlfeberh...@gmail.com *Date:* 2013-02-23 02:42 *To:* shenwiyn shenw...@foxmail.com *CC:* galaxy-dev@lists.bx.psu.edu *Subject:* Re: [galaxy-dev] all the data files in .loc files needed to download ? Hello shenwiyn, You may want to have a look at http://wiki.galaxyproject.org/Admin/Data%20Integration which provides more information specific to the indeces/data you'll need and how to install. Along with data management, that page has instructions on how to download (via rsync) the same data we use on our main server. It may be a good starting point. Thanks, Carl On Thu, Feb 21, 2013 at 8:58 PM, shenwiyn shenw...@foxmail.com wrote: ** Hi Carl Eberhard, Thank you very much for you help.I have another some questions : First, we need to install the needed data,for example sam_fa_indices.loc file of SAM Tools : index AaegL1 /afs/bx.psu.edu/depot/data/genome/AaegL1/sam_index/AaegL1 .fa index AgamP3 /afs/bx.psu.edu/depot/data/genome/AgamP3/sam_index/AgamP3 .fa index ailMel1 /afs/bx.psu.edu/depot/data/genome/ailMel1/sam_index/ ailMel1.fa index anoCar2 /afs/bx.psu.edu/depot/data/genome/anoCar2/sam_index/ anoCar2.fa index apiMel2 /afs/bx.psu.edu/depot/data/genome/apiMel2/sam_index/ apiMel2.fa index apiMel3 /afs/bx.psu.edu/depot/data/genome/apiMel3/sam_index/ apiMel3.fa index aplCal1 /afs/bx.psu.edu/depot/data/genome/aplCal1/sam_index/ aplCal1.fa so we need to download AaegL1.fa,AgamP3.fa,ailMel1.fa,anoCar2.fa,apiMel2. fa,apiMel3.fa,and so on ,then install all of this needed datas to /afs/bx .psu.edu/depot/data/genome/aplCal1/sam_index/,is it right? Second, from the needed datas mentioned in the .loc files in http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup ,we need to download too much data,can we get all these data from some website instead of searching them on Iternet one by one? Thanks, shenwiyn -- *From:* Carl Eberhard carlfeberh...@gmail.com *Date:* 2013-02-22 01:20 *To:* shenwiyn shenw...@gmail.com *Subject:* Re: [galaxy-dev] all the data files in .loc files needed to download ? Hello, Yes a local galaxy requires a few steps in order to install the needed data. The following wiki page should help get you started: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup Let me know if you need more information, Carl On Tue, Feb 19, 2013 at 3:59 AM, shenwiyn shenw...@gmail.com wrote: ** Hi all, Are all the data files in .loc files needed to download and put to the right directory when we install our local galaxy? thaks shenwiyn -- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How to test DRMAA configuration?
See if adding the default queue name to the job runner path - eg: default_cluster_job_runner = drmaa:///default works any better? Galaxy will default to the local runner if it can't find the nominated drmaa path AFAIK and I don't think 'default' is the default :) On Tue, Feb 26, 2013 at 4:51 PM, Joshua Orvis jor...@gmail.com wrote: I have a working local Galaxy instance and wanted to enable DRMAA support to utilize our SGE (or LSF) grid. Following the guide herehttp://wiki.galaxyproject.org/Admin/Config/Performance/Cluster I set what I appeared to need to make this work. From the DRMAA_LIBRARY_PATH env variable to all the configuration settings in universe_wsgi.ini, reconfiguring the server hosting Galaxy as a submit host, etc. Some specific config file changes made: new_file_path = /seq/gscidA/www/gscid_devel/htdocs/galaxy-dist/database/tmp start_job_runners = drmaa default_cluster_job_runner = drmaa:/// set_metadata_externally = True outputs_to_working_directory = True I then killed and restarted the Galaxy instance and tried a simple FASTQ - FASTA test execution, but it ran locally. I couldn't find any sort of errors or messages related to DRMAA in the server log, and the job ran to completion. I commented out the local tool runner overrides. What can I do to test my DRMAA configuration and where should I look for errors? Thanks - Joshua ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] History not updating for multiple instances of Galaxy
untested maybe the cookie path for each instance defined in universe_wsgi.ini needs to be distinguishable if you're swapping between authenticated instances ? /untested # If proxy-prefix is enabled and you're running more than one Galaxy instance # behind one hostname, you will want to set this to the same path as the prefix # in the filter above. This value becomes the path attribute set in the # cookie so the cookies from each instance will not clobber each other. cookie_path = /galaxy On Thu, Feb 21, 2013 at 7:07 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Thursday, February 21, 2013, wrote: Further, when I try “register” I add an email address and password, but the system still doesn’t log me in ** ** Maybe this is a database configuration issue? ** ** I didn’t think I’d need to state any specific entries in universe as I am using physically separate directories ** ** i.e. /home/galaxy/galaxy-cte/ i.e. /home/galaxy/galaxy-Newcastle/ ** ** Cheers Neil The logging in problem sounds like a cookie path issue to me, I had something similar when I added second public Galaxy to our server (in my case, a live on and a dev one). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] FastQC Question
Hello, Min
This tool has been stable for a while now and that variable cl is defined
as a list in line 52 just above line 63, so this error seems very strange
unless you made changes to the source code?
It's hard to know where to start helping because there are lots of useful
pieces of information you have not yet provided - at the very least, please
always provide the version of Galaxy source (the output of hg heads will
help) and python you are running.
Just to check that the source is as it should be, can you please try:
hg diff tools/rgenetics/rgFastQC.py
and also
hg diff tools/rgenetics/rgFastQC.xml
from your Galaxy root - if you get any output, try
hg revert tools/rgenetics/rgFastQC.py
hg revert tools/rgenetics/rgFastQC.xml
and then try running it again?
On Thu, Feb 21, 2013 at 6:46 AM, Min Cho m...@ba-lab.com wrote:
Hello,
When trying to run FastQC, I get this error:
Traceback (most recent call last):
File /home/bal/galaxy-dist/tools/rgenetics/rgFastQC.py, line 162, in
module
html,retval,serr = f.run_fastqc()
File /home/bal/galaxy-dist/tools/rgenetics/rgFastQC.py, line 63, in
run_fastqc
sout.write('# FastQC cl = \n' % ' '.join(cl))
TypeError: not all arguments converted during string formatting
Is there anything I can do to modify the code to fix this? I'm using data
run through FASTQ Groomer.
Thank you
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
[galaxy-dev] Fwd: Custom Cheetah filters?
Hi Russell, There may be a better way, but this works for me in the Toolfactory to create space and special character free names? param name=foo type=text value= label=Foo sanitizer invalid_char= valid initial=string.letters,string.digits/ /sanitizer /param On Wed, Feb 13, 2013 at 11:22 AM, Smithies, Russell russell.smith...@agresearch.co.nz wrote: I want to add a filter to strip whitespace and newlines from a text input box so I can pipe the sanitized string to a command. Documentation is a bit sparse (and my Python a bit basic) so does anyone have an example? Perhaps there’s a better way of doing it – regex maybe? ** ** Any ideas? ** ** Thanx, ** ** --Russell -- ** ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] tool integration SOAPdenovo
I don't think there's one in a toolshed but someone's clearly done some work on it at http://galaxy.cbiit.cuhk.edu.hk/ Perhaps you may be able to convince them of the many benefits of contributing back to the community by sharing some of their tool code? On Wed, Feb 13, 2013 at 12:32 PM, Jorge Andrade andrade.jo...@gmail.comwrote: Dear all, Is there a repository where I can finnd SOAPdenovo tool wrappers for Galaxy? I would like to install this tool in our local Galaxy server. Thanks, Jorge ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] What image formats can be displayed in the Galaxy view panel?
Luobin - one additional minor observation: In reality, Galaxy does not do the displaying - it just sends stuff to the users' web browser for display. So even when Galaxy knows what mimetype to attach to a specific image file, the users' web browser response to that mimetype will always remain the final frontier. Not everyone uses IE where dark magic just happens :) Out of the box, many linux distributions do not enable (potentially exploitable) browser plugin viewers for PDF or SVG for example - so correctly displaying some mimetypes will always be beyond the control of the Galaxy server. EG: in 12.04 Ubuntu on my desktop using chrome or firefox, automagic pdf viewing on a 64 bit system took a fair bit of fiddling to get working - until then, even when Galaxy supplies the required mimetype, the user has to download the PDF and open it by hand. On Fri, Feb 8, 2013 at 10:16 AM, Peter Cock p.j.a.c...@googlemail.comwrote: On Thu, Feb 7, 2013 at 9:41 PM, Luobin Yang yangl...@isu.edu wrote: Hi, I tried the pcx and ps formats, but the browser just downloads these kinds of files instead rendering them in the Galaxy window... It seems png and pdf files can be rendered in the Galaxy windows. How can I make Galaxy display other image formats like ps and pcx? Thanks, Luobin I would guess this is possible but only if those other image types are first defined in Galaxy as new datatypes (with sensible MIME type values). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Multiple Instances ...
Neil, If by 'multiple' you mean 'independent' galaxy instances, they must each talk to independent backend databases, so if you're thinking of running eg 2 or more independent instances at CSIRO, each for specific tool sets and sending each of your users to one or other of them based on some smart Apache code and their login, beware that users won't be able to share or see any histories or datasets from one instance on the other. That might work well - or not - but separate instances cannot safely share the same backend database tables - they're just separate Galaxy instances - like test and main are, and there's no specific documentation needed. If you are asking about multiple processes (web servers etc) to scale up a slow heavily loaded instance, that's documented in the wiki - eg a quick search finds http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance On Fri, Feb 8, 2013 at 1:40 PM, neil.burd...@csiro.au wrote: Hi, Can someone point me to the documentation to set up /configure multiple instances of Galaxy running on the same node please? ** ** I think this is the best method of hiding tools based upon users email logon... ** ** Thanks Neil ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Fwd: Multiple Instances ...
ok - sorry I misunderstood. Yes, assuming you already have one Galaxy instance configured right, cloning and editing the rewrite and authentication sections for the other paste process should work and that looks reasonable to me FWIW - OTOH, apache configuration is definitely one of the darker arts so the only way to tell is to run them and keep an eye on the two separate paste process logs as you hit each site separately. Of course, make sure the indepndent instance universe_wsgi.xml files specify independent postgresql databases and keep the two independent galaxy file store directories separated or very bad things will happen. On Fri, Feb 8, 2013 at 2:20 PM, neil.burd...@csiro.au wrote: Thanks Ross. ** ** I did mean separate Galaxy instances like test and main with their own independent backend databases. ** ** How could I run say a test and a main from the same node? ** ** I guess I’d need to modify the port number for each instance and then multiple entries in the apache config file i.e. So they show physically separate information when loaded... i.e. http://localhost/galaxyTest and . http://localhost/galaxyMain ** ** *RewriteEngine* *on* *RewriteRule* ^/galaxyTest$ */galaxyTest/* [R] *RewriteRule* ^/galaxyTest/static/style/(.*) * /home/nate/galaxy-dist-test/static/june_2007_style/blue/*$1 [L] *RewriteRule* ^/galaxyTest/static/scripts/(.*) * /home/nate/galaxy-dist-test/static/scripts/packed/*$1 [L] *RewriteRule* ^/galaxyTest/static/(.*) * /home/nate/galaxy-dist-test/static/*$1 [L] *RewriteRule* ^/galaxyTest/favicon.ico * /home/nate/galaxy-dist-test/static/favicon.ico* [L] *RewriteRule* ^/galaxyTest/robots.txt * /home/nate/galaxy-dist-test/static/robots.txt* [L] *RewriteRule* ^/galaxyTest (.*) http://localhost:9080$1 [P] ** ** *RewriteRule* ^/galaxyMain$ */galaxyMain/* [R] *RewriteRule* ^/galaxyMain/static/style/(.*) */home/nate/galaxy-dist-main/static/june_2007_style/blue/*$1 [L] *RewriteRule* ^/galaxyMain/static/scripts/(.*) */home/nate/galaxy-dist-main/static/scripts/packed/*$1 [L] *RewriteRule* ^/galaxyMain/static/(.*) */home/nate/galaxy-dist-main/static/*$1 [L] *RewriteRule* ^/galaxyMain/favicon.ico */home/nate/galaxy-dist-main/static/favicon.ico* [L] *RewriteRule* ^/galaxyMain/robots.txt */home/nate/galaxy-dist-main/static/robots.txt* [L] *RewriteRule* ^/galaxyMain (.*) http://localhost:9081$1 [P] ** ** Is that correct? ** ** Cheers Neil ** ** ** ** *From:* Ross [mailto:ross.laza...@gmail.com] *Sent:* Friday, 8 February 2013 12:54 PM *To:* Burdett, Neil (ICT Centre, Herston - RBWH) *Cc:* galaxy-dev@lists.bx.psu.edu *Subject:* Re: [galaxy-dev] Multiple Instances ... ** ** Neil, ** ** If by 'multiple' you mean 'independent' galaxy instances, they must each talk to independent backend databases, so if you're thinking of running eg 2 or more independent instances at CSIRO, each for specific tool sets and sending each of your users to one or other of them based on some smart Apache code and their login, beware that users won't be able to share or see any histories or datasets from one instance on the other. ** ** That might work well - or not - but separate instances cannot safely share the same backend database tables - they're just separate Galaxy instances - like test and main are, and there's no specific documentation needed. ** ** If you are asking about multiple processes (web servers etc) to scale up a slow heavily loaded instance, that's documented in the wiki - eg a quick search finds http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance ** ** On Fri, Feb 8, 2013 at 1:40 PM, neil.burd...@csiro.au wrote: Hi, Can someone point me to the documentation to set up /configure multiple instances of Galaxy running on the same node please? I think this is the best method of hiding tools based upon users email logon... Thanks Neil ** -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Best way to work with one directory and many files as 1 input
I'd suggest: 1) Make your new datatype a subclass of Html - it's a subclass of composite that contains an HTML document as the object's native display - so it can inform users what's there. 2) When constructing these new things, pass the file_path of the Html (composite) dataset subclass to your wrapper on the command line 3) Your wrapper code can construct any arbitrary structure as long as it's rooted in that directory - Galaxy stores it without any fuss. The wrapper should also populate the Html file itself with nicely laid annotation for the user to check out. 4) The key is that all tools that take this new datatype as input must know how to decode this structure - they must be passed the $input.extra_files_path which gives them that same path root. 5) Yes, it's odd and annoying that it's extra_files_path for files_path. Go figure. 6) grep extra_files tools/*.xml to find some examples - I think the velvetg one uses a complex subdirectory structure - but it doesn't really matter - as long as your tools know how to deal with it, it's just a directory to Galaxy! I hope all this helps... On Wed, Jan 30, 2013 at 8:22 PM, Pierre Pericard pierre.peric...@sb-roscoff.fr wrote: In that case, could anyone point me to an example of a Composite Datatype which could accept as input an unknown number of files in an unknown number of directories. I can't seem to understand how that would work based on the wiki. But maybe are we anticipating a near functionality of Galaxy. There were talks about changing the way Galaxy handle zip files, is it still on the table ? Thank in advance for any help, Pierre Pierre Pericard IE CDD - Projet Peptisan Service Informatique et Bio-informatique (SIB) Station Biologique de Roscoff CNRS-UPMC Place Georges Teissier CS 90074 29688 Roscoff CEDEX FRANCE http://abims.sb-roscoff.fr/ Le 29/01/2013 18:04, Peter Cock a écrit : On Tue, Jan 29, 2013 at 4:58 PM, Pierre Pericard pierre.peric...@sb-roscoff.fr** wrote: If I'm not mistaking, Composite Datatypes allow for only one directory, whereas we need to keep a constant directory structure with 2 or more sub-directories containing our input files. I'm not sure if that is true - the example of HTML output with images comes to mind as a common use-case where subfolder(s) would be expected. I've only had limited first hand experience with Galaxy's composite datatypes myself though. We have no way to change these tools behavior (obviously not Galaxy-friendly ;-) ) and therefore need to maintain this structure in the job working directory. Perhaps a tool wrapper could create a dummy folder using symlinks (faster and less wasted disk than copying files), but that isn't ideal. Peter __**_ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] filtering a param of type 'data' so only one type is available
Hi, Dan. Using format=rda will restrict the list to history items of the 'rda' datatype - but ONLY if the rda datatype has been defined! The easiest way to do that is to define a new datatype in datatypes_conf.xml and since rData is binary, you should subclass binary. Something like (untested - especially the mimetype !) datatype extension=rda type=galaxy.datatypes.binary:Binary mimetype= application/x-rlang-transport subclass=True / might work. A server restart is always required to load any new datatypes. Good luck! On Thu, Jan 24, 2013 at 10:41 AM, Dan Tenenbaum dtene...@fhcrc.org wrote: Hi, I have a tool wrapper with a param of type=data. Currently, this renders as a text box with a drop down list that has 52 items in it (the number of things I have in my history, I guess). I'd like to filter this list so that only some items in the history (in my case any item whose name ends with '.rda') are shown in this dropdown. It seems like maybe the 'format' parameter to the 'param' tag is what I want, but: 1) I tried format=rda and that didn't seem to change anything. 2) rda is not in the datatypes.conf file which the documentation suggests is required? (.rda is an extension used for serialized R objects.) The tool I'm working on only accepts rda files as input and the reason I'm asking for this is that it is all too easy to accidentally feed it a file of some other type. If the dropdown could be filtered so that only items which will work with the tool are shown, that would be great. Is there a way to do that? Thanks, Dan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] filtering a param of type 'data' so only one type is available
Hi, Dan Glad it worked and thanks for pointing out a more general mimetype - does any one else (other than me) want to vote to add datatype extension=rda type=galaxy.datatypes.binary:Binary mimetype=application/x-gzip subclass=True / to datatypes_conf.xml.sample permanently ? On Thu, Jan 24, 2013 at 10:58 AM, Dan Tenenbaum dtene...@fhcrc.org wrote: On Wed, Jan 23, 2013 at 3:51 PM, Ross ross.laza...@gmail.com wrote: Hi, Dan. Using format=rda will restrict the list to history items of the 'rda' datatype - but ONLY if the rda datatype has been defined! The easiest way to do that is to define a new datatype in datatypes_conf.xml and since rData is binary, you should subclass binary. Something like (untested - especially the mimetype !) datatype extension=rda type=galaxy.datatypes.binary:Binary mimetype=application/x-rlang-transport subclass=True / might work. A server restart is always required to load any new datatypes. Good luck! It works! I changed the mime type to application/x-gzip, since serialized R objects are gzipped. Thanks, Dan On Thu, Jan 24, 2013 at 10:41 AM, Dan Tenenbaum dtene...@fhcrc.org wrote: Hi, I have a tool wrapper with a param of type=data. Currently, this renders as a text box with a drop down list that has 52 items in it (the number of things I have in my history, I guess). I'd like to filter this list so that only some items in the history (in my case any item whose name ends with '.rda') are shown in this dropdown. It seems like maybe the 'format' parameter to the 'param' tag is what I want, but: 1) I tried format=rda and that didn't seem to change anything. 2) rda is not in the datatypes.conf file which the documentation suggests is required? (.rda is an extension used for serialized R objects.) The tool I'm working on only accepts rda files as input and the reason I'm asking for this is that it is all too easy to accidentally feed it a file of some other type. If the dropdown could be filtered so that only items which will work with the tool are shown, that would be great. Is there a way to do that? Thanks, Dan ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool error occurred when adding my tool to galaxy
I'm sure you checked carefully before posting, but your traceback ends with KeyError: 'thype' suggesting maybe a simple syntax/spelling problem to me. On Sat, Dec 29, 2012 at 7:36 PM, Tan Jun tanjun2...@gmail.com wrote: ** Hi, all I'm new here. These days I'm trying to add a tool in Galaxy, as showed in screencast. I did the same thing with that in screencast. But an error occurred. The error says: tool error An error occurred running this job:failure preparing job The message in the terminal says: galaxy.jobs.handler DEBUG 2012-12-29 15:54:25,504 dispatching job 7 to local runner galaxy.jobs.handler INFO 2012-12-29 15:54:25,733 (7) Job dispatched galaxy.jobs.runners.local DEBUG 2012-12-29 15:54:25,818 Local runner: starting job 7 galaxy.jobs.runners.local ERROR 2012-12-29 15:54:26,061 failure running job 7 Traceback (most recent call last): File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/jobs/runners/local.py, line 72, in run_job job_wrapper.prepare() File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/jobs/__init__.py, line 180, in prepare config_filenames = self.tool.build_config_files( param_dict, self.working_directory ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/tools/__init__.py, line 2430, in build_config_files f.write( fill_template( template_text, context=param_dict ) ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/util/template.py, line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py, line 1004, in __str__ return getattr(self, mainMethName)() File cheetah_DynamicallyCompiledCheetahTemplate_1356767666_06_36079.py, line 149, in respond KeyError: 'thype' 127.0.0.1 - - [29/Dec/2012:15:54:27 +0800] GET /api/histories/f597429621d6eb2b HTTP/1.1 200 - http://localhost:8080/history; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.10 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 127.0.0.1 - - [29/Dec/2012:15:54:27 +0800] GET /api/histories/f597429621d6eb2b/contents?ids=ebfb8f50c6abde6d HTTP/1.1 200 - http://localhost:8080/history; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.10 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 Could anybody help me? Thanks! ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Integrating generated graphics from cli tool?
Alex, xvfb http://www.xfree86.org/4.0.1/Xvfb.1.html works for our headless node R graphics and possibly other packages. I seem to remember writing a wiki entry but a search didn't find it... I see someone on the dev list found the toolfactory code which does a lot of sensible things automatically to create a pretty useful html index for a folder full of pdfs/pngs/xls/ On Mon, Dec 17, 2012 at 11:14 PM, Bossers, Alex alex.boss...@wur.nl wrote: Samual, We do the same as Peter mentioned. We generate pdfs or you could generate a mixed datatype html file linking to seperate documents/images (like the fastQC tool). If you get the export to png working on a headless server without desktop/gui..please let me know. In our hands it only works on full desktop linux environments somehow. Alex -Oorspronkelijk bericht- Van: galaxy-dev-boun...@lists.bx.psu.edu [mailto: galaxy-dev-boun...@lists.bx.psu.edu] Namens Samuel Lampa Verzonden: zondag 16 december 2012 23:14 Aan: Peter Cock CC: galaxy-dev@lists.bx.psu.edu Onderwerp: Re: [galaxy-dev] Integrating generated graphics from cli tool? On 12/14/2012 07:40 PM, Peter Cock wrote: On Fri, Dec 14, 2012 at 6:18 PM, Samuel Lampa samuel.la...@gmail.com wrote: There are existing tools which produce a PDF file as a Galaxy output, shown as its own 'green box' in the history. This works quite nicely if the browser shows the PDF in-line, but one one of my machines (not sure which browser off hand) it insists on downloading the PDF and opening it instead - which isn't quite as smooth an interface. Thanks for the info! Seems like that could be one way of doing it. I would probably output png rather than pdf, and that might work better. Best // Samuel ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool runners don't listen to settings in [galaxy:tool_runners]
Hi, Thon I'm guessing here: but now that bwa is kept in the toolshed, the tool name appears to be toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/bwa_wrapper/1.2.3 according to your job runner entry - so I wonder if that whole ugly string now needs to be in the tool_runners section to over-ride the default rather thant bwa_wrapper ? This might be an unforseen consequence of the way the toolshed adapts tool names to keep versions under control. Fixing this might be a little tricky since one goal is to keep multiple tool versions separate - but we will all want all versions of bwa_wrapper to use the same runner flags? On Thu, Dec 6, 2012 at 8:40 AM, Anthonius deBoer thondeb...@me.com wrote: Hi, In my fresh copy of galaxy-dist I am having a problem with tools being run only with the default tool runner and not with the assigned tool runner I created in the [galaxy:tool_runners] section... This is the setting for bwa_wrapper in my universe_wsgi.ini: bwa_wrapper = drmaa://-V -q long.q -pe smp 4/ and this is what the job admin page shows: It's in a different queue and does not use the smp setting I ask for...(It's the default setting) any ideas? 7X26 minutes ago toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/bwa_wrapper/1.2.3runningpython /mnt/ngs/analysis/svcgalaxy/shed_tools/ toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/ffa8aaa14f7c/bwa_wrappers/bwa_wrapper.py--threads=4 --fileSource=indexed --ref=/mnt/ngs/analysis/svcgalaxy/DATA/galaxy-indexes/bwa-indexes/human_g1k_v37_decoy.fasta --do_not_build_index --input1=/home/tdeboer/P/MUT190-MAIN/fastq-2_205/2_205_1_AAACAT_R1.fastq --input2=/home/tdeboer/P/MUT190-MAIN/fastq-2_205/2_205_1_AAACAT_R2.fastq --output=/mnt/ngs/analysis/svcgalaxy/DATA/files/000/dataset_776.dat --genAlignType=paired --params=pre_set --suppressHeader=falsedrmaa://-V -q all.q -pe smp 1/4440144 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Display of HTML type broken in new Galaxy ?
It's part of Galaxy security against XSS attack by sanitizing html. If you trust your users you don't need it. If you run a public server, it's your call. On Thu, Nov 15, 2012 at 7:19 PM, Lukasse, Pieter pieter.luka...@wur.nlwrote: Can anyone give some more information about this? Thanks, Pieter. -Original Message- From: Joachim Jacob [mailto:joachim.ja...@vib.be] Sent: donderdag 15 november 2012 9:17 To: Lukasse, Pieter Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Display of HTML type broken in new Galaxy ? I have no clue at all, sorry. Cheers, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be@bitsatvib On 11/15/2012 09:14 AM, Lukasse, Pieter wrote: Hi Joachim, By the way: do you know what the reason is for this setting? Is there a known security problem that triggered this feature? If you add only trusted tools to your Galaxy environment, then this is not needed, right? This change was mentioned briefly in March 12, 2012 Galaxy Development News Brief but no background information was given Thanks and regards, Pieter. -Original Message- From: galaxy-dev-boun...@lists.bx.psu.edu [mailto:galaxy-dev-boun...@lists.bx.psu.edu] On Behalf Of Lukasse, Pieter Sent: donderdag 18 oktober 2012 9:19 To: 'joachim.ja...@vib.be' Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Display of HTML type broken in new Galaxy ? Thanks Joachim, that solved the problem! -Original Message- From: Joachim Jacob [mailto:joachim.ja...@vib.be] Sent: woensdag 17 oktober 2012 10:47 To: Lukasse, Pieter Cc: galaxy-dev@lists.bx.psu.edu Subject: Re: [galaxy-dev] Display of HTML type broken in new Galaxy ? Hi, Make sure that in your universe.wsgi is set: sanitize_all_html = False Hope this helps, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 10/17/2012 10:09 AM, Lukasse, Pieter wrote: Hi, I noticed that in the new Galaxy version the preview of HTML files seems to be broken. When I try to preview a HTML file generated by a tool, it displays a HTML that has all style stripped off. So the HTML display is in fact changing the generated HTML before displaying it. Has anyone noticed this as well? What are the fixes/workarounds? Regards, Pieter Lukasse Wageningen UR, Plant Research International Departments of Bioscience and Bioinformatics Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB, Wageningen, the Netherlands +31-317480891; skype: pieter.lukasse.wur http://www.pri.wur.nl http://www.pri.wur.nl/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Multiple .CEL file import for microarray analysis
Hi, Jon. You're right, there is no cel file management and that's one of many warts needing fixing - it's not a lot of work to make a new tool to create an eSet from a tarball of cel files as you suggest. Unfortunately the rexpression toolkit will soon be deprecated and disappear from the default distribution. It's no longer under active development by me - the group I now work with don't do microarrays much and I don't have the resources to maintain tools we're not using. I would be very happy to see the code taken over if anyone has the resources but for now and will help any way I can. (FYI: We've had some success using chipster for routine microarray analyses from http://chipster.csc.fi/ although it's buggy and doesn't seem to be very actively maintained) On Mon, Jul 23, 2012 at 12:04 PM, Jon Manning jonathan.mann...@ed.ac.uk wrote: Hi all, My apologies if this seems obvious to many of you (seems like something that will have been dealt with)- but I've been rooting around in the docs and mailing list to no concrete avail. I'm looking to Galaxyise our existing (somewhat informal) microarray analysis workflows. I've found rexpression etc and the associated helpful datatypes. But I'm stuck at the import of .CEL files: Do I need to create a new datatype to represent .CEL? The fact this doesn't exist by default suggests it's not the right approach. Assuming I was to do the above, how would I create a function to generate an affyBatch (for example) from an unspecified number of input .CEL? Is the solution to have R work with a .tar and a config file to produce the affyBatch, thereby bypassing the .CEL requirement? But then there's no .tar datatype either. If I've missed the docs I'm happy to be chided and pushed in the right direction. Many thanks, Jon -- Dr Jonathan Manning Bioinformatics Team Centre for Cardiovascular Science University of Edinburgh Queens Medical Research Institute 47 Little France Crescent Edinburgh EH16 4TJ United Kingdom T: +44 131 242 6700 F: +44 131 242 6782 E: jmann...@staffmail.ed.ac.uk The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Configuring FastQC to run
Hi, Peter. A couple of things worth checking: 1) Is the file (it's a perl script) executable by the user running the Galaxy process? Unfortunately the FastQC distro is built in windows apparently so that script is not executable out of the box - fixing that may be all you need to do. 2) Does fastqc work correctly when executed by the user running the Galaxy process - eg is the perl binary path right and executable? It should open interactively if called directly ? 3) Does it work correctly when executed on all the nodes used to process Galaxy tasks (eg SGE/PBS etc) ? If all those 3 are true, it should work. On Fri, Jul 13, 2012 at 12:05 PM, Peter Schmitt peter.r.schm...@dartmouth.edu wrote: I get the following error when attempting to run FastQC from galaxy: job command line: python /opt/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py -i /opt/galaxy/galaxy-dist/database/files/000/dataset_75.dat -d /opt/galaxy/galaxy-dist/database/job_working_directory/000/114/dataset_128_files -o /opt/galaxy/galaxy-dist/database/files/000/dataset_128.dat -n FastQC -f fastqsanger -j SRS280832_1_FASTQ Groomer on data 14 -e /opt/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc - job stderr: Traceback (most recent call last): File /opt/galaxy/galaxy-dist/tools/rgenetics/rgFastQC.py, line 149, in module assert os.path.isfile(opts.executable),'##rgFastQC.py error - cannot find executable %s' % opts.executable AssertionError: ##rgFastQC.py error - cannot find executable /opt/galaxy/galaxy-dist/tool-data/shared/jars/FastQC/fastqc The fastqc binary is in galaxy's PATH. Any help would be appreciated. -- ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] FastQC Tool Errors
Do you run an X11 virtual frame buffer - eg Xvfb? Otherwise AFAIK R graphics and Java will complain on headless nodes. On Sat, Jun 23, 2012 at 6:30 AM, Josh Nielsen jniel...@hudsonalpha.com wrote: Hello, I am having an issue with getting the FastQC tool to work with Galaxy on our server. I downloaded the FastQC files (version 0.8.0) and changed the directory that the wrapper script looks for the 'fastqc' executable in, but when we run a job with it we have been getting the following output: Started analysis of Clip Approx 5% complete for Clip Approx 10% complete for Clip ... ... Approx 95% complete for Clip Approx 100% complete for Clip Analysis complete for Clip (.:9754): Gtk-WARNING **: cannot open display: And then the job shows as failed in Galaxy. The output .dat file just has that same output/error message in it (though it seems to indicate it got to 100%). Also when I try to execute the fastqc file directly (albeit with no arguments) I get this: Exception in thread main java.awt.HeadlessException: No X11 DISPLAY variable was set, but this program performed an operation which requires it. at java.awt.GraphicsEnvironment.checkHeadless(GraphicsEnvironment.java:173) at java.awt.Window.init(Window.java:437) at java.awt.Frame.init(Frame.java:419) at java.awt.Frame.init(Frame.java:384) at javax.swing.JFrame.init(JFrame.java:174) at uk.ac.bbsrc.babraham.FastQC.FastQCApplication.init(FastQCApplication.java:271) at uk.ac.bbsrc.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:102) Both errors seem to have something to do with the graphical GUI component of FastQC (which I have seen some screenshots for on the FastQC webpage). If this application is GUI-driven how did the online PSU Galaxy get it to work with their wrapper script when the tools are run in a command-line environment with no X11 or Gtk? Essentially I'm just wondering what steps I'm missing here to getting this to work with our Galaxy mirror, other than just dropping the executable in place? Any suggestions? Thanks, Josh ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Re : Composite output with self-declarated datatypes
Hi Marine, Other people may have better ideas, but the way I've always done it is to ensure that the tool knows how to find the input files inside the extra_files_path because that's easy to pass. If $i is the name of data parameter = composite file chosen from the user history (ie a data input on your form), then you can pass it to the script as (eg) --extra_files_path $i.extra_files_path You might also find --base_name $i.metadata.base_name handy sometimes for naming outputs I hope this helps. On Fri, Jun 15, 2012 at 7:29 PM, Marine Rohmer marine.roh...@yahoo.frwrote: Update : Now my tool creates a html composite output, made of 2 outputs .xxx and .yyy. I've added the def get_mime(self) function in the python file describing all my formats, and now it works. When I run my tool and click on the eye symbol, I can see a html page with links to download the two component files. I thought everything was now going perfect, but when I try to use this composite output as an input in another tool I've added, this other tool can't open it Which makes sense to me, since the other tool needs both of the component files, and not an html input. So I wondered how to change the html composite output into the two component files ? I've tried to retrieve them in a bash wrapper with : component= HTML_FILE=$1 for i in HTML_FILE do component=$i done But with a simple echo $component test, I get HTML_FILE as a result, and not one of the component file. So is there any specified thing to do to recover the component files of a composite file ? Best regards, Marine -- *De :* Ross ross.laza...@gmail.com *À :* Marine Rohmer marine.roh...@yahoo.fr *Envoyé le :* Mercredi 13 juin 2012 10h50 *Objet :* Re: [galaxy-dev] Re : Composite output with self-declarated datatypes Look at your xml. Output_name is a text parameter - it doesn't have any paths It's certainly not a new output file Galaxy will create or an existing composite object - so it quite correctly complains about not having a files_path or extra_files_path On Wed, Jun 13, 2012 at 6:34 PM, Marine Rohmer marine.roh...@yahoo.frwrote: Hi Ross, Thank you so much for your answer ! I've changed my command line in myTool.xml as followed : command path/to/myTool-wrapper.sh '$output_name.files_path/$output_name.metadata.base_name' $input_file /command But I still have the same error message, with files_path instead of extra_files_path : NotFound: cannot find 'files_path' while searching for 'output_name.files_path' Well I'm going to grep those files as you said, and see if it can help me... Thank you again, Marine -- *De :* Ross ross.laza...@gmail.com *À :* Marine Rohmer marine.roh...@yahoo.fr *Envoyé le :* Mercredi 13 juin 2012 6h44 *Objet :* Re: [galaxy-dev] Re : Composite output with self-declarated datatypes Marine, Sorry to hear you're having problems - composite objects definitely do work but they are definitely not simple or properly documented. I don't really have time to figure out exactly what the problem is but one very obvious error message NotFound: cannot find 'extra_files_path' while searching for 'output_name.extra_files_path' is telling you that a new output file does not have an extra_files_path at job submission. Try output_name.files_path - why it differs is something I do not understand but I've learned to live with If you grep for files_path in your tool/*/*.xml files, you'll find lots of examples of tools using files_path and extra_files_path (mostly html files) and studying those working examples might be useful in getting your code to work? cheers... On Tue, Jun 12, 2012 at 6:24 PM, Marine Rohmer marine.roh...@yahoo.fr wrote: Hi, Maybe my message was not understandable enough. I really need your help, so I'll try to be more concise : How do I make a composite output from 2 datatypes that I have declared myself ? I've followed the Composite Datatypes wiki but it seems that I've missed something... My composite datatype appears well in file format from Get Data's upload file section, but when I run my tool, I have 2 outputs which are the components of my primary datatype, instead of only one output. Best regards, Marine De : Marine Rohmer marine.roh...@yahoo.fr À : galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu Envoyé le : Vendredi 8 juin 2012 15h15 Objet : Composite output with self-declarated datatype Hi everyone, I'm trying to add a tool which generates 2 files, that I will call .xxx (a text file) and .yyy (a binary file) . Both files are needed to use the result of my tool with an other tool I've added. So I wanted to create a composite datatype , that I will call .composite, whose components are .xxx and .yyy. I've declared the datatype .xxx
Re: [galaxy-dev] Re : Composite output with self-declarated datatypes
Marine,
It seems that the files your tool needs are in the extra_files_path and you
just have to figure out how to get at them for processing.
From my experience, it is trivial to make this work with a wrapper script
in Python.
Given a directory listing of the passed in extra_files_path, you can easily
figure out which input is which.
It's probably possible with bash but it would definitely not be my personal
preference.
A general purpose scripting language like python would probably make it
easier but if you insist on bash, I am sure you already have enough
information to get it to work.
On Sat, Jun 16, 2012 at 2:08 AM, Marine Rohmer marine.roh...@yahoo.frwrote:
New try :
${os.path.join( $input.extra_files_path, 'first_component_file.xxx' )}
${os.path.join( $input.extra_files_path, 'second_component_file.yyy')}
With this, I can see my tool takes as parameters :
myToolpath/to/first_component_file.xxx
path/to/second_component_file.yyy
This sounds great, isn't it ?
But I have now a message quoting the ''os.path.join'' line, and saying
mauvaise substitution which means bad substitution.
(First why isn't it in English ? Does it mean it's not a Galaxy problem ?)
So I generated a new composite file, made of 2 components files, named
exactly as the metadata.base_name is set.
I used this two component files as an input in my other tool, and now I no
longer have this message !
That means Galaxy wants the components files' name to be the same as the
metadata.base_name , if I understood well... So if user want to change
the name, it sure will fail...
How to fix this problem ?
Furthermore, it doesn't completely work. I think my tool thinks there is
two parameters instead of only one, when seeing the two paths for my
components files. I don't know how to fix it either...
Any ideas ?
Thank you for your attention,
Marine
--
*De :* Ross ross.laza...@gmail.com
*À :* Marine Rohmer marine.roh...@yahoo.fr
*Cc :* galaxy-dev@lists.bx.psu.edu galaxy-dev@lists.bx.psu.edu
*Envoyé le :* Vendredi 15 juin 2012 12h40
*Objet :* Re: Re : Composite output with self-declarated datatypes
Hi Marine,
Other people may have better ideas, but the way I've always done it is to
ensure that the tool knows how to find the input files inside the
extra_files_path because that's easy to pass.
If $i is the name of data parameter = composite file chosen from the user
history (ie a data input on your form), then you can pass it to the script
as (eg)
--extra_files_path $i.extra_files_path
You might also find
--base_name $i.metadata.base_name
handy sometimes for naming outputs
I hope this helps.
On Fri, Jun 15, 2012 at 7:29 PM, Marine Rohmer marine.roh...@yahoo.frwrote:
Update : Now my tool creates a html composite output, made of 2 outputs
.xxx and .yyy.
I've added the def get_mime(self) function in the python file describing
all my formats, and now it works.
When I run my tool and click on the eye symbol, I can see a html page with
links to download the two component files.
I thought everything was now going perfect, but when I try to use this
composite output as an input in another tool I've added, this other tool
can't open it Which makes sense to me, since the other tool needs both
of the component files, and not an html input.
So I wondered how to change the html composite output into the two
component files ?
I've tried to retrieve them in a bash wrapper with :
component=
HTML_FILE=$1
for i in HTML_FILE
do component=$i
done
But with a simple echo $component test, I get HTML_FILE as a result,
and not one of the component file.
So is there any specified thing to do to recover the component files of a
composite file ?
Best regards,
Marine
--
*De :* Ross ross.laza...@gmail.com
*À :* Marine Rohmer marine.roh...@yahoo.fr
*Envoyé le :* Mercredi 13 juin 2012 10h50
*Objet :* Re: [galaxy-dev] Re : Composite output with self-declarated
datatypes
Look at your xml.
Output_name is a text parameter - it doesn't have any paths It's certainly
not a new output file Galaxy will create or an existing composite object -
so it quite correctly complains about not having a files_path or
extra_files_path
On Wed, Jun 13, 2012 at 6:34 PM, Marine Rohmer marine.roh...@yahoo.frwrote:
Hi Ross,
Thank you so much for your answer !
I've changed my command line in myTool.xml as followed :
command path/to/myTool-wrapper.sh
'$output_name.files_path/$output_name.metadata.base_name'
$input_file
/command
But I still have the same error message, with files_path instead of
extra_files_path :
NotFound: cannot find 'files_path' while searching for
'output_name.files_path'
Well I'm going to grep those files as you said, and see if it can help
me...
Thank you again,
Marine
--
*De :* Ross ross.laza...@gmail.com
*À :* Marine Rohmer
Re: [galaxy-dev] Tool Shed Workflow
John, Why not separate toolshed updates from dist updates - tool xml and other code should be robust wrt dist version. One thing at a time - tools get updated less often than dist I'd wager, and you can subscribe to repository update emails. After a dist update you want all the tool functional tests green as evidence that at least the test cases are running! As always, YMMV On Sat, Jun 9, 2012 at 2:38 PM, John Chilton chil...@msi.umn.edu wrote: On Fri, Jun 8, 2012 at 3:27 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi John, On Jun 8, 2012, at 1:22 PM, John Chilton wrote: Hello Greg, Thanks for the prompt and detailed response (though it did make me sad). I think deploying tested, static components and configurations to production environments and having production environments not depending on outside services (like the tool shed) should be considered best practices. I'm not sure I understand this issue. What processes are you using to upgrade your test and production servers with new Galaxy distributions? If you are pulling new Galaxy distributions from our Galaxy dist repository in bitbucket, then pulling tools from the Galaxy tool shed is not much different - both are outside services. Updating your test environment, determining it is functionally correct, and then updating your production environment using the same approach would generally follow a best practice approach. This is the approach we are currently using for our public test and main Galaxy instances at Penn State. We don't pull down from bitbucket directly to our production environment, we pull galaxy-dist changes into our testing repository, merge (that can be quite complicated, sometimes a multihour process), auto-deploy to a testing server, and then finally we push the tested changes into a bare production repo. Our sys admins then pull in changes from that bare production repo in our production environment. We also prebuild eggs in our testing environment not live on our production system. Given the complicated merges we need to do and the configuration files that need to be updated each dist update it would seem making those changes on a live production system would be problematic. Even if one was pulling changes directly from bitbucket into a production codebase, I think the dependency on bitbucket would be very different than on N toolsheds. If our sys admin is going to update Galaxy and bitbucket is down, that is no problem he or she can just bring Galaxy back up and update later. Now lets imagine they shutdown our galaxy instance, updated the code base, did a database migration, and went to do a toolshed migration and that failed. In this case instead of just bringing Galaxy back up they will now need to restore the database from backup and pullout of the mercurial changes. Anyway all of that is a digression right, I understand that we will need to have the deploy-time dependencies on tool sheds and make these tool migration script calls part of our workflow. My lingering hope is for a way of programmatically importing and updating new tools that were never part of Galaxy (Qiime, upload_local_file, etc...) using tool sheds. My previous e-mail was proposing or positing a mechanism for doing that, but I think you read it like I was trying to describe a way to script the migrations of the existing official Galaxy tools (I definitely get that you have done that). Thanks again for your time and detailed responses, -John ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] I can't see or access pictures in FastQC
Hi, Daniel, The configuration suggested at http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy is what you followed to set your proxy up? That obfuscated link hash element is generated from internal database ids to help secure against malicious users viewing specific datasets - not something your web server configuration normally needs to know about or do anything about. The Galaxy paste process takes care of generating and interpreting them. If you want to dig deeper, the external scripting api has code for accessing datasets by generating the right hashes - see code and minimal docs at scripts/api On Fri, Jun 8, 2012 at 7:59 AM, Dorset, Daniel C daniel.dor...@vanderbilt.edu wrote: I appreciate you taking the time to help! I'm almost positive it's a local configuration issue, like you said, but without knowing how Galaxy generates or parses its internal links (in the form of [Galaxy web address]/datasets/[16 character hash]/display/[filename] ) it's tough to know how to attack the problem. I dug through some of the database tables in psql, but I couldn't find much of interest there either. hg tip result is: 7148:17d57db9a7c0 -- that's the May 9th release Thanks again for your help! Dan -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] wrapping qvalue in Galaxy
This is probably because R on a server requires an X11 framebuffer since your process is not able to open an X11 server connection while it's running. Does http://stackoverflow.com/questions/1710853/how-to-run-r-on-a-server-without-x11-and-avoid-broken-dependencies help? On Thu, Jun 7, 2012 at 8:57 AM, Christophe Antoniewski droso...@gmail.com wrote: Hi, I am trying to wrap the qvalue R package in a script (as I have already done for a dozen), but this time I have repeatedly the same error In fun(libname, pkgname) : no DISPLAY variable so Tk is not available or Loading Tcl/Tk interface ... done (and red, desperately red box). I already had this type of bugs and fixed them by using the suppressWarnings or suppressMessages R methods. But this time, no way. I really don't need the Tcl/Tk interface and I am not using it with qvalue... frustrating. (my code run into R in command lines like a charm) Any clue to shut off Galaxy and get this miserable pvalue vector output ? Christophe PS. I browsed the net, found traces of problems similar to mine, but unfortunately no trace of solutions. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] I can't see or access pictures in FastQC
Hi, Daniel. The wrapper doesn't introduce any new dependencies other than a working FastQC install in the right place so can you please let us know what happens when you run FastQC from the command line as the user running your Galaxy processes - it should start an interactive FastQC session where you can try running a real fastq input and check that you get the HTML outputs. Once it's all working from the command line, try the tool again and let us know how that turns out please? On Thu, Jun 7, 2012 at 6:58 AM, Dorset, Daniel C daniel.dor...@vanderbilt.edu wrote: I’m running two separate instances of Galaxy on two different servers. On both servers, whenever I run FastQC, I can see the result HTML file, but none of the pictures show up. Furthermore, if I try to access the links at the bottom of the page, they don’t lead to any real files. If I look in my database/files/subfolder, I see a dataset_### file that contains the HTML info, which I am able to see in Galaxy. There is another folder named “dataset_###_files,” and all of the pictures and the FastQC results zip file are contained in there. I can access them with full read permissions, and the sizes appear correct. Has anyone else experienced this problem? ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Re : Error executing tool: cannot find 'database'
. Otherwise it is going to be difficult to guess, what is going wrong Regards, Hans On 05/30/2012 11:10 AM, Marine Rohmer wrote: Hi everyone, I'm trying with all my best to add an NGS tool into my local Galaxy instance. I've added my tool in tool_conf.xml, and made a XML file with all the command, param etc... I think it's pretty ok, and everything is correctly displayed on the Galaxy interface. However, when I execute my tool, Galaxy says to me : Error executing tool: cannot find 'database' And in my shell, I can see : 127.0.0.1 - - [30/May/2012:10:47:01 +0200] GET /tool_runner?tool_id=myTool HTTP/1.1 200 - http://localhost:8080/root/tool_menu; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0 galaxy.tools ERROR 2012-05-30 10:47:07,424 Exception caught while attempting tool execution: Traceback (most recent call last): File /home/myName/work/galaxy-dist/lib/galaxy/tools/__init__.py, line 1345, in handle_input _, out_data = self.execute( trans, incoming=params, history=history ) File /home/myName/work/galaxy-dist/lib/galaxy/tools/__init__.py, line 1661, in execute return self.tool_action.execute( self, trans, incoming=incoming, set_output_hid=set_output_hid, history=history, **kwargs ) File /home/myName/work/galaxy-dist/lib/galaxy/tools/actions/__init__.py, line 333, in execute data.name = fill_template( output.label, context=params ) File /home/myName/work/galaxy-dist/lib/galaxy/util/template.py, line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File /home/myName/work/galaxy-dist/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py, line 1004, in __str__ return getattr(self, mainMethName)() File DynamicallyCompiledCheetahTemplate.py, line 83, in respond NotFound: cannot find 'database' 127.0.0.1 - - [30/May/2012:10:47:07 +0200] POST /tool_runner/index HTTP/1.1 200 - http://localhost:8080/tool_runner?tool_id=myTool; Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:12.0) Gecko/20100101 Firefox/12.0 I'm a beginner in computer sciences, so I can't understand since over 1 week what's happening ... Any idea what's wrong with this ? (I'm using Ubuntu LTS 12.04) All help will be very very appreciated ! Thank you, Marine ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] About Galaxy Configuration
Hi, Ciara, AFAIK new tools always require a server restart at present, but an already loaded tool can have updates to the tool XML re-read using the 'Reload a tool's configuration' option from the administrative menu if your login is listed as an administrator in the config file. Restarting Galaxy seems relatively benign if jobs are running on a cluster. SGE jobs (eg) will be correctly handled and managed. Busy users may see a service unavailable message from Apache for the time it takes to restart but in my experience they can generally cope with the occasional unscheduled server bounce to import a new tool - YMMV On Thu, May 24, 2012 at 1:10 PM, Ciara Ledero lede...@gmail.com wrote: Hi there, I understand that everytime I create a new tool, I would have to restart Galaxy to see the changes that I've made. But if somebody else has already started Galaxy on our server, I won't be able to see my newly developed tool. I would be able to access the site, yes, but I won't see my tool. Is there a way around this? Cheers, CL ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Interested in speaking with other institutions deploying Galaxy locally?
To all interested in Dave's suggestion: IMHO, an Australian/New Zealand Galaxy user/deployer/developer group will be worth setting up and will be viable if it solves specific problems that aren't adequately addressed by existing networks and communication channels - otherwise it will be JAFUG (just another fleeting user group) and will likely fizzle due to lack of genuine utility to potential participants. EG: One challenge for us when it comes to (eg) the proposed teleconferences is the tyranny of time zones - being 14 hours ahead of EST eg in Melbourne/Sydney. The other big challenge is that travel to meetings in Europe or the USA is expensive and exhausting, so workshops and hands on training are harder to get to. In general, I think we're probably going to be able to find plenty of problems to do with geographic isolation and maybe even some specific Australian challenges - but let's be clear about exactly what those are before we start building any structures. One problem I keep seeing is that there are many organisations happy to invest in hardware infrastructure, but far less willing to invest in the training and staff development needed for success. Two things I'd like to suggest to help with planning: 1) to get a better idea of scope, it would be very handy to assemble a register of potential organisations and their primary contacts. I already know of some large Australian organisations investing heavily in Galaxy deployment - eg - VLSCI in Melbourne; UQ in Brisbane, CSIRO all over the place, the Victorian DPI, AGRF, and the BakerIDI where I'm based - no doubt there are plenty of others too? 2) to make sure the idea is viable, it would be useful to collect some specific deployment challenges that people are experiencing that some kind of local user's group could help overcome? One focus I have a personal interest in exploring would be in face to face educational activities: eg for local Galaxy instance system administrators? Local tool developers? Local biologist users in specific contexts like short read sequencing? Suggestions welcomed... On Mon, Apr 30, 2012 at 12:37 PM, Dave Clements cleme...@galaxyproject.org wrote: Hi Ryan, Ann, and everyone else I second what Nate says (I always do :-). I too like that it is user driven. And, while I am not a developer, I do plan on being on the call as often as possible. I can also offer my support for logistical and any other support. Ann, please let me know if you want help with getting this going and getting the word out. On a related note, With Matloob's email, we have at least 3 organizations in Australia and New Zealand that are interested. I know that there are many more Galaxy installations in that part of the world (and you have Enis and Ross in Australia now). Would this be a good opportunity to set up the first (to my knowledge) regional user group? If anyone thinks this is a good idea, please respond (and create a new thread). Finally, Ann, thanks for starting this. Dave C I also that both Ross and Enis are in Australia now On Sun, Apr 29, 2012 at 10:24 AM, Nate Coraor n...@bx.psu.edu wrote: Hi Ryan, I like that it's user-directed, and we could be there to provide input, although if help is needed to organize the call we could probably assist with that. I plan to be on the call as often as my schedule allows, and I believe some of the other developers on the team would also be interested. --nate On Apr 28, 2012, at 8:49 AM, Ryan Golhar wrote: One question - Are the Galaxy developers involved in this or is this for user's only? It may be helpful to have developers on the call to provide information that users do not necessarily have. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found
Hi, Cai - sorry to hear it's still not working right.
You've compiled/installed samtools using the --prefix configure option
and tested that all the samtools based Galaxy wrappers work if you use
the local job runner? That means that both the appropriate lib and bin
directories are on the Galaxy user's path ready to be inherited by SGE
with the -V SGE option.
If the local job runner works, that seems to isolate the problem to
SGE configuration and operation - the steps I've suggested seem to
work for us so I've run out of ideas on why it's not working at your
shop and hope somebody with more experience can help out.
On Wed, Apr 18, 2012 at 10:50 AM, zhengqiu cai caizhq2...@yahoo.com.cn wrote:
Hi Ross,
Thank you for your reply.
It still did not work.
I think SGE knows the path of samtools. Here is a test case I added to
Galaxy, and it creates the index for fasta file. I pasted the samtools faidx
wrapper and xml files below:
ubuntu@master:/mnt/galaxyTools/galaxy-dist/tools/samtools$ more
samtools_faidx.pl
#!/usr/bin/perl
use strict;
use warnings;
if (@ARGV 1) {
die Usage: perl $0 input fasta file\n;
}
`samtools faidx $ARGV[0]`;
`cp ${ARGV[0]}.fai $ARGV[1]`;
exit;
ubuntu@master:/mnt/galaxyTools/galaxy-dist/tools/samtools$ more
samtools_faidx.xml
tool id=samtools_faidx name=samtools faidx version=1.0.3
requirements
requirement type=packagesamtools/requirement
/requirements
descriptioncreates fasta index/description
command interpreter=perl
samtools_faidx.pl $input1 $output1
/command
inputs
param name=input1 type=data format=fasta label=fasta file to be
indexed /
/inputs
outputs
data format=fai name=output1 label=${tool.name} on ${on_string}:
fasta /
/outputs
/tool
It works, which shows that SGE knows the path of samtools.
Cai
--- 12年4月18日，周三, Ross ross.laza...@gmail.com 写道：
发件人: Ross ross.laza...@gmail.com
主题: Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh:
samtools: not found
收件人: zhengqiu cai caizhq2...@yahoo.com.cn
抄送: Jennifer Jackson j...@bx.psu.edu, galaxy-dev@lists.bx.psu.edu
日期: 2012年4月18日,周三,上午5:18
Ah. Then you probably have a library
path problem.
Here's how we do it FWIW:
Assume your want your shared cluster apps in
/data/apps/bin
To share the compiled libraries and binaries together,
remember to
compile apps like samtools using the
--prefix /data/app
flag so then you can add the /data/app/lib path to the
galaxy user's
path so the nodes running jobs inherit those paths because
of the -V
switch.
I hope this works!
On Tue, Apr 17, 2012 at 5:10 PM, zhengqiu cai caizhq2...@yahoo.com.cn
wrote:
Hi Ross,
Thank you for your reply.
I tried it, and it was still not working.
Only samtools has this problem, and the other tools
works fine.
samtools was compiled from source correctly. If I
install samtools using sudo apt-get install samtools, this
problem is gone, but this is not a solution for since it
does not work across the cluster.
Cai
--- 12年4月18日，周三, Ross ross.laza...@gmail.com
写道：
发件人: Ross ross.laza...@gmail.com
主题: Re: [galaxy-dev] Exception: Error Setting
BAM Metadata: /bin/sh: samtools: not found
收件人: zhengqiu cai caizhq2...@yahoo.com.cn
抄送: Jennifer Jackson j...@bx.psu.edu,
galaxy-dev@lists.bx.psu.edu
日期: 2012年4月18日,周三,上午4:59
Cai - if you are using SGE you
probably want to pass the Galaxy user's
path to the job using the
-V
switch in the system wide (or Galaxy user's home
directory)
sge_request file
eg I have
rlazarus@iaas1-int:~$ cat
/var/lib/gridengine/default/common/sge_request
-cwd
-V
Without that switch, the job gets no path.
With -V, it inherits the job submitter's full
path.
On Tue, Apr 17, 2012 at 4:26 PM, zhengqiu cai
caizhq2...@yahoo.com.cn
wrote:
Hi Jen,
Thank you for your quick response.
I changed the env.sh file to what you
seggested, and
the problem still existed.
I am pasting more details below:
I submitted a job to convert sam to bam, and
the job
was running forever without outputing the result. I
then
checked the log, and it read:
Traceback (most recent call last):
File
/mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py,
line 336, in finish_job
drm_job_state.job_wrapper.finish(
stdout,
stderr )
File
/mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/__init__.py,
line 637, in finish
dataset.set_meta( overwrite =
False )
File
/mnt/galaxyTools/galaxy-dist/lib/galaxy/model/__init__.py,
line 875, in set_meta
return self.datatype.set_meta(
self, **kwd
)
File
/mnt/galaxyTools/galaxy-dist/lib/galaxy/datatypes/binary.py,
line 179, in set_meta
raise Exception, Error Setting
BAM
Metadata: %s % stderr
Exception: Error Setting BAM Metadata:
/bin/sh:
samtools: not found
It means that the samtools is not in the PATH.
I tried
to set the PATH in a couple
Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found
Hi, Cai.
Apologies but I had not noticed any reference to the cloud
distribution before - that changes everything - I was talking about a
normal cluster/SGE install - you should definitely not need to do
anything at all to the AMI - maybe someone with cloudman experience
can be more helpful than I can.
On Wed, Apr 18, 2012 at 11:39 AM, zhengqiu cai caizhq2...@yahoo.com.cn wrote:
Hi Ross,
I did not compile samtools using the --prefix. The samtools I used comes with
Galaxy AMI image, and I copied it to the same path on my EC2. How to compile
samtools using --prefix? I just typed 'make', and there is not a
configuration file. In the cloudman version of Galaxy (it is working well for
me), only two executables (samtools and maq2sam-long) in the directory
/mnt/galaxyTools/tools/samtools/0.1.12. I copies the same files to my EC2. Is
the only compiled executable 'samtools' fully functional for the cluster(it
is on the local machine)? or should I set some other libraries?
Thank you,
Cai
--- 12年4月18日，周三, Ross ross.laza...@gmail.com 写道：
发件人: Ross ross.laza...@gmail.com
主题: Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh:
samtools: not found
收件人: zhengqiu cai caizhq2...@yahoo.com.cn
抄送: Jennifer Jackson j...@bx.psu.edu, galaxy-dev@lists.bx.psu.edu
日期: 2012年4月18日,周三,下午11:07
Hi, Cai - sorry to hear it's still
not working right.
You've compiled/installed samtools using the --prefix
configure option
and tested that all the samtools based Galaxy wrappers work
if you use
the local job runner? That means that both the appropriate
lib and bin
directories are on the Galaxy user's path ready to be
inherited by SGE
with the -V SGE option.
If the local job runner works, that seems to isolate the
problem to
SGE configuration and operation - the steps I've suggested
seem to
work for us so I've run out of ideas on why it's not working
at your
shop and hope somebody with more experience can help out.
On Wed, Apr 18, 2012 at 10:50 AM, zhengqiu cai caizhq2...@yahoo.com.cn
wrote:
Hi Ross,
Thank you for your reply.
It still did not work.
I think SGE knows the path of samtools. Here is a test
case I added to Galaxy, and it creates the index for fasta
file. I pasted the samtools faidx wrapper and xml files
below:
ubuntu@master:/mnt/galaxyTools/galaxy-dist/tools/samtools$
more samtools_faidx.pl
#!/usr/bin/perl
use strict;
use warnings;
if (@ARGV 1) {
die Usage:
perl $0 input fasta file\n;
}
`samtools faidx $ARGV[0]`;
`cp ${ARGV[0]}.fai $ARGV[1]`;
exit;
ubuntu@master:/mnt/galaxyTools/galaxy-dist/tools/samtools$
more samtools_faidx.xml
tool id=samtools_faidx name=samtools faidx
version=1.0.3
requirements
requirement
type=packagesamtools/requirement
/requirements
descriptioncreates fasta
index/description
command interpreter=perl
samtools_faidx.pl $input1 $output1
/command
inputs
param name=input1 type=data
format=fasta label=fasta file to be indexed /
/inputs
outputs
data format=fai name=output1
label=${tool.name} on ${on_string}: fasta /
/outputs
/tool
It works, which shows that SGE knows the path of
samtools.
Cai
--- 12年4月18日，周三, Ross ross.laza...@gmail.com
写道：
发件人: Ross ross.laza...@gmail.com
主题: Re: [galaxy-dev] Exception: Error Setting
BAM Metadata: /bin/sh: samtools: not found
收件人: zhengqiu cai caizhq2...@yahoo.com.cn
抄送: Jennifer Jackson j...@bx.psu.edu,
galaxy-dev@lists.bx.psu.edu
日期: 2012年4月18日,周三,上午5:18
Ah. Then you probably have a library
path problem.
Here's how we do it FWIW:
Assume your want your shared cluster apps in
/data/apps/bin
To share the compiled libraries and binaries
together,
remember to
compile apps like samtools using the
--prefix /data/app
flag so then you can add the /data/app/lib path to
the
galaxy user's
path so the nodes running jobs inherit those paths
because
of the -V
switch.
I hope this works!
On Tue, Apr 17, 2012 at 5:10 PM, zhengqiu cai
caizhq2...@yahoo.com.cn
wrote:
Hi Ross,
Thank you for your reply.
I tried it, and it was still not working.
Only samtools has this problem, and the other
tools
works fine.
samtools was compiled from source correctly.
If I
install samtools using sudo apt-get install
samtools, this
problem is gone, but this is not a solution for
since it
does not work across the cluster.
Cai
--- 12年4月18日，周三, Ross ross.laza...@gmail.com
写道：
发件人: Ross ross.laza...@gmail.com
主题: Re: [galaxy-dev] Exception: Error
Setting
BAM Metadata: /bin/sh: samtools: not found
收件人: zhengqiu cai caizhq2...@yahoo.com.cn
抄送: Jennifer Jackson j...@bx.psu.edu,
galaxy-dev@lists.bx.psu.edu
日期:
2012年4月18日,周三,上午4:59
Cai - if you are using SGE you
probably want to pass the Galaxy user's
path to the job using the
-V
switch in the system wide (or Galaxy
user's home
Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found
Ah. Then you probably have a library path problem. Here's how we do it FWIW: Assume your want your shared cluster apps in /data/apps/bin To share the compiled libraries and binaries together, remember to compile apps like samtools using the --prefix /data/app flag so then you can add the /data/app/lib path to the galaxy user's path so the nodes running jobs inherit those paths because of the -V switch. I hope this works! On Tue, Apr 17, 2012 at 5:10 PM, zhengqiu cai caizhq2...@yahoo.com.cn wrote: Hi Ross, Thank you for your reply. I tried it, and it was still not working. Only samtools has this problem, and the other tools works fine. samtools was compiled from source correctly. If I install samtools using sudo apt-get install samtools, this problem is gone, but this is not a solution for since it does not work across the cluster. Cai --- 12年4月18日，周三, Ross ross.laza...@gmail.com 写道： 发件人: Ross ross.laza...@gmail.com 主题: Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found 收件人: zhengqiu cai caizhq2...@yahoo.com.cn 抄送: Jennifer Jackson j...@bx.psu.edu, galaxy-dev@lists.bx.psu.edu 日期: 2012年4月18日,周三,上午4:59 Cai - if you are using SGE you probably want to pass the Galaxy user's path to the job using the -V switch in the system wide (or Galaxy user's home directory) sge_request file eg I have rlazarus@iaas1-int:~$ cat /var/lib/gridengine/default/common/sge_request -cwd -V Without that switch, the job gets no path. With -V, it inherits the job submitter's full path. On Tue, Apr 17, 2012 at 4:26 PM, zhengqiu cai caizhq2...@yahoo.com.cn wrote: Hi Jen, Thank you for your quick response. I changed the env.sh file to what you seggested, and the problem still existed. I am pasting more details below: I submitted a job to convert sam to bam, and the job was running forever without outputing the result. I then checked the log, and it read: Traceback (most recent call last): File /mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/runners/drmaa.py, line 336, in finish_job drm_job_state.job_wrapper.finish( stdout, stderr ) File /mnt/galaxyTools/galaxy-dist/lib/galaxy/jobs/__init__.py, line 637, in finish dataset.set_meta( overwrite = False ) File /mnt/galaxyTools/galaxy-dist/lib/galaxy/model/__init__.py, line 875, in set_meta return self.datatype.set_meta( self, **kwd ) File /mnt/galaxyTools/galaxy-dist/lib/galaxy/datatypes/binary.py, line 179, in set_meta raise Exception, Error Setting BAM Metadata: %s % stderr Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found It means that the samtools is not in the PATH. I tried to set the PATH in a couple of methods according the Galaxy documentation: 1. put the path in the env.sh in the tool directory and symbolink default to the tool directory, e.g. default - =/mnt/galaxyTools/tools/samtools/0.1.18 2. put -v PATH=/mnt/galaxyTools/tools/samtools/0.1.18 in ~/.sge_request 3. put -v PATH=/mnt/galaxyTools/tools/samtools/0.1.18 in /path/sge_request none of them worked, and I got the above same problem. Then I checked the job log file in the job_working_directory, and it read: Samtools Version: 0.1.18 (r982:295) SAM file converted to BAM which shows that sge knows the PATH of samtools. To double check it, I added samtools index to Galaxy, and it worked well. I am very confused why SGE knows the tool path but cannot run the job correctly. The system I am using is ubuntu on EC2 (the image id is: ami-999d49f0, which is the one used by StarCluster). I checked out the code from galaxy-dist on bitbucket. Other tools such as bwa and bowtie worked well using the same setting method(put env.sh in the tools directory to set the tool path) Thanks you very much, Cai --- 12年4月18日，周三, Jennifer Jackson j...@bx.psu.edu 写道： 发件人: Jennifer Jackson j...@bx.psu.edu 主题: Re: [galaxy-dev] Exception: Error Setting BAM Metadata: /bin/sh: samtools: not found 收件人: zhengqiu cai caizhq2...@yahoo.com.cn 抄送: galaxy-dev@lists.bx.psu.edu 日期: 2012年4月18日,周三,上午2:12 Hi Cai, Please double check that your env.sh file contains the following: PATH=/mnt/galaxyTools/tools/samtools/0.1.12:$PATH export PATH http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies Hopefully this helps, Jen Galaxy team On 4/16/12 12:43 PM, zhengqiu cai wrote: Hi All, I was trying to setup the Galaxy tool dependencies, and I met the following problem when running sam-to-bam in samtools: Traceback (most recent call last): File /mnt/galaxyTools/cai_galaxy-dist/lib/galaxy/jobs/runners/drmaa.py, line 336, in finish_job drm_job_state.job_wrapper.finish( stdout, stderr ) File /mnt/galaxyTools/cai_galaxy-dist/lib/galaxy/jobs/__init__.py, line 637, in finish dataset.set_meta( overwrite = False ) File /mnt/galaxyTools/cai_galaxy
Re: [galaxy-dev] Workflow import error
) step.upgrade_messages = module.check_and_update_state() AttributeError: 'NoneType' object has no attribute 'check_and_update_state' -Rob On Mon, Apr 16, 2012 at 9:19 AM, Robert Chase re...@channing.harvard.edu wrote: Thanks Greg, I'll try to merge the change into my version. -Rob On Mon, Apr 16, 2012 at 9:18 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Robert, Thanks for reporting this issue - it has been fixed in change set 7038:1fdcce63a06f, which is currently available only in our central repository. It will be available in the dist in the next update. Greg Von Kuster On Apr 13, 2012, at 5:22 PM, Robert Chase wrote: Hello All, I'm pretty sure I found a bug in the main galaxy distribution. The following code diff shows the introduction of the line tool_version = self.__get_tool_version( trans, tool_id ) but if you look at the function definition containing that line, you see def from_workflow_step( Class, trans, step ): which does not half a self in it. Then we get an error. https://bitbucket.org/galaxy/galaxy-dist/diff/lib/galaxy/workflow/modules.py?diff2=e6464387ed3fdiff1=c60760713d26 -Rob ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] incorrect tmp dir handling in picard AddOrReplaceReadGroups
Hi, Matt.
Thanks for reporting this - it's definitely a misfeature...
The tool wrapper tools/picard/picard_AddOrReplaceReadGroups.xml does
not explicitly set the tmpdir parameter, so I've added --tmpdir
${__new_file_path__} to the command line generated. That will
over-ride the default value /tmp for opts.tmpdir in
picard_wrapper.py to use the new_file_path.
It is now in galaxy-central - please update your local copy and test
it to confirm this solves your problem?
On Tue, Apr 3, 2012 at 2:55 PM, Matt Shirley matt.shir...@jhmi.edu wrote:
Every time I run Picard's AddOrReplaceReadGroups, my system tmp directory
fills, and the tool stops. I'm left scratching my head, because it seems
that the JVM sets the system /tmp directory on my local galaxy-central
branch, instead of the new_file_path I have defined in universe_wsgi.ini.
This issue seems to have been fixed in commit 07560293b167, and I can
confirm that my picard_wrapper.py includes the amended line. I am using
OpenJDK:
java version 1.6.0_20
OpenJDK Runtime Environment (IcedTea6 1.9.10)
(rhel-1.40.1.9.10.el6_1-x86_64)
OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
Since my config is pointing to /scratch/tmp, but the JVM is using /tmp, and
other tools in galaxy are respecting the config, what could be the
disconnect? Thank you,
Matt
--
Matt Shirley
Ph.D Candidate - BCMB
Pevsner Lab
Johns Hopkins Medicine
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
--
Ross Lazarus MBBS MPH;
Associate Professor, Harvard Medical School;
Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] strange error having to do with stderr redirection?
\n\n Vignettes contain introductory material. To view, type\n \'browseVignettes()\'. To cite Bioconductor, see\n \'citation(Biobase)\' and for packages \'citation(pkgname)\'.\n\nLoading required package: locfit\nLoading required package: akima\nLoading required package: lattice\nWarning messages:\n1: In xy.coords(x, y, xlabel, ylabel, log) :\n 1710 x values = 0 omitted from logarithmic plot\n2: In xy.coords(x, y, xlabel, ylabel, log) :\n 2880 y values = 0 omitted from logarithmic plot\nWarning message:\nIn xy.coords(x, y, xlabel, ylabel, log) :\n 1710 x values = 0 omitted from logarithmic plot\nLoading required package: R.methodsS3\nR.methodsS3 v1.2.1 (2010-09-18) successfully loaded. See ?R.methodsS3 for help.\nLoading required package: R.oo\nR.oo v1.8.3 (2011-11-01) successfully loaded. See ?R.oo for help.\n\nAttaching package: \xe2\x80\x98R.oo\xe2\x80\x99\n\nThe following object(s) are masked from \xe2\x80\x98package:R.methodsS3\xe2\x80\x99:\n\n throw.default\n\nThe following object(s) are masked from \xe2\x80\x98package:methods\xe2\x80\x99:\n\ngetClass, getClasses, getMethods\n\nThe following object(s) are masked from \xe2\x80\x98package:base\xe2\x80\x99:\n\nattach, detach, environment, gc, load, save\n\naroma.light v1.22.0 (2011-10-31) successfully loaded. See ?aroma.light for help.\n', 1345] It seems that R produces some output on stdout that I can't get rid of yet, but it doesn't seem to affect the outcome... it doesn't explain why this error happens... Any ideas? Is R using some sort of weird encoding that the python can't get? - Nik. -- Nikhil Joshi Bioinformatics Programmer UC Davis Genome Center Davis, CA ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Extracting the metrics files from a composite file such as Picard
Thon, I just had an idea - write a new tool that takes an Html dataset from the user's history and a file specification (eg foo.xls) and 'promotes' the file(s) in the extra_files_path that match the file specification into new history items? That way you can automate the process - of course, including the outputs in workflows when their number is not known at execution may be tricky, but at least this is a generic approach and won't require any changes to any of the tools that generate Html outputs? On Tue, Feb 21, 2012 at 8:45 AM, Anthonius deBoer thondeb...@me.com wrote: Hi Ross, Thanks for the reply...It is indeed as I feared...I'll probably refactor the code so the user is able to get the actual output files. Thon On Feb 20, 2012, at 01:29 PM, Ross ross.laza...@gmail.com wrote: Hi, Thon, On Tue, Feb 21, 2012 at 6:47 AM, Thon Deboer thondeb...@me.com wrote: Hi, Is there a way to extract the individual files from a composite file, such as the HTML files created by the picard tools? The decision to hide multiple outputs in a single history html object has this as a downside to the benefits of less cluttered histories. The results you want can be manually extracted in the usual ways - eg pasting the relevant html page url into an upload box.. but that doesn't solve your challenge of automating the process for a very large number of datasets. One possible solution is to add some complexity to each of the relevant tool forms to allow the user to specifically nominate outputs to be returned as individual new history datasets. It is not a huge task but it's not as far as I'm aware, high on the list of priorities for the team. If you are motivated sufficiently to fix this so it can do what you need, contributions of code to improve Galaxy tools are always very welcome ? I would like to take the metrics files and use them further down in some workflow, but I only get an HTML file... While these HTML files are nice for quickly looking at the results of one or two files, it becomes a problem if you have 88 samples like I do... I hate to have to re-write all the picard tools to produce actual files, but maybe there is something I am missing about composite datatype files? The wiki only explains how to CREATE them, not how to use those files downstream Regards, Thon de Boer, Ph.D. Bioinformatics Guru +1-650-799-6839 thondeb...@me.com LinkedIn Profile http://www.linkedin.com/pub/thon-de-boer/1/1ba/a5b ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; -- Ross Lazarus MBBS MPH; Associate Professor, Harvard Medical School; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444; ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
