[galaxy-dev] Hiding a tool via tool_conf.xml
Hi all, I'm aware that in an individual tool's XML wrapper you can add hidden=true to the tool element to hide the tool from the listing shown in Galaxy's left hand panel (but the tool is still loaded and can be called by old workflows etc). Can this be done via the tool_conf.xml file as well? This would seem useful from a system configuration point of view where it seems wrong to edit the tool wrappers themselves. In fact, it might also be nice to add hidden=true support to section as well as tool in tool_conf.xml - although I'm not so sure how useful that would be. Peter ___ galaxy-dev mailing list galaxy-dev@lists.bx.psu.edu http://lists.bx.psu.edu/listinfo/galaxy-dev
Re: [galaxy-dev] unable to login user to local instance
On Fri, Feb 18, 2011 at 5:01 PM, Ryan Golhar golha...@umdnj.edu wrote:
Ok, I don't know what happened with this...I reinstalled from scratch and ...
Traceback (most recent call last):
File /home/galaxy/galaxy-dist/tools/new_operations/get_flanks.py, line
16, in ?
...
File
/home/galaxy/galaxy-dist/lib/galaxy/web/framework/helpers/__init__.py,
line 29
return (content if len(content) = length else content[:length].rsplit('
', 1)[0]+suffix)
^
SyntaxError: invalid syntax
I deduce you're using Python 2.4, since that line of code
is using the ternary operator added in Python 2.5
http://www.python.org/dev/peps/pep-0308/
Now that it has been reported I think it will be fixed to work with
Python 2.4. However, the Galaxy team have indicated they will
be dropping Python 2.4 (probably this year) so if I were you
since this is a new setup I'd install Python 2.6 on this machine
(Galaxy doesn't yet support Python 2.7).
Peter
___
To manage your subscriptions to this and other Galaxy lists, please use the
interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Hiding a tool via tool_conf.xml
On Thu, Feb 10, 2011 at 5:58 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I'm aware that in an individual tool's XML wrapper you can add hidden=true to the tool element to hide the tool from the listing shown in Galaxy's left hand panel (but the tool is still loaded and can be called by old workflows etc). Can this be done via the tool_conf.xml file as well? This would seem useful from a system configuration point of view where it seems wrong to edit the tool wrappers themselves. In fact, it might also be nice to add hidden=true support to section as well as tool in tool_conf.xml - although I'm not so sure how useful that would be. Peter I've filed an enhancement request for this: https://bitbucket.org/galaxy/galaxy-central/issue/481/ Peter ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Shared a value for multiple set-at-runtime workflow parameters
Hi all, I have a multi-step workflow where I've marked some parameters to be set at run time (in this case, a minimum read length, used in two different workflow steps). I want to use the same parameter value in multiple tools, rather than give the user one prompt for each tool. Is this possible? Related to this, it would be nice to specify a default in the workflow (overriding any default in the tool wrapper), and give some caption text to the workflow level parameter. I guess some of this functionality would be related to any enhancement to make a workflow act like a tool (e.g. to combine subworkflows into larger workflows). Thanks, Peter ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Shared a value for multiple set-at-runtime workflow parameters
On Wed, Mar 2, 2011 at 3:42 PM, Peter Cock p.j.a.c...@googlemail.com wrote:
P.S. I do like the color highlighting and live substitution of
the workflow parameters into each tool step that uses them.
Very slick!
That was the good news. Now the bad news - two bug reports.
Firstly if you clear the workflow parameter (leaving it empty), then
the caption is used for the highlighted placeholders BUT only the
first word is used, e.g.
${Minimum read length (after any clipping)}
Initially the the placeholders show Minimum read length (after any
clipping), and if you enter a value like 30 then 30 is shown. If
you then edit the field to become empty, the placeholder now
shows just Minimum.
Furthermore, if I leave my workflow parameters empty, and
press execute, I get a traceback ending with:
AttributeError: 'WorkflowStep' object has no attribute 'module'
Since an empty value is valid for some tools for some parameters
(e.g. optional text fields) I would have expected any error later -
from the tools themselves if they don't like the empty value.
Peter
___
To manage your subscriptions to this and other Galaxy lists, please use the
interface at:
http://lists.bx.psu.edu/
[galaxy-dev] QUAL file format(s)
Hi all, Is there any documentation on what Galaxy means by the different qual formats? In particular why is there a qual454 and not a qualsanger (to match fastq)? Historically QUAL files were used back in Sanger/capillary sequencing by PHRED, and just hold PHRED scores as integers. This was followed by the Roche 454 output. Peter ___ To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Fwd: [Bosc] Bioinformatics Open Source Conference (BOSC 2011)--Call for Abstracts
Hopefully this will be of interest to some of you... -- Forwarded message -- From: Nomi Harris nlhar...@lbl.gov Date: Thu, Mar 3, 2011 at 7:37 PM Subject: [Bosc] Bioinformatics Open Source Conference (BOSC 2011)--Call for Abstracts To: bosc-annou...@lists.open-bio.org, memb...@open-bio.org, GMOD Announcements List gmod-annou...@lists.sourceforge.net, GMOD Developers List gmod-de...@lists.sourceforge.net Cc: Nomi Harris nlhar...@lbl.gov We invite you to submit an abstract to BOSC 2011! Please forward this message as appropriate, and forgive multiple postings. Call for Abstracts for the 12th Annual Bioinformatics Open Source Conference (BOSC 2011) An ISMB 2011 Special Interest Group (SIG) Dates: July 15-16, 2011 Location: Vienna, Austria Web site: http://www.open-bio.org/wiki/BOSC_2011 Email: b...@open-bio.org BOSC announcements mailing list: http://lists.open-bio.org/mailman/listinfo/bosc-announce Important Dates: April 18, 2011: Deadline for submitting abstracts to BOSC 2011 May 9, 2011: Notifications of accepted abstracts emailed to corresponding authors July 13-14, 2011: Codefest 2011 programming session (see http://www.open-bio.org/wiki/Codefest_2011 for details) July 15-16, 2011: BOSC 2011 July 17-19, 2011: ISMB 2011 The Bioinformatics Open Source Conference (BOSC) is sponsored by the Open Bioinformatics Foundation (O|B|F), a non-profit group dedicated to promoting the practice and philosophy of Open Source software development within the biological research community. To be considered for acceptance, software systems representing the central topic in a presentation submitted to BOSC must be licensed with a recognized Open Source License, and be freely available for download in source code form. We invite you to submit abstracts for talks and posters. Sessions include: - Approaches to parallel processing - Cloud-based approaches to improving software and data accessibility - The Semantic Web in open source bioinformatics - Data visualization - Tools for next-generation sequencing - Other Open Source software In addition to the above sessions, there will be a panel discussion about Meeting the challenges of inter-institutional collaboration. We are also working to arrange a joint session with one of the other ISMB SIGs. Thanks to generous sponsorship from Eagle Genomics and an anonymous donor, we are pleased to announce a competition for three Student Travel Awards for BOSC 2011. Each winner will be awarded $250 to defray the costs of travel to BOSC 2011. For instructions on submitting your abstract, please visit http://www.open-bio.org/wiki/BOSC_2011#Abstract_Submission_Information BOSC 2011 Organizing Committee: Nomi Harris and Peter Rice (co-chairs); Brad Chapman, Peter Cock, Erwin Frise, Darin London, Ron Taylor ___ BOSC mailing list b...@lists.open-bio.org http://lists.open-bio.org/mailman/listinfo/bosc ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] missing setup.sh in galaxy source code
On Sun, Mar 6, 2011 at 6:36 AM, Zhi-Wei Lu z...@ucdavis.edu wrote: Hi Galaxy developers, I have download the galaxy code via mercurial hg clone https://bitbucket.org/galaxy/galaxy-central The downloaded code, however, does not contain the setup.sh file. Could you let me know how do I run setup now? Thank you. There is no setup.sh file any more, see for example: http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-February/004346.html If there is some out of date documentation that needs fixing, please report it Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] possible bug - user log in / browser cache
On Mon, Mar 7, 2011 at 5:56 PM, Nate Coraor n...@bx.psu.edu wrote: Ryan Golhar wrote: I have a local instance running. When I'm logged in as user1, log out, then try to log in as user2, I keep getting the You have been logged out message. Its as if the browser's cache is retaining some information. To get around this, after I log out as user1, I have to click on Analyze Data, then click on Log in to log in as user2. I'm running Safari on Mac, but have noticed this with Firefox as well. Hi Ryan, The login page will redirect you back to whatever page you to whatever page you were on before clicking log in. If this page is the logout page, the behavior you reported will be what happens. If you log in from any other page, this should go away. Bug confirmed on Firefox on Mac, and Chrome on Windows using http://test.g2.bx.psu.edu/ I guess you just need to say if the previous page was the logout page, go to the main page instead, rather than a blind redirect. Issue filed: https://bitbucket.org/galaxy/galaxy-central/issue/486 I expect many people to be caught out by this (back when I was debugging a proxy problem I was probably hitting this bug too). Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Upload files from filesystem paths - backwards checkboxes
On Thu, Mar 10, 2011 at 5:53 PM, Glen Beane glen.be...@jax.org wrote: On Mar 10, 2011, at 12:24 PM, Peter Cock wrote: Hi all, Via the Galaxy admin interface, I'm currently importing some BAM files into a library from the local file system. Since they are big, I don't want Galaxy to copy them. So I looked at the Copy data into Galaxy? option and made sure it wasn't ticked. Galaxy did copy the file :( I looked at the interface again, and see you have: Copy data into Galaxy? [check box] No Normally data uploaded with this tool is copied into Galaxy's files directory so any later changes to the data will not affect Galaxy. However, this may not be desired (especially for large NGS datasets), so use of this option will force Galaxy to always read the data from its original path. You've got a tick box labelled No so it does the opposite to the question above it. That is a HORRIBLE interface. I can't be the first person to be tricked by it, can I? I had a co-worker mess this up too. I thought this was a poor design decision. I thought maybe yes/no radio options would work. Copy data into galaxy? yes(*) no( ) I would urge you to remove the work No and flip the meaning (and default) to match. But that could confuse existing admin users used to clicking from previous habit. How about making it a select parameter instead, with options Copy files (default) and Link to files which is very explicit? I think this would be better than the current no check box, and more explicit than my yes(*) no () radio buttons You may be right, but switching from a checkbox to a select isn't as easy as it could be... at least not with my familiarity with the code base. Likewise for the Preserve directory structure? setting, I'd remove the No caption and flip the setting, or just replace the backwards tick box with a select parameter offering Preserve directory structure and Place in selected folder (or something like that). Pretty please? How about if I write and test the patch? Patch for flipping the Preserve directory structure? setting, which I hope you can test then merge/transplant to the trunk: https://bitbucket.org/peterjc/galaxy-central/changeset/669dcc2803ba I'm still working on the Copy data into Galaxy? setting... Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Upload files from filesystem paths - backwards checkboxes
On Fri, Mar 11, 2011 at 3:04 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Peter, Thanks for the patch - I've applied a corrected version of it in change set 5215:74c4dd43485a. Let me know if you'd rather have me flip the Copy data into Galaxy? feature. I'd be happy to, but I don't want to reinvent the wheel if you're close. Thanks! Greg Von Kuster Using util.string_as_bool looks good - I didn't know about that and it looks far more rhobust. Would you might doing Copy data into Galaxy? as well? I got most of the way but got dragged off to a meeting in the middle. Thanks! Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] History information popup broken on library entries?
Hi all, I've been exploring how to use Galaxy's shared library functionality, and have noticed that once imported into a working history, that the little 'i' information icon's black popup window isn't working for these datasets. All it says is 'Could not find the job for this dataset.' This appears to affect multiple file types (FASTA and BAM tested) and both files copied into Galaxy or linked to at import. I'm using the current latest code from galaxy-central, https://bitbucket.org/galaxy/galaxy-central/changeset/e39495db6071 This also seems to affect http://test.g2.bx.psu.edu/ and also http://main.g2.bx.psu.edu/ Can anyone confirm this behaviour? I'm using Firefox on the Mac, but I doubt that is relevant. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Metadata recursion error with empty tabular file (with patch)
On Fri, Mar 18, 2011 at 6:12 PM, Kanwei Li kan...@gmail.com wrote: Thanks for reporting, committed fix https://bitbucket.org/galaxy/galaxy-central/changeset/3070455afbdd Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Alternative bowtie tools
On Tue, Mar 29, 2011 at 1:31 AM, Assaf Gordon gor...@cshl.edu wrote: Hello all, We're developing alternative bowtie tools that more closely suit our needs, are we're happy to share (and get comments). The main differences are: 1. separate tools for paired-end and single-end Sounds sensible to me. 2. the tools accepts FASTA, FASTQ in both Sanger and Illumina format (no more need for grooming). Illumina is the default for newly uploaded FASTQ files. I think that's a bad idea - use Sanger FASTQ as the default to be consistent with the rest of Galaxy, and also with CASAVA 1.8 Illumina machines will produce that too, see: http://seqanswers.com/forums/showthread.php?t=8895 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Alternative bowtie tools
On Tue, Mar 29, 2011 at 4:46 PM, Assaf Gordon gor...@cshl.edu wrote: Hi Dan, Daniel Blankenberg wrote, On 03/29/2011 10:55 AM: When files are added to Galaxy, the datatype can be directly set to any of the fastq variants (e.g. fastqillumina), which removes the requirement of grooming (but should only be done when users know what they are doing). I'm not using the get data tool, we have our own import tools (uploading huge files with HTTP is not stable enough for me). You're right that I should change the format of this tool from 'fastq' to 'fastqillumina' (but the tool pre-dated all those built-in formats in galaxy, so I never bothered to update it...). Why not do the Illumina to Sanger conversion as part of your pipeline that gets the data into Galaxy (and mark the files as fastqsanger)? As Glen said, with a C tool that isn't really so slow. That future proofs you for the pending Illumina CASAVA 1.8 release, and means you don't need to maintain divergent Bowtie wrappers for Galaxy. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Updating BLAST+ wrapper
Hi all, The NCBI have just released BLAST 2.2.25+ which includes some interesting new stuff of interest to the tabular output, i.e. Added support for query and subject length to tabular output I would therefore like to update my BLAST+ wrappers in Galaxy to add these two columns to the 'extended tabular' output option. They are going to be very helpful as you can now calculate things like the percentage identity (or similarity) compared to the query or sequence. If there are no objections I propose to add these as columns 23 and 24 (i.e. at the end), so minimise disruption to anyone already using the tabular output. I would hope to have a branch ready for merging next week... Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Default value for data_column not working?
On Fri, Apr 1, 2011 at 3:55 PM, Kanwei Li kan...@gmail.com wrote: Hi Peter, I don't see your attempt to use default in the example given. A bit more detail? -K Hi Kanwei, My example tool is now here: https://bitbucket.org/peterjc/galaxy-central/changeset/198bf927ca30 As per the original email, I have three parameters which are column numbers, and I want the defaults to be columns 1, 2 and 12. However, on my Galaxy installation they all default to the first column. I have tried giving the default values as 1, 2, and 12, and also as c1, c2 and c12 - neither works: https://bitbucket.org/peterjc/galaxy-central/changeset/374143be8e45 If you need more information, let me know. Thanks, Peter P.S. Bug filed here: https://bitbucket.org/galaxy/galaxy-central/issue/507/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating BLAST+ wrapper
On Mon, Apr 4, 2011 at 12:19 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Fri, Apr 1, 2011 at 12:01 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Thu, Mar 31, 2011 at 6:28 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, The NCBI have just released BLAST 2.2.25+ which includes some interesting new stuff of interest to the tabular output, ... One reason why I want to move to BLAST 2.2.25+ on our local machine is they fixed the subject IDs in tabular output, ... Hi Kanwei, Could you transplant or merge this commit please? https://bitbucket.org/peterjc/galaxy-central/changeset/ab40f95393ec It updates the test output files to work with BLAST 2.2.25+, which I will be using locally due to several important bug fixes. As you don't offer BLAST+ on the Penn State server I don't expect this to cause you any problems. Also (and unrelated to this change), do you see this from the test suite? $ ./run_functional_tests.sh -id ncbi_blastp_wrapper ... == ERROR: NCBI BLAST+ blastp ( ncbi_blastp_wrapper ) Test-1 -- Traceback (most recent call last): File /home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py, line 155, in test_tool self.do_it( td ) File /home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py, line 71, in do_it page_inputs = self.__expand_grouping(testdef.tool.inputs_by_page[0], all_inputs) File /home/pjcock/repositories/galaxy-central/test/functional/test_toolbox.py, line 122, in __expand_grouping elif isinstance(declared_inputs[value.name], str): KeyError: 'blast_type' Solved, it seems I can't omit any parameters in the test and expect the defaults to be used. This doesn't seem desirable (new bug?), but the quick fix is to add a few lines to the test. Could you commit this too: https://bitbucket.org/peterjc/galaxy-central/changeset/124e4556f3c5 Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating BLAST+ wrapper
On Mon, Apr 4, 2011 at 4:03 PM, Kanwei Li kan...@gmail.com wrote: I tried both ways, no dice (you can try yourself with a clean galaxy-central base) I'm a bit puzzled and not an hg expert. Was there an error message? And for plan (B), what went wrong with using patch and the raw changes? Surely that should work? https://bitbucket.org/peterjc/galaxy-central/changeset/ab40f95393ec/raw/galaxy-central-ab40f95393ec.diff https://bitbucket.org/peterjc/galaxy-central/changeset/124e4556f3c5/raw/galaxy-central-124e4556f3c5.diff Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating BLAST+ wrapper
On Mon, Apr 4, 2011 at 4:36 PM, Kanwei Li kan...@gmail.com wrote: dhcp243253:galaxy-central kanwei$ hg transplant -s ~/peter/galaxy-central/ -b blast25 searching for changes changeset: 5585:ab40f95393ec branch: blast25 parent: 5583:086c9c2c52b9 user: peterjc p.j.a.c...@googlemail.com date: Mon Apr 04 12:12:11 2011 +0100 summary: Update tests for NCBI BLAST 2.2.25+ which fixed subject ID in tabular output apply changeset? [ynmpcq?]: y changeset: 5586:124e4556f3c5 branch: blast25 user: peterjc p.j.a.c...@googlemail.com date: Mon Apr 04 14:44:52 2011 +0100 summary: Include optional parameters in BLASTP test to avoid KeyError apply changeset? [ynmpcq?]: y searching for changes adding changesets adding manifests adding file changes added 2 changesets with 4 changes to 4 files dhcp243253:galaxy-central kanwei$ hg tip changeset: 5338:124e4556f3c5 branch: blast25 tag: tip user: peterjc p.j.a.c...@googlemail.com date: Mon Apr 04 14:44:52 2011 +0100 summary: Include optional parameters in BLASTP test to avoid KeyError The above is strange - why have you got a blast25 branch? Maybe the transplant isn't working as expected... dhcp243253:galaxy-central kanwei$ hg push pushing to https://kan...@bitbucket.org/galaxy/galaxy-central/ searching for changes abort: push creates new remote branches: blast25! (use 'hg push --new-branch' to create new remote branches) dhcp243253:galaxy-central kanwei$ hg merge blast25 abort: merging with a working directory ancestor has no effect It looks like you are on the blast25 branch, try checking out default then merging the new blast25 branch. It should be a fast-forward merge (just the two commits). Then you can push default back to galaxy-central on bitbucket. I think. Importing the diffs gives merge error Are you using a hg command here? What I meant for the Plan B was just using the Unix command patch to apply my changes by hand to the default branch. Then commit it under your name. It won't be associated with my bitbucket account, but it should just work. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Ignoring white space differences in test output
On Tue, Apr 5, 2011 at 10:58 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I'd like to write a tool test where the sample output and the generated output are the same except for white space. i.e. running diff with the -w switch reports no differences. Is this possible at the moment with the Galaxy test framework? I'm hoping for a test output attribute. I don't see anything like this mentioned here: https://bitbucket.org/galaxy/galaxy-central/wiki/WritingTests Thanks, Peter I've filed an enhancement request for this, https://bitbucket.org/galaxy/galaxy-central/issue/510 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating BLAST+ wrapper
On Mon, Apr 4, 2011 at 6:07 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Apr 4, 2011 at 5:55 PM, Kanwei Li kan...@gmail.com wrote: Done the manual way Thanks. I was hoping to have finished this today, but it turns out there are some subtle but annoying changes to how BLAST records the hit id/name/description in their XML file which make the script to convert from XML to tabular unhappy. Hi Kanwei, I think I'm nearly done now, but I am still working on the blast25 branch you had trouble merging/transplanting from. I guess I will need to make a new clean branch from the galaxy tip for submission. Do you prefer several small atomic commits (which I understand is the recommend practice in git), or a few large commits (which seems quite common on the Galaxy hg history)? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Ratings on Tool Shed broken?
Hi all, I had wondered why none of the tools on the Galaxy Tool Shed had a rating, and had assumed it was just that so far no-one had bothered. I have a new theory, it doesn't work? I just viewed Brad's BAM to BigWig tool, clicked on Tool Actions then Rate Tool, and got: Server Error An error occurred. See the error logs for more information. (Turn debug on to display exception reports here) Can anyone reproduce this? I've tried a couple of other tools and the same thing happens. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Relative file path in Galaxy
On Wed, Apr 6, 2011 at 12:44 AM, Zhe Chen z...@lanl.gov wrote: Hi, My tool has a script reads a file in the same directory as the script. When I try to use relative path to read the file, it works by directly run it, but does not work when calling from Galaxy. Absolute path will work, but I want to know is there a way to do it using relative path. Can you give me some suggestion? myTool.pl use the following code the read the file in the same directory my $genusfiles = genus.txt; open (GENUSGP, $genusfiles) or die $genusfiles: $! \n; close(GENUSGP); Thanks I would expect that you can look at the first entry in argv which should give you the path to the script being run, myTool.pl, then use that to construct the path to genus.txt Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] another update problem
On Mon, Apr 11, 2011 at 7:26 PM, Ryan Golhar golha...@umdnj.edu wrote: I just updated my local galaxy instance using hg pull, and the history items are no longer expandable. What happened? Which browser(s) are affected? I recall reporting some issues with Internet Explorer only and the history peep view not working. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Getting tool dirpath in a Python code file
On Thu, Apr 14, 2011 at 2:08 PM, Leandro Hermida soft...@leandrohermida.com wrote: Hi, I have a tool with a code file=my_script.py/ tag and in that code file I'm trying to get the tool dirpath where that script and the tool XML exist. I've tried: os.path.abspath(os.path.dirname(sys.argv[0])) os.path.abspath(os.path.dirname(__file__)) And both don't work as expected. Is there a galaxy class I could import which will have the tool directory path? regards, Leandro For standard Python tools in Galaxy, I'm using os.path.split(sys.argv[0])[0] to get the path, which on reflection probably should be written as os.path.dirname(sys.argv[0]) as you suggest. What do __file__ and sys.argv[0] give you? The simplest way to debug this is to add a print statement, since Galaxy will show the stdout. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] NLStradamus wrapper (for nuclear localization signals)
Hi all, I just wanted to mention (to avoid duplicate work) that I have a working Galaxy wrapper for NLStradamus v1.6, a tool for prediction of nuclear localization signals (NLSs) from a protein FASTA file. The wrapper does a little reformatting of the output to get a clean tabular file for use in Galaxy. I would release this to the Galaxy Tool Shed now, but the tool's author Alex Nguyen Ba tells me he hopes to have an update released fairly soon which should include a native tabular output - probably with a richer column structure. So rather than having to change the tool output and potentially break workflows, I plan to wait till then. However, if anyone wants the wrapper before then, drop me an email. Regards, Peter References: A. N. Nguyen Ba, A. Pogoutse, N. Provart, A. M. Moses. NLStradamus: a simple Hidden Markov Model for nuclear localization signal prediction. BMC Bioinformatics. 2009 Jun 29;10(1):202. http://www.moseslab.csb.utoronto.ca/NLStradamus ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Getting tool dirpath in a Python code file
On Thu, Apr 14, 2011 at 2:56 PM, Leandro Hermida soft...@leandrohermida.com wrote: On Thu, Apr 14, 2011 at 3:17 PM, Peter Cock p.j.a.c...@googlemail.com wrote: For standard Python tools in Galaxy, I'm using os.path.split(sys.argv[0])[0] to get the path, which on reflection probably should be written as os.path.dirname(sys.argv[0]) as you suggest. What do __file__ and sys.argv[0] give you? The simplest way to debug this is to add a print statement, since Galaxy will show the stdout. Hi Peter, __file__ throws an error: global name '__file__' is not defined I guess the script is being loaded as a string, and run with eval(...) or something like that. It would also explain why sys.argv[0] would be one of the Galaxy script files. os.path.abspath(os.path.dirname(sys.argv[0])) gives me /path/to/galaxy/scripts directory which is two levels up from what the tool directory I want for example /path/to/galaxy/tools/mytool So combine that with ../tools/mytool/ and you're done? OK, you have to know the name of the folder your tool *should* be in... so not a perfect solution. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Setting min/max limit on repeat tag in tool XML file
Hi all, I'm working on a tool where the repeat tag looks like an elegant solution - however I want to limit the number of repeats to be 1, 2 or 3. i.e. I want to set min=1 and max=3 attributes for the repeat, based on analogy to the param tag. Is this possible, but not documented here? https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax In terms of the GUI, I'd like the page to preload with as many entries as the min setting (zero by default), and disable the add button once the max setting is reach (default no limit). (Alternatively you could validate the max/min when the form is submitted by clicking the execute button, but that seems less user friendly.) If this isn't already possible, would it be generally useful to add this? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Setting min/max limit on repeat tag in tool XML file
On Tue, Apr 19, 2011 at 1:46 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Peter, Yes, setting min and max values in the repeat tag should work as you need. As you point out, it is probably not well documented, so please let us know if it isn't working as you imagine it should. Thanks for using Galaxy, Dan Hi Dan, Excellent - that does pretty much what I was expecting. Thanks for updating the wiki page. It would be a nice refinement to hide the Remove buttons when at the minimum, and hide the Add button when at the maximum - but not essential. What I need to work out now is how to use repeat parameters in the unit tests. There seems to be examples in tools/mutation/visualize.xml and tools/plotting/xy_plot.xml which only use the first repeat... Also, can I access the repeat number for default values - e.g. I'd like to have a caption parameter defaulting to Set 1, Set 2 etc. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] BLAST+ HTML output broken
Hi Kanwei, I noticed that the BLAST+ output options for HTML are not working, a side effect of my recent refactoring. Could you transplant/merge this fix and test please: https://bitbucket.org/peterjc/galaxy-central/changeset/e1e6f92e07bf https://bitbucket.org/peterjc/galaxy-central/changeset/c60551f7d13c These are the only commits on this new branch from the default: https://bitbucket.org/peterjc/galaxy-central/src/blast_2011_04_19 Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Plotting tools - best output format? PDF, PNG, ...
Hi all, I'm working on a simple plotting tool, and currently I am using PDF output since this is vector based (so it can be rescaled without loss of quality) and can be edited easily enough for use in publications. However, at least on my OS/Browser, Galaxy does not show the PDF files in situ, the eye button acts like a download link. Is this a bug? If I switch to making a PNG file, then clicking on the eye button shows the image directly in the center panel (nice), but PNG is not a great format for detailed figures or printing. Looking over the tools which come with Galaxy under tools/plotting/*.xml all use PDF or PNG. How about SVG? Don't most of the mainline browsers have reasonably good SVG support built in (possibly via a plugin)? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Plotting tools - best output format? PDF, PNG, ...
On Wed, Apr 20, 2011 at 10:07 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I'm working on a simple plotting tool, and currently I am using PDF output since this is vector based (so it can be rescaled without loss of quality) and can be edited easily enough for use in publications. However, at least on my OS/Browser, Galaxy does not show the PDF files in situ, the eye button acts like a download link. Is this a bug? Looks like a Firefox issue/setting - false alarm: On our local Galaxy, with a PDF entry in the history, and as a control http://nar.oxfordjournals.org/content/39/7/2492.full.pdf+html which should show a PDF embedded on the left: Windows XP, Internet Explorer 7.05730.13 - NAR shows PDF on left, good - Clicking on the eye, shows the PDF in the central panel, good. - Clicking on the disk, saves the PDF, good. Windows XP, Chrome 10.0.648.204 - NAR shows PDF on left, good - Clicking on the eye, shows the PDF in the central panel, good. - Clicking on the disk, saves the PDF, good. Windows XP, Firefox 3.6.13 - NAR shows PDF on left, good - Clicking on the eye, shows the PDF in the central panel, good. - Clicking on the disk, saves the PDF, good. Linux CentOS, Firefox 3.6.13 - NAR downloads PDF, shows left pane empty, usable. - Clicking on the eye, downloads the PDF, usable. - Clicking on the disk, saves the PDF, good. Mac OS X 10.6, Firefox 4.0 - NAR downloads PDF, shows left pane empty, usable. - Clicking on the eye, downloads the PDF, usable. - Clicking on the disk, saves the PDF, good. Mac OS X 10.6, Safari 5.0.5 - NAR shows PDF on left, good - Clicking on the eye, shows the PDF in the central panel, good. - Clicking on the disk, saves the PDF, good. So it looks like Firefox can (probably dependent on the user's settings) download PDF files rather than show them in situ. That's fine, and not specific to Galaxy. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Plotting tools - best output format? PDF, PNG, ... SVG!
On Wed, Apr 20, 2011 at 4:30 PM, Pieter Neerincx pieter.neeri...@gmail.com wrote: Hi Peter, On Apr 20, 2011, at 11:07 AM, Peter Cock wrote: Hi all, ...cut... How about SVG? Don't most of the mainline browsers have reasonably good SVG support built in (possibly via a plugin)? Yep, SVG support is pretty good nowadays especially in FireFox and Opera. Safari, IE and Chrome have only partial support, but for simple charts without 3D filters it works fine. I've added SVG as a datatype to my Galaxy and this works great. You won't need a plugin; in fact the old Adobe SVG plugin is depreciated already for a few years now and no longer compatible with recent browser versions. I looked at SVG about a year ago, and was pretty impressed. I did run into some issues, particularly with links, and opted to use PNG files in the end. Since then we've finally dropped IE6, so hopefully SVG would be safe now. Are your changes to add SVG as a datatype to Galaxy public? I'd like to suggest that be merged to the trunk. The nice benefit of SVG in addition to not needing a plugin is that the image can scale with the surrounding text if you zoom in your browser (without loosing resolution). The disadvantage is that although my users know what a PDF file is and can process them for posters/manuscripts, most of them never heard of SVG and get stuck when it doesn't import into PowerPoint et al. :( Do they cope with PDF? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Plotting tools - best output format? PDF, PNG, ... SVG!
On Thu, Apr 21, 2011 at 10:50 AM, Pieter Neerincx pieter.neeri...@gmail.com wrote: Hi Peter, On Apr 20, 2011, at 5:47 PM, Peter Cock wrote: On Wed, Apr 20, 2011 at 4:30 PM, Pieter Neerincx pieter.neeri...@gmail.com wrote: Hi Peter, On Apr 20, 2011, at 11:07 AM, Peter Cock wrote: Hi all, ...cut... How about SVG? Don't most of the mainline browsers have reasonably good SVG support built in (possibly via a plugin)? Yep, SVG support is pretty good nowadays especially in FireFox and Opera. Safari, IE and Chrome have only partial support, but for simple charts without 3D filters it works fine. I've added SVG as a datatype to my Galaxy and this works great. You won't need a plugin; in fact the old Adobe SVG plugin is depreciated already for a few years now and no longer compatible with recent browser versions. I looked at SVG about a year ago, and was pretty impressed. I did run into some issues, particularly with links, and opted to use PNG files in the end. Since then we've finally dropped IE6, so hopefully SVG would be safe now. Are your changes to add SVG as a datatype to Galaxy public? I'd like to suggest that be merged to the trunk. I simply added this line to my datatypes_conf.xml: datatype extension=svg type=galaxy.datatypes.images:Image mimetype=image/svg+xml/ I didn't make a sniffer as I don't have a use case for users uploading SVGs, so I didn't have to create a Python class and SVG is just another instance of galaxy.datatypes.images:Image from a plain vanilla Galaxy. That makes sense if the sniffer is only used on upload. To make sure the client displays the SVG you may have to add the SVG mimetype to your web server's config too. Where this is defined may differ per linux distro. I use apache as a proxy and had to add image/svg+xml svg svgz to /etc/mime.types. Thanks for the details - it looks simple enough to add here too. If/when any SVG producing tools are added to Galaxy or the Galaxy Tool Shed, then it would be nice to have this done in the main repository. The nice benefit of SVG in addition to not needing a plugin is that the image can scale with the surrounding text if you zoom in your browser (without loosing resolution). The disadvantage is that although my users know what a PDF file is and can process them for posters/manuscripts, most of them never heard of SVG and get stuck when it doesn't import into PowerPoint et al. :( Do they cope with PDF? Well, they know how to view and print one :). Importing them directly into PowerPoint and Word is just as problematic though, so that usually involves opening the PDF in a different program and copy/paste or exporting to another format. Still a hassle, but they manage... Getting a PDF into PowerPoint or Word is easy on a Mac ;) But yes, point taken - even though a PDF is easier to work with than an SVG (or a postscript file), it can still be a hassle. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy tools' priority level (nice)
Hi all, Our Galaxy is running on a shared server which will sometimes be running other computationally demanding jobs (outside of Galaxy). In some cases I'd like these to have priority, perhaps by having Galaxy run the tool child processes at a nice level of 10 (say). Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScaling Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy tools' priority level (nice)
On Thu, Apr 21, 2011 at 5:43 PM, Assaf Gordon gor...@cshl.edu wrote: Hi Peter, Peter Cock wrote, On 04/21/2011 11:00 AM: Is there any built in way to control the Unix priority level (e.g. nice or ionice) used to run tasks? I don't see anything on here, but perhaps I'm looking in the wrong place: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/WebApplicationScaling I've been experimenting with 'nice' as well (for other servers), and it seems the consensus is that 'nice' is broken. Linus puts it in a colorful way here: https://lwn.net/Articles/418739/ [...] Seriously. Nobody _ever_ does nice make, unless they are seriously repressed beta-males (eg MIS people who get shouted at when they do system maintenance unless they hide in dark corners and don't get discovered). It just doesn't happen. I know nice isn't perfect, but in the case of the sys admin setting up Galaxy, we don't have the human laziness problem to overcome: We could ensure all job tasks get run with nice 10 automatically, without the Galaxy users having to do anything special. One recommended solution is to use cgroups, which can control CPU, Disk I/O, Memory usage and network load, as explained here: http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-and-projects.html With cgroups there will be no need to change anything in galaxy (just apply a cgroup to the galaxy user, or something similar). Yes, but I want to treat the Galaxy webserver differently from the compute jobs it launches. Unfortunately, cgroups requires a recent kernel, so it might not be applicable in your case. -gordon Are you using cgroups on your Galaxy? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Could anyone body tell me why our C program do not work in galaxy
2011/4/22 liyanhui607 liyanhui...@163.com: Dear Sir, We have a program writen in C lanuage. It can run nomally in linux environment. We plan to add it to galaxy, but it did not work. The C file after complie named shortest-path The shortest-path.xml is as follow: Could anybody tell me what is wrong ? why it does not work? Thank you very much! Best Wishes! Yan-Hui Li What error message do you get? Did you try the two changes I suggested (the same things Dan just suggested as well): https://bitbucket.org/galaxy/galaxy-central/issue/525 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Could anyone body tell me why our C program do not work in galaxy
2011/4/25 liyanhui607 liyanhui...@163.com: Thank you very much! After I have added it to the path, it can work now. Best Wishes! Yan-Hui Li Oh good. Peter P.S. Please keep the mailing list on replies. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Staged Method for cluster running SGE?
Hi all, So far we've been running our local Galaxy instance on a single machine, but I would like to be able to offload (some) jobs onto our local SGE cluster. I've been reading https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster Unfortunately in our setup the SGE cluster head node is a different machine to the Galaxy server, and they do not (currently) have a shared file system. Once on the cluster, the head node and the compute nodes do have a shared file system. Therefore we will need some way of copying input data from the Galaxy server to the cluster, running the job, and once the job is done, copying the results back to the Galaxy server. The Staged Method on the wiki sounds relevant, but appears to be for TORQUE only (via pbs_python), not any of the other back ends (via DRMAA). Have I overlooked anything on the Cluster wiki page? Has anyone attempted anything similar, and could you offer any guidance or tips? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Staged Method for cluster running SGE?
On Tue, Apr 26, 2011 at 12:10 PM, Sean Davis sdav...@mail.nih.gov wrote: On Tue, Apr 26, 2011 at 5:11 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, So far we've been running our local Galaxy instance on a single machine, but I would like to be able to offload (some) jobs onto our local SGE cluster. I've been reading https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster Unfortunately in our setup the SGE cluster head node is a different machine to the Galaxy server, and they do not (currently) have a shared file system. Once on the cluster, the head node and the compute nodes do have a shared file system. Therefore we will need some way of copying input data from the Galaxy server to the cluster, running the job, and once the job is done, copying the results back to the Galaxy server. The Staged Method on the wiki sounds relevant, but appears to be for TORQUE only (via pbs_python), not any of the other back ends (via DRMAA). Have I overlooked anything on the Cluster wiki page? Has anyone attempted anything similar, and could you offer any guidance or tips? Hi, Peter. You might consider setting up a separate queue for SGE jobs. Then, you could specify a prolog and epilog script that will copy files from the galaxy machine into the cluster (in the prolog) and back to galaxy (in the epilog). I see - the prolog and epilog scripts are per SGE queue, so in order to have Galaxy specific scripts, the simplest solution is a Galaxy specific queue. This assumes that there is a way to map from one file system to the other, but for Galaxy, that is probably the case (galaxy files on the galaxy server are under the galaxy instance and galaxy files on the cluster will probably be run as a single user in that home directory). We have a user account galaxy on the Galaxy Server, and it would make sense to have a matching user account galaxy on the Cluster which would submit the SGE jobs and own their data files. I have not done this myself, but the advantage to using prolog and epilog scripts is that galaxy jobs then do not need any special configuration--all the work is done transparently by SGE. Sean Thanks for the pointers - I have some reading ahead of me... Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Reciprocal Best Hits (RBH) from BLAST tabular output
Hi all, I mentioned just over a month ago that I had written a Galaxy tool to do find Reciprocal Best Hits (RBH) from BLAST tabular output or similar (e.g. Bill Pearson's FASTA tabular output). http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-March/004799.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-April/004825.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-April/004829.html The only problem I had was setting default columns, which I consider to be a minor bug in Galaxy: https://bitbucket.org/galaxy/galaxy-central/issue/507 Regardless of this small usability issue, would the Galaxy team like to add this tool to the main distribution (to sit along with my BLAST+ wrappers), or should I submit it to the tool shed? If you want to merge it, the following change set should suffice: https://bitbucket.org/peterjc/galaxy-central/changeset/198bf927ca30 I made a subsequent change set to test giving column defaults as value=c2 rather than value=c2 which makes no difference, but if this is how default column values are meant to be given, take this change too: https://bitbucket.org/peterjc/galaxy-central/changeset/198bf927ca30 Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] quick question: how can i supply the user's email address to a tool?
On Fri, May 6, 2011 at 8:18 AM, Hans-Rudolf Hotz h...@fmi.ch wrote: Hi Ed You can use the variable $userEmail in the 'command' tag of the tool definition file Regards, Hans That should probably be listed on the wiki, https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax Are there any other similar variables worth knowing about? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] options from file
On Fri, May 6, 2011 at 10:17 AM, SHAUN WEBB swe...@staffmail.ed.ac.uk wrote: Hi, I have created a tool that will fetch sequences for selected IDs from a tabular file containing multiple IDs and additional info. I want the tool config to scan the first column of the tab file for IDs and provide the user with a selection box where they can select a single ID or multiple IDs and get output for all selected. The following method does this: param name=tabfile type=data format=tabular label=ID File/ param name=selection type=select multiple=true accept_default=true label=ID options from_dataset=tabfile column name=name index=0/ column name=value index=0/ /options /param The issue is, if the top file in my history is a SAM file containing ~30,000 IDs in the first column the tool initially attempts to load these all in to the selection box and effectively crashes my local instance. I only want to use this on tab files that ill have ~100 IDs at most. Maybe the options from_dataset=tabfile tag could have a max setting? e.g. options from_dataset=tabfile max=100 could load just the first 100 entries in the tabular file. That seems much more general than the new filetype idea. It could have a default max value which should useful. Thinking of the example of a tabular file of gene IDs for an organism, you might well want 20 to 30 thousand entries. Since Galaxy puts a search function on the selection, the UI should be OK. Its just the performance we need to worry about. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Megablast GIs
On Saturday, May 7, 2011, drho...@uark.edu wrote: Thanks Peter, I debated which to post to. This is just using the Megablast tool in Galaxy. My IT people are not Biologists. They had this working fine and some tweak has screwed up the GI column in the results file. It should be the same NR and HTGS datafiles that we had before. So you recommend that I should just have them reinstall the databases and make sure to use the correct switches? You don't know what tweak broke things? You've got at least three factors, (1) Galaxy and the wrapper, but I don't see how an update of these would have affected the IDs. (2) The BLAST command line tool (blast all in this case), perhaps it was updated and this has altered the output? (3) The BLAST database(s) were updated (or rebuilt if you have local databases, in which case it could have been done with a newer version of the formatdb tool). If you are using NR does you have it regularly updated from the NCBI FTP site? Hopefully your admin team will have some sort of log of system updates that might help work out what happened. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy administrators - admin_users setting
Hi all, To make someone an administrator on a local Galaxy install, do I just need to add their email (login) to the comma separated setting admin_users in universe_wsgi.ini? https://bitbucket.org/galaxy/galaxy-central/wiki/Admin/AdminInterface I have this working on one server, but it doesn't seem to have any effect on a second server. Both are now running the current release (changeset 50e249442c5a). Is there some other setting needed to enable the admin interface? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy administrators - admin_users setting
On Mon, May 9, 2011 at 12:48 PM, Hans-Rudolf Hotz h...@fmi.ch wrote: Hi Peter First,the obvious question: Have you restarted galaxy? Yes, and I tried hard reloads in two different browsers (both of which are working on the primary server). Second: I vaguely remember a similar problem, and if I remember correctly, the problem we had was the space between the comma and the next e-mail address. Hence, make sure you write the line like this: admin_users = f...@bar.com,f...@bar.com,f...@bar.com (ie: without any blank spaces) I hope this helps, Hans I'll double check that. As an aside, it would be interesting to check if the comparison is done case insensitively or not (emails and domain names are case insensitive). Fingers crossed... thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] param type=float optional=true doesn't work
On Thu, May 12, 2011 at 6:03 PM, Leandro Hermida wrote: Hi, how do you make float parameter types fully optional? optional=true doesn't work. -Leandro Sadly this is a known bug I reported some time ago: https://bitbucket.org/galaxy/galaxy-central/issue/403 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Reload .loc files without restarting Galaxy system?
On Wed, Apr 6, 2011 at 10:30 PM, Luobin Yang yangl...@isu.edu wrote: Hi, The reload a tool's configuration menu avoids restarting Galaxy system when a tool's web interface changed, but it doesn't seem to reload the .loc files that a tool's XML file uses. I am wondering if it's possible to reload the .loc file also? Thanks, Luobin I'd like to be able to do this too. I filed an enhancement bug: https://bitbucket.org/galaxy/galaxy-central/issue/538 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Wrapper generator.
On Fri, May 13, 2011 at 2:12 PM, J. F. J. Laros j.f.j.la...@lumc.nl wrote: Hi Pieter, Thanks for the reply, but from the looks of it, this is a way to incorporate webservice clients into galaxy. What I want is slightly different. I'm talking about adding a normal command line tool (an aligner for example) but I'm not willing to write a wrapper each time (the wrapper for BWA for example is 327 lines). If a computer readable description of the interface exists, this description can be used to generate a python wrapper. I mentioned WSDL because this also gives a description of an interface, but any equivalent description would of course be fine. Ideally, such a description is maintained by the developer of the tool in question. With kind regards, Jeroen. How many tools can you name that come with such a machine readable file describing their command line interface? I can think of only one - the EMBOSS tool suite and their ACD files (used internally by EMBOSS to generate the command line help, documentation, and do the command line option parsing). Unfortunately I don't think there is an easy answer - even if wrapping a tool where all the input and output formats are already supported in Galaxy. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Select first/last N rows from grouped tabular files (e.g. top BLAST hits)
Hi all, I'm wondering if the following task can be done in Galaxy with the standard tools. The specific example is selecting the top (e.g. 3) match sequences for each blast query, but I see this problem as much more general than a Select top BLAST hits tool. I want to select the first few (e.g. 1) rows of each group in a tabular file, where the group criteria is having certain columns equal (e.g. the first 2). e.g. Tabular BLAST output has columns of query ID, match ID, etc. queryA match1 ... queryA match2 ... queryA match2 ... queryA match3 ... queryA match4 ... queryA match4 ... queryA match4 ... queryB match5 ... queryB match5 ... queryC match6 ... queryC match7 ... In this example, some of my queries have more than one HSP per match (more than one line with the same first two columns). If I group on the first two columns, the groups are: queryA match1 ... queryA match2 ... queryA match2 ... queryA match3 ... queryA match4 ... queryA match4 ... queryA match4 ... queryB match5 ... queryB match5 ... queryC match6 ... queryC match7 ... If I then take the first row in each group, that gives me just the first HSP for each query+match combination. queryA match1 ... queryA match2 ... queryA match3 ... queryA match4 ... queryB match5 ... queryC match6 ... queryC match7 ... If for example I wanted only the top 3 matches for each query, I could repeat the proposed tool one more time but with different settings - this time grouping on the first column only: queryA match1 ... queryA match2 ... queryA match3 ... queryB match5 ... queryC match6 ... queryC match7 ... I hope I've conveyed the idea here. The existing tools Select first lines from a dataset and Select last lines from a dataset are related, but do this at the file level. Does this make sense? Does it seem like a useful tool to write if there isn't anything like this already present? Or might it be simpler to just write a Select top BLAST hits tool? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Published Workflows menu location
Hi all, I recently started trying out the Share workflow functionality on our local Galaxy, so that I could make available some general interest workflows to all our users, rather than sharing them with individuals one by one. However, once published, it was not easy to find them! I eventually found them under the Shared Data drop down menu entry Published Workflows (yet Shared workflows are not on this menu). The captions Shared and Published seem to be used inconsistently. Maybe the top level Shared Data menu should be renamed to Shared Resources or Published Resources? To me, workflows are not data, and so should not be under Shared Data. Rather, I expected them to be under the Workflow menu. Is the separation because in order to run or edit a Published Workflows, you must first import it? Perhaps a simple solution would be a note and link at the bottom of the main Workflow page directing people to the Published Workflows page under the Shared Data drop down menu. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Reciprocal Best Hits (RBH) from BLAST tabular output
On Wed, May 18, 2011 at 3:54 AM, Kanwei Li kan...@gmail.com wrote: Hi Peter, I think the tool shed would be appropriate for this... -K OK - I'll do that. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Select first/last N rows from grouped tabular files (e.g. top BLAST hits)
On Tue, May 17, 2011 at 5:30 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, I'm wondering if the following task can be done in Galaxy with the standard tools. The specific example is selecting the top (e.g. 3) match sequences for each blast query, but I see this problem as much more general than a Select top BLAST hits tool. ... Does this make sense? Does it seem like a useful tool to write if there isn't anything like this already present? Or might it be simpler to just write a Select top BLAST hits tool? While I still think the above task could be useful in general, I am now considering a general BLAST filter tool to offer this and some other commonly used filters like a minimum coverage threshold (which is possible with a filter on the extended tabular output, but not trivial). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Select first/last N rows from grouped tabular files (e.g. top BLAST hits)
On Thu, May 19, 2011 at 7:33 PM, madduri gal...@ci.uchicago.edu wrote: I wonder if somebody can give me more context around this issue.. On 3rd May I emailed IBX about their Galaxy install and one of the (in house) tools mentioned on the workflow image here: https://ibi.uchicago.edu/resources/galaxy/index.html I recognised the NCBI BLAST+ tools but the Filter Top Blast Results tool was new to me, and asked what it did and if it or any the other IBX tools would be available at the Galaxy Tool Shed: http://community.g2.bx.psu.edu/ I had a reply from Alex Rodriguez (iBi/CI University of Chicago) that they haven't put any of the wrappers on the Galaxy tool shed yet as they are still being worked on. The IBI system assigned the number [Galaxy #13918]. This thread Select first/last N rows from grouped tabular files (e.g. top BLAST hits) could have similarities to the IBI Filter Top Blast Results tool, so I forwarded the email to the IBI galaxy email address to encourage you (e.g. Alex) to comment on the thread. The IBI system assigned the number [Galaxy #14246]. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tools: dynamic check of file type?
2011/5/20 Louise-Amélie Schmitt louise-amelie.schm...@embl.de: Well I'm still missing something... Is there a way to use the conditional tag sets in the tool's xml with it? To suppress the unnecessary options for a given file type? or more than one? It sounds like you want to use the conditional and param tags, but rather than responding to another parameter directly (e.g. a select), you want to look at an input file parameter's file type? I don't know if that is possible, but I can thing of a couple of examples where I could use that functionality. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] egg install failure: docutils, GeneTrack
On Mon, May 16, 2011 at 8:59 PM, Spunde, Alex aspu...@illumina.com wrote: Same thing: [aspunde@ilmnhw-biolin10 galaxy-dist]$ python ./scripts/fetch_eggs.py Fetched http://eggs.g2.bx.psu.edu/Babel/Babel-0.9.4-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/docutils/docutils-0.7-py2.6.egg Fetched http://eggs.g2.bx.psu.edu/pysqlite/pysqlite-2.5.6_3.6.17_static-py2.6-linux-x86_64-ucs2.egg Fetched http://eggs.g2.bx.psu.edu/GeneTrack/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.6.egg One or more of the python eggs necessary to run Galaxy couldn't be downloaded automatically. You can try building them by hand (all at once) with: python scripts/scramble.py Or individually: python scripts/scramble.py Babel python scripts/scramble.py docutils python scripts/scramble.py GeneTrack I had a similar issue which was down to the file permissions - sudo fixed it, and I redid the chown afterwards. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Filter data and cut column bugs
On Mon, May 23, 2011 at 3:09 PM, Anton Nekrutenko an...@bx.psu.edu wrote: Dear Peter: Yes, that would help. One possibility would be to have all new bugs CC'd to the dev mailing list? Not sure if everyone here would like that or not... But, you patches are definitely not unnoticed. We'll apply them (likely at the conference) and we are also in the process of devising a strategy for doing so quickly and consistently. That sounds good :) See you on Tuesday. a. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Display column headers in data_column parameter
On Tue, May 24, 2011 at 3:12 PM, Johannes Eichner johannes.eich...@uni-tuebingen.de wrote: Dear Galaxy-Dev Team, is it possible to display the headers of the columns which can be selected in a data_column parameter? Currently, only the column index (e.g., c1, c2, etc.) is displayed in Galaxy. If it would be possible to additionally display the column headers, the selection of the desired columns would be easier for the user. Is there a way to do this using the current version of Galaxy? Kind regards, Johannes I completely agree that in general the UI for column parameter isn't as user friendly as it could be. The filter data UI is even worse for non-programmers, but due to its complexity and flexibility, that is harder to improve too. This was one of the things I wanted to bring up with the Galaxy team at the conference (starting tonight). Unfortunately due to the Grimsvotn Volcano's cloud of ash I won't be there. In some cases yes, Galaxy knows the column meaning. In many of my data files there is a # header line with tab separated column names, which could be detected and shown. Given there may or not be suitable column meta data, what I was picturing was an expanded column picking dialogue that could show the first few rows of data in each column (not using a simple HTML widget anymore), or simply the first value (less drastic change). So, rather than having as now something like: Pick column(s): [ ] c1 [ ] c2 [ ] c3 [ ] c4 [ ] c5 If Galaxy has names in the column meta data you might have Pick columns(s): [ ] c1 - Identifier [ ] c2 - Start [ ] c3 - End [ ] c4 - Strand [ ] c5 - Sequence Or, showing the first data entry: Pick columns(s): [ ] c1, e.g. gi|12345678 [ ] c2, e.g. 123 [ ] c3, e.g. 456 [ ] c4, e.g. + [ ] c4, e.g. ACGTACTGT... In the final example I envision truncating very long values with an ellipsis. Both the above proposals don't require drastic changes to the HTML widget and layout. I guess we should file an enhancement issue on bitbucket: https://bitbucket.org/galaxy/galaxy-central/issues/new Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy BLAST XML to tabular fix
On Wed, May 18, 2011 at 6:40 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Kanwei, Could you merge this branch or transplant this fix please? https://bitbucket.org/peterjc/galaxy-central/changeset/3c0b903c86a9 https://bitbucket.org/peterjc/galaxy-central/src/blast_2011_05_18 New branch, blast_2011_05_18 There is no change to the logic, just to the assert statement where the sanity test didn't consider doubly masked sequences. Should I also update the XML wrapper's version number as well? Thanks, Peter Hi Kanwei, Did this email reach you? Let me know if you want a new unit test to go with it and/or to bump the tool's version number. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy BLAST XML to tabular fix
On Wed, May 25, 2011 at 1:10 PM, Kanwei Li kan...@gmail.com wrote: Sorry, it slipped by. Committed! No problem - thank you, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Is dynamic associated information per dataset possible?
On Thu, May 26, 2011 at 10:34 AM, Roman Valls brainst...@nopcode.org wrote: Nate, is it just me or all the .png links on that wikipage are broken/missing ? :-S I've noticed this on other pages, I think some stuff got moved a while back. They just need the folder name prefixed... Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] stdout and stderr
On Thu, May 26, 2011 at 1:36 PM, shashi shekhar meshash...@gmail.com wrote: Hi all, I am using one tool which gives stdout and stderr . it is showing red color (job failed) in browser.How can i resolve such type of problem. Regards shashi shekhar See issue 325, https://bitbucket.org/galaxy/galaxy-central/issue/325/ Until that is fixed, you'll probably need a wrapper script, there is an example of this in tools/ncbi_blast_plus/hide_stderr.py Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] The powers of Cheetah, how?
On Thu, May 26, 2011 at 1:22 PM, Kempenaar, M (med) m.kempen...@med.umcg.nl wrote: Hi list, I've been working on a single Galaxy tool for a while now, and I'm running into all sorts of difficulties that can probably all be fixed using Cheetah. However I've been trying several ways of utilizing the (enormous) powers of Cheetah but so far I'm not successful. I'll try to ask my questions as clear as possible, please let me know if anything is unclear. In my tool I use the following input parameter: --- param name=report_options type=select multiple=True display=checkboxes option value=overviewphyla distribution/option option value=phyladendphyla dendograms/option option value=heatmapheatmap/option option value=dendogramdendogram/option option value=pcaprincipal component analysis/option option value=mdsmulti dimensional scaling/option /param --- Each of these options corresponds to a function in my R script, thus I want to do an if-else if-else to execute these functions that the user selected (note that multiple=True). Now the problem is, I also use R Sweave (sweaving R output (text/graphics) into LaTeX) to generate a nice looking PDF report, and since that's not a 'real' programming/ scripting language, I can't for instance split the string (as I used to do in R) and check for the boxes a user selected, so my questions are: I planned on using Cheetah to include sections of Sweave LaTeX code for each box checked by the user, perfectly doable with an #if-#elif you'd think, however since the $report_options contains a comma-separated single string, I would need to split (or use regular expressions) to be able to do such an #if-#elif like: #if heatmap in $report_options.split(,) results in: NotFound: cannot find 'split' while searching for 'split' Try changing this: #if heatmap in $report_options.split(,) to: #if heatmap in str($report_options).split(,) Although $report_options acts a bit like a Python string, it isn't one, and lacks lots of useful string methods. We can turn it into a string using the Python str function. Q: Followup from previous question, where within the sections of my tool XML file can I actually use Cheetah? This of course has to do with the order in which a tool is executed. I believe only within the command.../command tag. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] stdout and stderr
On Thu, May 26, 2011 at 2:42 PM, Bossers, Alex alex.boss...@wur.nl wrote: PS: 2 is STDERR and 1 is STDOUT. One could do a redirect of STDERR to STDOUT and STDOUT to a logfile. You won’t get a red box though when errors occur! Right - if you need to detect an error state via the return code, currently you have to do this with a wrapper script. If you don't ever expect any errors, then you can treat stderr as an output, or just redirect to to /dev/nul - but I don't like that idea - silent failures are bad. I regard fixing issue 325 as one of the top priorities in Galaxy, but in the short term I'd settle for treating any output on stderr OR a non-zero return code as a failure. i.e. Something like this patch, even if more work is needed for PBS jobs: https://bitbucket.org/galaxy/galaxy-central/issue/325/#comment-331776 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Display column headers in data_column parameter
On Thu, May 26, 2011 at 3:45 PM, Johannes Eichner johannes.eich...@uni-tuebingen.de wrote: Hi Peter, I defined an additional metadata element for the column names analogously to the column_types element in the module galaxy_basedir/lib/galaxy/tools/datatypes/tabular.py: MetadataElement( name=column_names, default=[], desc=Column names, param=metadata.ColumnTypesParameter, readonly=True, visible=False, no_value=[] ) Then I read the column names in the function set.meta and saved them in the new metadata element: dataset.metadata.column_names = column_names Do you look for a tab separated # header line at the start of the file for these names? Finally, I changed a line of code in the function get.column.list in galaxy_basedir/lib/galaxy/tools/parameters/basic.py, such that Galaxy displays the column names in the data_column parameter: original line: column_list.append( str( i + 1 ) ) changed line: column_list.append( str( i + 1 ) + : + dataset.metadata.column_names[i]) ) Kind regards, Johannes From a Python style point of view, I'd use string formatting here: column_list.append(%i: %s % (i+1, dataset.metadata.column_names[i])) Do you have a branch on bitbucket for this, or a patch? I'd like to try it. You could attach a patch to issue 554: https://bitbucket.org/galaxy/galaxy-central/issue/554 Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Filter data and cut column bugs
On Mon, May 23, 2011 at 3:34 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, May 23, 2011 at 3:09 PM, Anton Nekrutenko an...@bx.psu.edu wrote: Dear Peter: Yes, that would help. One possibility would be to have all new bugs CC'd to the dev mailing list? Not sure if everyone here would like that or not... But, you patches are definitely not unnoticed. We'll apply them (likely at the conference) ... Kanwei has just applied the fix for issues 535 and 537, thanks! https://bitbucket.org/galaxy/galaxy-central/issue/535/ Filter data on any column tool complains about hash comment lines https://bitbucket.org/galaxy/galaxy-central/issue/537/ Filter data on any column tool casts unused columns That leaves these two pending, https://bitbucket.org/galaxy/galaxy-central/issue/534/ Cut column tool messes up # header lines https://bitbucket.org/galaxy/galaxy-central/issue/536/ Filter data on any column tools shows nonsense % vs total on stdout While I'm looking over minor bugs where I submitted a patch, the following should also be fairly quick to review: FASTQ to Tabular tool doesn't accept plain FASTQ https://bitbucket.org/galaxy/galaxy-central/issue/436/ Conditional does not work with scripts in different path https://bitbucket.org/galaxy/galaxy-central/issue/159/ Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Implemented adding tools via admin console
On Fri, May 27, 2011 at 11:21 AM, Vossen, Bodo bodo.vos...@mpi-bn.mpg.de wrote: Hi all, I've implemented functionality for adding tools via the admin menu. That sounds very useful in principle, although perhaps the less flexible alternative of reloading tool_conf.xml would achieve the same aim? ... Then it is checked if script and XML are compatible by checking if they have the same number of arguments. This seems very fragile, and having looked at the code for detecting the number of arguments in a Python script it looks like it will get it wrong in many situations. Even for something quite simple like this: #sys.argv[0] is the script file itself assert len(sys,argv)==4, Expect 3 arguments arg1, arg2, arg3 = sys.argv[1:] Also, what about conditional code like this? assert len(sys.argv)==4, Expect 3 arguments if sys.argv[1] == db: database = sys.argv[2] filename = None else: database = None filename = sys.argv[2] out_filename = sys.argv[3] If I have read your get_parameter_number code right, it would claim there are four arguments since there are four lines of the basic form variable = sys.argv[i] Also, it cannot possibly work when there are a varying number of arguments, for example a repeat argument. If you want a test case, my Venn Diagram tool on the Tool Shed should be relevant (its a python script). If you have any comments or question please do not hesitate to contact me, I'm open for every kind of critics. I would remove the attempt to check the number of arguments. It is practically impossible to get this right in all cases, so I don't think it is worth doing. Having valid script/xml combinations rejected would be quite annoying. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Implemented adding tools via admin console
On Mon, May 30, 2011 at 10:42 PM, Bossers, Alex wrote: Peter, if you were just interested in having a dynamic refresh it was posted long time ago and we have it operational. It comes as a tool in your tool list and refreshes all from the tool_conf.xml. I'd like to see something like this functionality built into Galaxy, on the admin page next to reload a single tool. Having an extra standard tool (if world visible) is not an ideal solution. If that is what you meant? I'll have to try it and see. Not the loc filtes though! Reloading loc files is also on my wish list :) As far as i remember it doesn't kill running jobs. Good - because otherwise you might as well restart Galaxy. It requires two lines of code in galaxy source and the addition of a tool to the tools section. I pasted it from my archive below. But it is quite self explanatory. Alex Thanks, now I have two solutions to play with ;) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Dynamic default values
Dear all, I was thinking about how to improve some of my tools and wondered if there is any way to have dynamic default values for parameters (depending on other parameters). Obviously this won't make sense in all cases, but as a specific example, consider my Venn Diagram tool: https://bitbucket.org/peterjc/galaxy-central/src/9f66c5d1fca8/tools/plotting/venn_list.xml I have separate parameters for each dataset (file input) and its caption (text input). Like so: param name=set type=data format=tabular,fasta,fastq,sff label=Members of set help=Tabular file (uses column one), FASTA, FASTQ or SFF file./ param name=lab size=30 type=text value=Group label=Caption for set/ Is something like this possible (or generally useful?) param name=set type=data format=tabular,fasta,fastq,sff label=Members of set help=Tabular file (uses column one), FASTA, FASTQ or SFF file./ param name=lab size=30 type=text value=$set.name label=Caption for set/ i.e. The default value of the label text parameter lab would be the human friendly name of the associated data file parameter set. I'd expect changing the file to also change the text box. I have thought of a workaround, but not yet tried it: Use the empty string as the default, and use an if statement in the Cheetah template for constructing the command line string. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] The new hg based Galaxy Tool Shed
On Wed, Jun 1, 2011 at 3:22 PM, Nate Coraor n...@bx.psu.edu wrote: Hi Peter, Greg will probably reply, but I'll throw in my $0.02 as well. Great - but with your answers you've triggered more questions ;) Peter Cock wrote: Hi Greg et al, I've just been looking over your slides from last week about the new 'Galaxy Tool Shed', which are posted online here: http://wiki.g2.bx.psu.edu/GCC2011 http://wiki.g2.bx.psu.edu/GCC2011?action=AttachFiledo=gettarget=GalaxyToolShed.pdf They talk about how you will be tracking individual tools in hg repositories. I can see two ways this might work: (1) Each of these tool specific repositories (or branches if you just make one repository for each tool owner) would be a full fork of the Galaxy code base. This allows in principle tools to include changes to core functionality (but that seems dangerous due to potential merge clashes), and any existing tool contributor's pre-existing hg forks on bitbucket might be reused. The tool shed isn't really intended for framework changes - I would suggest keeping these as bitbucket forks, although it would certainly be good if we had a way to locate the list of such forks centrally. Well, as long as the repository is created by forking on bitbucket, then the link existing in the bitbucket web interface. https://bitbucket.org/galaxy/galaxy-central/descendants (2) Each of these tool specific repositories would ONLY track the tool specific files you'd add to Galaxy to install the tool. So, typically there would be an XML file, perhaps a wrapper script, maybe a sample loc file, and a plain text readme file. I'm guessing you've gone for something along the lines of idea (2), but I Yep. It did seem the most likely route. would love to hear more about how this will all work. e.g. Where would the tool shed repositories be hosted, and would tool authors use hg to work with them, or something like the current web based tool upload? They're hosted here, and you can check them out and work with them locally as you do the Galaxy source itself, or use the new web-based upload to upload individual files or tarballs. Have a look at the test instance of the next-gen toolshed here if you'd like to see how it works: http://testtoolshed.g2.bx.psu.edu/ Please feel free to use this as a sandbox and report any issues you find. I see the existing usernames and passwords from the old Tool Shed were transferred - that makes life easier. And it lists the hg information, e.g. hg clone http://pete...@testtoolshed.g2.bx.psu.edu/repos/peterjc/venn_list hg clone http://pete...@testtoolshed.g2.bx.psu.edu/repos/peterjc/tmhmm_and_signalp What happens with branches? Would the Tool Shed just show the default branch? That seems best for a simple UI. I have a query regarding the way the tools are shown in tables and the version column, which shows a changeset and revision number. According to Greg's slides (slide #10, titled Simpler tool versioning which seems ironic to me), the old numerical version is still there in the XML - and I'd prefer to see that. How about having both shown (two columns, perhaps call them Public version and hg version or hg revision). With regards to the planned installation functionality, what happens when a tool repository (aka Tool Suite in the old model) contains several XML wrappers - would you be able to choose which are wanted? The use case I have here is when several tools share some common dependency (which should be tracked in a single repository), and were therefore useful to bundle together as a suite, but where not all the tools will be of global interest (e.g. My TMHMM, SignalP, etc suite). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] The new hg based Galaxy Tool Shed
On Wed, Jun 1, 2011 at 4:22 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Peter - I finally got a chance to jump in - see my inline comments... Hi :) What happens with branches? Would the Tool Shed just show the default branch? That seems best for a simple UI. Some of the branching details are yet to be worked out, but forks are easy because repository urls include the unique username of the Galaxy user. Well, yes and no - as long as there are competing versions of a Galaxy tool (e.g. from an original author and a fork by a second author), and they use the same ID in their XML, you have a clash. This will have to be considered in the (automated) install interface. i.e. In general, when installing or updating any tool, there may be existing versions of some components already present. In fact two completely unrelated tools could even have the same XML ID by accident. I have a query regarding the way the tools are shown in tables and the version column, which shows a changeset and revision number. According to Greg's slides (slide #10, titled Simpler tool versioning which seems ironic to me), the old numerical version is still there in the XML - and I'd prefer to see that. How about having both shown (two columns, perhaps call them Public version and hg version or hg revision). We can certainly do this, but what would you like to see for tool suites and other tool types? The old Galaxy tool shed strictly required a suite_config.xml file that included the overall version of the suite. To make tool development easier, we're no longer requiring the inclusion of a suite_config.xml file ( we don't even differentiate types of tools since everything is a repository ). The definition of a tool in the next gen tool shed, is fairly loose. A tool could be data, it could be an exported workflow, it could be a suite of tools, a single tool, or just a set of files. So we'll need to define an easy way to provide a version of the tool if it will be different than the version of the repository tip. I see what you mean for the suite case. Maybe on the view details page each constituent tool could be shown with its classical version number from the XML file? Here's the future big picture highlights. Many of the details are yet to be defined and fleshed out... We're hoping that in the near future there will be many local tool sheds ( just like Galaxy instances ). I'm thinking that there will be a central tool shed broker of sorts that is hosted by the Galaxy team. This broker will provide 2 basic functions. It will enable local tool sheds ( including the current tool shed hosted by the Galaxy team ) to advertise their tools, and it will allow local Galaxy instances to use those advertisements to find tools that the local Galaxy instance's users are interested in. This specific point has not yet been discussed to any depth, so consider it fluid for now. I'm not immediately sold on this plan. To me one of the big plus points of having a single Official Tool Shed looked after by the Galaxy team is the convenience factor (a one stop shop), which requires critical mass, plus whatever QA happens as part of the current approval process. I would regard it as a step backwards if in order to hunt for a wrapper for a given tool, I had to resort to Google in order to find all the individual Galaxy Tool Sheds. When a Galaxy instance's admin locates tools within a specific tool shed that they want to install, they will be able to install them via a Galaxy tool installation control panel. Think of a UI that provides a check-boxed list of tools that have been found in some tool shed or sheds. The Galaxy admin will check those tools he wants to install, and the tools, along with all dependencies will automatically be installed in the local Galaxy instance. Dependencies could include 3rd party binaries, maybe some form of data, and other forms of dependencies. This is another good reason to keep tools separated in their own repositories. If you mean by dependencies the small task of installing the tool XML and associated scripts and data files currently bundled in the tar balls on the current Tool Shed, that seems fine. Anything beyond that seems difficult and likely to impose a significant extra load on tool wrapper authors. The installation will be virtually automatic, requiring little or no manual intervention via a package manage of sorts. This will be done using a combination of fabric scripts, and other components. All of the underlying mercurial stuff will be handled beneath the UI layer. This larger aim of installing the underlying dependencies is impossible in general - but that seems to be what you want to aim for. Consider obvious use case of closed source (non-redistributable) 3rd party binaries. I can think of several examples from the current Tool Shed wrappers, including the Roche Newbler off instrument applications, TMHMM and SignalP.
Re: [galaxy-dev] The new hg based Galaxy Tool Shed
On Wed, Jun 1, 2011 at 5:25 PM, Nate Coraor n...@bx.psu.edu wrote: Peter Cock wrote: Well, yes and no - as long as there are competing versions of a Galaxy tool (e.g. from an original author and a fork by a second author), and they use the same ID in their XML, you have a clash. This will have to be considered in the (automated) install interface. i.e. In general, when installing or updating any tool, there may be existing versions of some components already present. In fact two completely unrelated tools could even have the same XML ID by accident. I agree there could be a problem with tool ID uniqueness. We've talked about suggesting that people namespace their tool IDs to prevent this, but nothing formal has materialized at this point. That sounds sensible, and the sooner the better. I'm not immediately sold on this plan. To me one of the big plus points of having a single Official Tool Shed looked after by the Galaxy team is the convenience factor (a one stop shop), which requires critical mass, plus whatever QA happens as part of the current approval process. I would regard it as a step backwards if in order to hunt for a wrapper for a given tool, I had to resort to Google in order to find all the individual Galaxy Tool Sheds. It'll be possible for people to run their own Tool Sheds if they'd like, for whatever purpose - and this may be necessary for sharing extremely large data which we can't possibly host at the main Shed, but there should be an aggregator somewhere which lists all of the available public Sheds and makes it easy to add them as new sources to your Galaxy install. Like a slightly more organized Debian APT system. If there is an official meta tool shed aggregator, that would address my main concern about fragmenting things. If you mean by dependencies the small task of installing the tool XML and associated scripts and data files currently bundled in the tar balls on the current Tool Shed, that seems fine. Anything beyond that seems difficult and likely to impose a significant extra load on tool wrapper authors. It'll be up to the authors to decide what level of complexity they care to handle, Good - that silences a lot of my worries. ... but we want to move away from the situation where someone installs a tool but finds that it's unusable because the actual underlying dependency doesn't exist and is non-trivial to install. Improving the documentation shown on the tool shed could help here - make it easier for the tool wrapper to tell the Tool Shed user what will be required. Currently we get a short plain text box as part of the upload (no markup), and can include a (plain text) readme file which is easily viewable from the tool shed. I've just filed an enhancement request on a related idea: https://bitbucket.org/galaxy/galaxy-central/issue/565/ Show mockup of tool GUI in Galaxy Tool Shed This larger aim of installing the underlying dependencies is impossible in general - but that seems to be what you want to aim for. Consider obvious use case of closed source (non-redistributable) 3rd party binaries. I can think of several examples from the current Tool Shed wrappers, including the Roche Newbler off instrument applications, TMHMM and SignalP. Agreed, thankfully, the current dependency system (tool_dependency_dir in the config file (not in the sample config, sorry, I'll rememdy that shortly!)) only requires that you have an environment file that configures whatever is necessary (generally just $PATH) to find a dependency. So the tools in the Tool Shed would provide the XML, wrapper script (if necessary), and then instructions or perhaps an interface to configure the env file. I'd hope the common case where all that is required is the tool binary to be on the path, would not require any extra configuration files. See also: https://bitbucket.org/galaxy/galaxy-central/issue/82 [cut] Are you biting off more than you can chew? I hope I am misinterpreting your plans. Hopefully not! We're trying to think this through pretty thoroughly before we get started, thanks for joining in the discussion. =) I've been reassured :) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Blast2GO in Galaxy
On Tue, May 31, 2011 at 11:26 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi all, As I mentioned on Twitter, at the end of last week I wrapped Blast2GO for Galaxy, using the b2g4pipe program (Blast2GO for pipelines). See http://blast2go.org/ Currently current code is on bitbucket under my tools branch, https://bitbucket.org/peterjc/galaxy-central/src/tools/ Specifically files tools/ncbi_blast_plus/blast2go.* viewable here: https://bitbucket.org/peterjc/galaxy-central/src/tools/tools/ncbi_blast_plus/ I've using a Galaxy location file, tool-data/blast2go.loc, to offer one or more Blast2GO configurations (properties files), mapping this to the -prop argument. This way you could have for example the Spanish Blast2GO server with its current database (May 2010), and a local Blast2GO database. I want to setup a local database and try this before submitting the wrapper to the Tool Shed. I've done that using the current latest data from May 2011, so one year newer than the current default public Blast2GO database provided by the Blast2GO developers (dated May 2011). This requires downloaded some large data files and importing them into a MySQL database, and that took about 24 hours to process in all. The last step is done via their Java tool and took most of the time. For more details, see: http://blast2go.org/localgodb However, the end result is worth the effort as running Blast2GO is now much much faster. I need to try it on some larger files, but we're talking at least an order of magnitude quicker, maybe two. The input to the tool is a BLAST XML file, specifically blasting against a protein database like NR (so blastp or blastx, not blastn etc). I want to try some very large BLAST XML files to confirm b2g4pipe copes with the current BLAST+ output files - I gather there were some problems in this area in the past, so having the wrapper script fragment the XML might be a workaround. Currently the only real function of the wrapper script is to rename the output file - b2g4pipe insists on using the *.annot extension. Now that I have a local Blast2GO database, I'll be able to try out b2g4pipe on some bigger XML files (without having to wait ages). Right now the only output is a tabular three column *.annot file, which can be loaded into the Blast2GO GUI. For analysis within Galaxy, I'm wondering about an option to split the first column (which holds the original FASTA query's identifier and any description) in two. i.e. Split at the first white space to give the FASTA identifier, and any optional description as a separate column. That would make linking/joining/filtering on the ID much easier. If anyone has any comments or feedback now, that would be welcome. I've submitted the initial version to the Galaxy Tool Shed now, but would still welcome feedback and would even consider non-backwards compatible changes in the short term. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Updates to Galaxy Community Conference website
Hi all, I have a couple of minor suggestions for improving the website http://galaxy.psu.edu/gcc2011/Home.html First, please fill in a meaningful HTML titles since that will be used as the default caption in a bookmark, tab or window title. Second, please add a link to the presentation slides and videos, e.g. from the programme page and main page? http://wiki.g2.bx.psu.edu/GCC2011 I realise this isn't complete yet, but it is already a useful resource. Also once you have all the slides, could the links in the wiki table be made to point straight to the file? Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Sharing tool definitions between XML files
On Wed, Oct 6, 2010 at 11:16 AM, Peter pe...@maubp.freeserve.co.uk wrote: On Tue, Sep 28, 2010 at 12:04 PM, Peter pe...@maubp.freeserve.co.uk wrote: Hi all, I'm wondering if there is any established way to share parameter definitions between tool wrapper XML files? For example, I am currently working on NCBI BLAST+ wrappers, and these tools have a lot in common. For example, the output format option will be a select parameter, and it will be the same for blastn, blastp, tblastn, etc. I can just cut and paste the definition but this seems inelegant and will be a long term maintenance burden if it ever needs updating (lots of places would need to be updated the same way). I'd like to have a shared XML snippet defining this parameter which I could then reference/include from each tool wrapper than needs it. I'm thinking of something like XML's !ENTITY ... might work. Is this possible? Peter Alex Bossers replied on the BLAST+ thread to express interest in sharing definitions between wrapper XML files: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-September/003376.html As I add more parameters to my BLAST+ wrappers, keeping all the files in sync is getting a bit tedious... Peter Hi all, I'm replying to this old thread after finding this old (fixed) issue on the bug tracker, https://bitbucket.org/galaxy/galaxy-central/issue/131/allow-xml-includes-in-all-xml-files Does this offer a solution to sharing definitions in tool XML files? Has anyone got an example using this? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Defining new file formats in Galaxy (for new tool wrappers)
Hi all, Something I've not needed to do until now is define a new file format in Galaxy. I understand the basic principle and defining a subclass in Python... however, how does this work with new tools on the Tool Shed? In particular, if an output format is likely to be used by more than one tool, can we get it added to the Galaxy core? As an example, the basic functionality of the Blast2GO for pipelines tool (b2g4pipe) takes a BLAST XML input file, and gives a tab separated annotation output file. Galaxy already has 'blastxml' and 'tabular' file formats defined, so I didn't need to do anything extra. However, the tool can also take (a directory of) InterProScan XML files as input, so here a new 'interproscanxml' format would useful. Then any wrapper using or producing InterProScan XML could take advantage of this. e.g. Konrad's InterProScan wrapper could then offer the XML output as an option in addition to or instead of the tabular output. Related to this example, why isn't there a generic base class for XML formats in general? https://bitbucket.org/galaxy/galaxy-central/issue/568/missing-xml-datatype-base-class Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Defining new file formats in Galaxy (for new tool wrappers)
On Thu, Jun 2, 2011 at 6:39 PM, Greg Von Kuster g...@bx.psu.edu wrote: We will certainly include support for new data formats into the Galaxy core. In case you haven't seen it, details for adding new formats is available in our wiki at https://bitbucket.org/galaxy/galaxy-central/wiki/AddingDatatypes. Hi Greg, Should that page talk about lib/galaxy/datatypes/registry.py as well? That seems to be where mime types are specified, and for some reason (a historical fall back?), there is another sniffer listing here too (as well as in datatypes_conf.xml). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Components
On Mon, Jun 6, 2011 at 2:45 PM, Mariette jmari...@toulouse.inra.fr wrote: Hi everyone, I'm new in galaxy and I want to create a component, so I have multiple questions. The component executes a script which takes different formats : fasta|fastq|sff. As output it takes a folder, to do so I used the extra_files_path attribute of an output data (--out='$output.extra_files_path'). The result file is stored in this folder with the log file and should be in the same format as the input one. I wrote the xml file and it works just fine, however as it's a folder there is no way to use the result file(s) for further analysis. How can I manage this ? I use a wrapper script which moves the output file from the sub-directory to the location Galaxy requested. Regarding the file format, if you only have one input file you can use format=input, otherwise use the change_format tag in the XML. Also it can happen that there is more than 1 file outputted, how can I manage multiple outputs ? An other question about the input format, I have an option to the script to specify the format. I saw there is auto-format detection routines in galaxy, there is a way so I can call them to automatically complete this option ? Normally the wrapper specifies the output format explicitly. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Color blindness and Galaxy UI
On Mon, Jun 6, 2011 at 11:18 AM, James Taylor ja...@jamestaylor.org wrote: Peter, most of the current UI theme was designed by someone who is red-green colorblind (me), and we avoid using color alone for encoding information. The change has been reverted, thanks for brining it to our attention. I expect you would have spotted this commit soon anyway then :) On Mon, Jun 6, 2011 at 4:30 PM, Kanwei Li kan...@gmail.com wrote: Maybe the background color for the error can be changed as well? I just got a colorblind tester and #FA for example provides a much better contrast than the current color. (ss attached) That sounds sensible - see what James thinks ;) Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Components
On Tue, Jun 7, 2011 at 9:24 AM, Mariette jmari...@toulouse.inra.fr wrote: Thanks for the answer, so I tryed to write a wrapper, but the main script is not recognize. When I'm calling my wrapper in the xml file, I don't have to specify the full path but then from my wrapper if I want to call my main script I can't if this one is not on the PATH ? the wrapper current directory is the job_working_directory, there is anyway to access my main script just like it's done for the wrapper ? Generally put the wrapper script next to your tool XML file, and don't prefix it with any path in the command tag in the XML, e.g. /opt/galaxy/tools/mariette/my_tool.xml /opt/galaxy/tools/mariette/my_tool.sh (or my_tool.py or my_tool.pl etc depending on what language you used) Your wrapper script then has to call the real executable, and the normal way to do this on Unix is to have it on the PATH. You could also hard code an absolute path. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Components
On Tue, Jun 7, 2011 at 10:56 AM, Mariette jmari...@toulouse.inra.fr wrote: Thanks for the answer, Still considering the number of outputed files problem. In my script I know that if a cleaning option is set 2 other files will be created, so I check if the option is asked by the user and try this : outputs data name=log format=txt / #if $clean_pairends.clean_pairends_select==y: data name=output format=input/ #end if /outputs but this is not working ... there is a way to do what I want ? thx Jerome The #if stuff is Cheetah syntax, and only applies within the command tag - not to the XML file in general. You probably want a conditional output using when, see: https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] galaxy-dev] format=input
On Wed, Jun 8, 2011 at 9:26 PM, Jim Johnson johns...@umn.edu wrote: I added an attribute format_source (analogous to metadata_source) to the data tag to deal with that kind of issue: see: https://bitbucket.org/galaxy/galaxy-central/wiki/ToolConfigSyntax Where is the code to handle this? Maybe I missed the commit with all the recent activity. What happens if both format and format_source are given? I would hope an error - which means the wiki is misleading in that format is listed as required. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] User list and disk space?
On Tue, May 17, 2011 at 2:21 PM, Nate Coraor n...@bx.psu.edu wrote: Peter Cock wrote: That sounds really useful - it is something that you plan on adding to Galaxy officially? Yes, I'm working on adding a lot of user- and admin- side access to various numbers about disk usage (used by histories, used by deleted data, etc. and disk quotas. I hope to have this done shortly after the GCC. Thanks, --nate Hi Nate, Since you're looking at this kind of thing, would the Saved Histories page be worth updating to show the size on disk of each history? As a Galaxy user that seems moderately useful - although perhaps more of interest to Galaxy admins? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Setting up Galaxy with SGE
On Wed, Jun 15, 2011 at 2:30 AM, Ka Ming Nip km...@bcgsc.ca wrote: Hello, I have been trying to set up Galaxy to run its jobs on my SGE cluster using the Unified Method, based on the steps described in the wiki: https://bitbucket.org/galaxy/galaxy-central/wiki/Config/Cluster My local instance of Galaxy is installed under my home directory, but my cluster is on a different file system. I need to ssh to the head node before I can submit jobs. Is there any way to set up Galaxy with a SGE cluster on a different file system? I haven't looked into it in great detail, but we currently have a similar network setup, and would also like to link Galaxy to SGE. The main problem is the lack of a shared file system - in order to run a job on the cluster we (Galaxy) has to get the data onto the cluster, run the job, and get the results back. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
Hi Dan,
I just started work on updating my MIRA wrapper, and fell over a
regression present of 6079:5e6254f2b9e3, steps to reproduce on Linux:-
1. Install latest Galaxy
2. Install my MIRA wrapper v0.0.1 from the tool shed (don't need MIRA for this)
3. Run Galaxy
4. Select the MIRA tool
5. Change Backbones/reference chromosomes? or
Sanger/Capillary reads? or 454 reads? or
Solexa/Illumina reads? to yes
6. Instead of new file parameter appearing, get a KeyError:
'__current_case__' exception (details below).
Shorter example without a 3rd party tool:
1. Install latest Galaxy
2. Run Galaxy
3. Select the Convert SOLiD output to fastq tool
4. Change Is this a mate-pair run? to yes
5. Instead of new file parameters appearing, get a KeyError:
'__current_case__' exception.
By going through recent commits and retesting, I found the commit
which caused this regression:
changeset: 6078:894112e0b9d5
user:Daniel Blankenberg d...@bx.psu.edu
date:Thu Jun 09 11:26:41 2011 -0400
summary: Some fixes for interplay between DataToolParameter,
implicit datatype conversion and dynamic select
lists. Closes #491.
I guess there are downsides to running my development environment off
galaxy-central rather than galaxy-dist ...
Peter
-
Exception details,
KeyError: '__current_case__'
clear this
clear this
URL: http://127.0.0.1:8081/tool_runner/index
Module weberror.evalexception.middleware:364 in respond view
try:
__traceback_supplement__ = errormiddleware.Supplement,
self, environ
app_iter = self.application(environ, detect_start_response)
try:
return_iter = list(app_iter) app_iter =
self.application(environ, detect_start_response)
Module paste.debug.prints:98 in __call__ view
try:
status, headers, body = wsgilib.intercept_output(
environ, self.app)
if status is None:
# Some error occurred environ, self.app)
Module paste.wsgilib:539 in intercept_output view
data.append(headers)
return output.write
app_iter = application(environ, replacement_start_response)
if data[0] is None:
return (None, None, app_iter) app_iter =
application(environ, replacement_start_response)
Module paste.recursive:80 in __call__ view
environ['paste.recursive.script_name'] = my_script_name
try:
return self.application(environ, start_response)
except ForwardRequestException, e:
middleware = CheckForRecursionMiddleware( return
self.application(environ, start_response)
Module paste.httpexceptions:632 in __call__ view
[]).append(HTTPException)
try:
return self.application(environ, start_response)
except HTTPException, exc:
return exc(environ, start_response) return
self.application(environ, start_response)
Module galaxy.web.framework.base:145 in __call__ view
kwargs.pop( '_', None )
try:
body = method( trans, **kwargs )
except Exception, e:
body = self.handle_controller_exception( e, trans,
**kwargs ) body = method( trans, **kwargs )
Module galaxy.web.controllers.tool_runner:68 in index view
# so make sure to create a new history if we've never had
one before.
history = trans.get_history( create=True )
template, vars = tool.handle_input( trans, params.__dict__ )
if len(params) 0:
trans.log_event( Tool params: %s % (str(params)),
tool_id=tool_id ) template, vars = tool.handle_input( trans,
params.__dict__ )
Module galaxy.tools:966 in handle_input view
# Update state for all inputs on the current page taking new
# values from `incoming`.
errors = self.update_state( trans,
self.inputs_by_page[state.page], state.inputs, incoming )
# If the tool provides a `validate_input` hook, call it.
validate_input = self.get_hook( 'validate_input' )
errors = self.update_state( trans, self.inputs_by_page[state.page],
state.inputs, incoming )
Module galaxy.tools:1159 in update_state view
group_state = state[input.name] = {}
# TODO: we should try to preserve values if we can
self.fill_in_new_state( trans,
input.cases[current_case].inputs, group_state, context )
group_errors = dict()
group_old_errors = dict()
self.fill_in_new_state( trans, input.cases[current_case].inputs,
group_state, context )
Module galaxy.tools:876 in fill_in_new_state view
context = ExpressionContext( state, context )
for input in inputs.itervalues():
state[ input.name ] =
Re: [galaxy-dev] User list and disk space?
On Thu, Jun 9, 2011 at 4:20 PM, Nate Coraor n...@bx.psu.edu wrote: Peter Cock wrote: Hi Nate, Since you're looking at this kind of thing, would the Saved Histories page be worth updating to show the size on disk of each history? As a Galaxy user that seems moderately useful - although perhaps more of interest to Galaxy admins? This is already done in development, I was going to finish up the code and commit it within the next couple of weeks. This has now been committed to galaxy-central, and looks very useful. I notice the Saved Histories page now has buttons Rename, Delete, Delete and remove datasets from disk (new) and Undelete. That distinction between hide it away and really delete should make sense to users from the Recycle Bin/Trash idea for files on Windows/MacOSX. Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] The new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 3:00 AM, Ravi Madduri madd...@mcs.anl.gov wrote: I apologize for jumping on to this thread a bit late. I read below that there is a plan to pull tools into a galaxy installation automagically. I wonder if you plan on providing some kind of API to query the tool registry and discover the tools and install them into an existing galaxy installation. Yes, have a look at Greg's slides from the Galaxy Community Conference http://wiki.g2.bx.psu.edu/GCC2011 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
Hi all, I just tried updating one of my tools on the new hg based Galaxy Tool Shed and ran into some issues. I guess I'm the first person to try this... First of all, there is no clear update action like there used to me. Could that be restored please? https://bitbucket.org/galaxy/galaxy-central/issue/585/hg-tool-shed-lacks-update-tool-action I tried using the upload files to repository option, and picked my tarball. I did not pick a location since I wanted the tar ball unpacked at the repository root, but this isn't what happened: https://bitbucket.org/galaxy/galaxy-central/issue/586/hg-tool-shed-upload-files-to-repository Finally, it seems there is currently no delete files action, so I can't fix this (delete the misplaced files) via the web interface. https://bitbucket.org/galaxy/galaxy-central/issue/584/cant-delete-files-in-new-hg-tool-shed Are you expecting tool authors to work primarily at the hg level? Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
On Thu, Jun 16, 2011 at 2:23 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Peter, If you were curious it was fixed about 7 hours later in 5681:0886ed0a8c9f Thanks for using Galaxy, Dan Thanks - I know this kind of unexpected breakage is inevitable sometimes, but would recommend tool developers in general run with galaxy-dist or galaxy-central? Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Conditional parameters regression: KeyError: '__current_case__'
On Thu, Jun 16, 2011 at 3:01 PM, Daniel Blankenberg d...@bx.psu.edu wrote: Hi Peter, Production servers, or any server where you have users, should always track Galaxy-dist. Yes, that's very clear. Galaxy-central is a development repo (you get access to the latest and greatest, but also should Expect more bugs to pop up at any given time), from the Galaxy-central header on bitbucket: Main development repository for Galaxy. Active development happens here, and this repository is thus intended for those working on Galaxy development. See http://bitbucket.org/galaxy/galaxy-dist/ for a more stable repository intended for end-users. Its really a personal choice that each (tool) developer will have to make based upon whether they consider themselves to be a pure end-user (just adding tools or running a Galaxy server) who only wants to work on a stable branch; or if they want to contribute to Core development or gain early access to development features (and bugs). I would say that finalized tools and e.g. tools submitted to the tool shed should be vetted against galaxy-dist. OK, that's basically how I've been working - quite often during tool development I've hit bugs in Galaxy and wanted to make branches with fixes etc, or needed new functionality for a tool that isn't yet in galaxy-dist. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 3:26 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, I understand, and that is why I am working feverishly to add features that do not force the use of hg form the command line (I've got several features functional, but not everything yet). The current tool shed requires some use of hg if you want to delete files from the repo, but other than that, hg is not really required. That's great, but I do feel the new hg went live a bit prematurely. The current tool shed also provides some very useful features (e.g., browsing tool code files) that were not available in the old tool shed. Really? I find the browsing the tool code files has regressed. In old tool shed had a simple non-interactive tree of the files visible on the main page for a tool or suite, where the individual files could be clicked on to view/download. It was simple and effective (at least when there weren't many files which seems typical). The new tool shed makes you click on tool actions, browse repo, and then gives an interactive tree which is fully collapsed by default, forcing lots more clicking to drill down to the files. All that clicking makes it slow and fiddly to use. In addition, tool developers are no longer forced to to the extra work that was necessary in the old tool shed to add a tool (i.e., there is no longer a requirement for tool suite config definitions ). For tool suites you have a point, that was a bit of a hassle. My hope is that the current features make everyone happy enough that they will be able to wait for those messing featuers that are wanted, using the current features in the meantime. It'll get there :) Another big regression I would prioritise is the complete lack of any user provided text on a tool's main page. The old tool shed had the minimal text box where you could summarise the tool, its requirements, recent changes etc. The new tool shed has nothing to replace this as far as I can see. A simple wiki like, ReST, or just plain text details box would do. Automatic detection of a readme text file in the repo, to be displayed on the tools' main page would be an elegant solution (are you familiar with how github.com does this?). It also keeps the description in the hg repo which is a plus point. The mock up idea would be another (more complex) way to tackle this, https://bitbucket.org/galaxy/galaxy-central/issue/565 Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Cluster Usage
On Thu, Jun 16, 2011 at 4:31 PM, Robert Jackson rj...@utpa.edu wrote: Does anyone have practical information for installation and set up for the galaxy distribution on a cluster?? Did you not get these replies? http://lists.bx.psu.edu/pipermail/galaxy-user/2011-June/002715.html http://lists.bx.psu.edu/pipermail/galaxy-user/2011-June/002715.html http://lists.bx.psu.edu/pipermail/galaxy-user/2011-June/002716.html On the plus side, the galaxy-dev list is more appropriate for this question than galaxy-user list. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Updating tools on new hg based Galaxy Tool Shed
On Thu, Jun 16, 2011 at 5:40 PM, Chris Fields cjfie...@illinois.edu wrote: I'm of the opposite mind; I'm not sure there is an absolute need to have one's tools be a branch or fork of the main tool shed, though I agree there is also utility in allowing that (having a customized 'main' repo, or optimizing tools already present). To clarify, I have been using branches on an hg fork of the main Galaxy repository up until now, then preparing tarballs for upload to the Galaxy Tool Shed. I could equally have tracked my local changes under git, svn or cvs. The point is under this model, the Tool Shed has no connection to whatever repository (or repositories) the tool author uses on their machine. Barring teething issues like Bug 584/585/586 tool authors could continue to use this development model, where the fact that the Galaxy Tool Shed uses an hg repository internally is an irrelevant internal implementation detail. As far as I know, there are no plans to make the Tool Shed work directly with a full Galaxy hg repo - only mini-repositories, one for each tool or tool suite. Having completely separate repos for tools seems cleaner, focusing development on those tools alone (not any of the others present in a branch) and pushes maintenance of the tool back to the tool developer themselves Yes, and that is the model the new Galaxy Tool Shed is following. Although I'm a little hazy on the details of tool suites in the new model (and there are lots of situations where it makes sense to bundle several tools). (e.g. there is no need for the galaxy devs to 'pull' in changes at any point into a main repo). Well, there still is for general bug fixes, adding new file formats, and merging any tools which they decide to include in the core set. This might also allow the galaxy devs to designate a set of 'blessed' or supported tools in various repositories. Indeed. Re: git: as Peter knows I'm also primarily a git/github user. It is feasible at some future point to allow git/github repos. For instance, one could possibly integrate github usage via this: http://hg-git.github.com/ Of course, I think it's much more important that any additional vcs integration wait until the new tool shed interface stabilizes somewhat, but (at least from the github perspective) seems like it shouldn't be terribly hard to do. True - if there were sufficient demand from Tool Shed contributors. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Question about installing NCBI BLAST+ onto Galaxy
On Tue, Jun 28, 2011 at 9:52 PM, George Michopoulos gior...@stanford.edu wrote: Hey everyone, Hope all is well! I was wondering if someone could help me with another error I ran into. I recently downloaded the NCBI BLAST+ toolkit and it automatically installed itself into Galaxy. I'm just wondering if anyone knows where I am supposed to put the directory with the database files it needs to run correctly, or if it makes a difference. It shouldn't matter as long a you use an appropriate path in the blastdb.loc and blastdb_p.loc files. We use /data/blastdb/ for ours. I have configured the blastdb.loc file as shown below, and the database now appears in the drop-down menu for the NCBI tools, but when I try executing any of the BLASTs Galaxy returns the following error, regardless of the path permutation I try: At least Galaxy is finding the loc file :) When I tried putting the databases within the galaxy installation (within galaxy_test) and I used the whole path: BLAST Database error: No alias or index file found for nucleotide database [/Users/burtonigenomics/Rosa_Files/bin/fastx_bin/galaxy_test/tool-data/blastdb/refseq_rna] in search path [/Users/burtonigenomics/Rosa_Files/bin/fastx_bin/galaxy_test/database/job_working_directory/28::] Return error code 2 from command: tblastx -query /Users/burtonigenomics/Rosa_Files/bin/fastx_bin/galaxy_test/database/files/000/dataset_27.dat -db /Users/burtonigenomics/Rosa_Files/bin/fastx_bin/galaxy_test/tool-data/blastdb/refseq_rna -evalue 0.001 -out /Users/burtonigenomics/Rosa_Files/bin/fastx_bin/galaxy_test/database/files/000/dataset_32.dat -outfmt 6 -num_threads 8 Does BLAST+ work at the command line? Does BLAST+ work within Galaxy for FASTA vs FASTA (rather than FASTA vs database)? Also what does this give: ls /Users/burtonigenomics/Rosa_Files/bin/fastx_bin/galaxy_test/tool-data/blastdb/refseq_rna.* My guess is you have tried a valid path, but that the Galaxy user does not have read permission so BLAST+ can't open the database. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Question about installing NCBI BLAST+ onto Galaxy
On Thu, Jun 30, 2011 at 1:26 AM, George Yianni Michopoulos wrote: Hey Peter, Thanks so much for your help, your comments helped me realize what was wrong. The issue was that I wasn't including the name of the database within the path, just the folder it was in! Best, George Michopoulos Easily done, I'm glad its working now. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] optional integer value in tool config
On Mon, Jul 4, 2011 at 1:46 PM, Sarah Diehl di...@immunbio.mpg.de wrote: Hello, in the config of a tool I want to integrate into Galaxy I would like to have a parameter with the following properties: param name=var type=integer label=Var value=50 optional=true min=10 max=100 / So it should be possible to leave this field empty, but when you choose to enter a number it should be between 10 and 100. The problem now is that Galaxy refuses to accept it when this field is empty. Which version of Galaxy are you running? Optional float and integer parameters was only recently made possible under issue 403 with an initial fix committed two weeks ago, https://bitbucket.org/galaxy/galaxy-central/issue/403 https://bitbucket.org/galaxy/galaxy-central/changeset/adcc600effa4 After encountering this problem I tried to solve the issue within Cheetah and tried the following: #if $var.value 10 or $var.value 100 #set $var.value = #end if You'd need to try something like this: #if int($var.value) 10 or int($var.value) 100 #set $var.value = #end if but you also have to first exclude the case where is is already empty as then int would fail. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] New Tool Shed: Inconsistent field names
On Tue, Jul 5, 2011 at 7:24 PM, Greg Von Kuster g...@bx.psu.edu wrote: I have previously commented on the problem with showing the description on its own without the context of the tool name as displayed in Galaxy: http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-June/005640.html and: http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-December/004012.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-December/004017.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2010-December/004019.html One idea would be to show just the name in the left hand pane of Galaxy, with the description as a tool tip AND show this at the top of the tool main page in the central panel. This would encourage tool authors to write this field as a self contained synopsis, and make the left hand panel less text heavy. This may be considered for the Galaxy instance (the decision is in the hands of others), but would probably not work in the tool shed. Again, we'll see what we can improve as I make progress on the repository metadata implementation. Sorry if that was unclear, I did mean the main Galaxy interface there. Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] From NCBI SRA to UCSC viewer pipeline.
On Thu, Jul 7, 2011 at 7:14 AM, colin molter colin.mol...@gmail.com wrote: Hi all, i am trying to use a local instance of my galaxy to pre-format data stored at sra-ncbi. Does anyone has a working pipeline that (s)he could share. Here is the pipeline I am using, with some questions. 1/ download sra files to my server. 2/ transform them in fastq using the sra toolbox. 3/ upload them in galaxy, by using the 'add to data library' 4/ use the fastq groomer to enable to use the fast q in galaxy. Note: i guess that the data at sra are already in the fastq sanger format. So it could be nice to be able to skip that point (it took 10 hours to groom a fastq of 25Gb). Just upload load them and set the file format to fastqsanger (either when you upload, or afterwards via the pencil icon to edit the attributes - this is fast as it doesn't change the file). Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] New version_string tag in tool XML
Hi all, This new feature looks very exciting for tracking reproducibility - now in theory both the tool wrapper version (explicit in the XML) and any underlying tool version will both be recorded: https://bitbucket.org/galaxy/galaxy-central/changeset/84b20f29dfdf I have updated the NCBI BLAST+ wrappers to use the new feature, please merge/transplant this branch/commit: https://bitbucket.org/peterjc/galaxy-central/src/blast_2011_07_07 https://bitbucket.org/peterjc/galaxy-central/changeset/11f425f4e137 I hope the new tool version string will get its own field in the database soon (and be viewable via the web interface). Note that string may well be multi-line, e.g. $ blastp -version blastp: 2.2.25+ Package: blast 2.2.25, build Apr 6 2011 13:59:28 Unless that is you go for more fragile solutions like these to ensure a single line: $ samtools 21 | grep -i ^Version Version: 0.1.16 (r963:234) $ bwa 21 | grep -i ^Version Version: 0.5.9-r16 Thanks, Peter P.S. I would say the name version_string is a bit confusing, I'd have picked version_cmd or version_command instead. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
