Re: [galaxy-dev] Loss of sub-task order with parallelism

2012-06-14 Thread Peter Cock
On Thu, Jun 14, 2012 at 6:15 PM, Dannon Baker dannonba...@me.com wrote: On Jun 14, 2012, at 12:48 PM, Peter Cock wrote: In a separate example with 33 sub-tasks, there were two of these inversions, while in yet another example with 33 sub-tasks there was a trio submitted out of order. This non

Re: [galaxy-dev] input and output extensions

2012-06-08 Thread Peter Cock
On Fri, Jun 8, 2012 at 9:15 AM, Nicholas Robinson nicholas.robin...@nofima.no wrote: Hi, 1. INPUT I am trying to make a simple tool that sends a command to run crimap without a wrapper ie. . command crimap $param_file  $option  gt;$output1 /command .. However,

Re: [galaxy-dev] How to upload local files in Galaxy

2012-06-08 Thread Peter Cock
On Fri, Jun 8, 2012 at 2:38 PM, Raj Ayyampalayam ra...@uga.edu wrote: Hello, I am interested in this local upload file tool. Can you please send the files to me as well. Thanks. -Raj This seems a potentially very useful tool, so putting it on the Galaxy Tool Shed seems like a better idea

[galaxy-dev] Galaxy cluster jobs - automatic priority setting

2012-06-04 Thread Peter Cock
Dear all, Does Galaxy have any mechanisms to set the priority of jobs submitted to the cluster? Specifically I am interested in SGE via DRMAA, but this is a general issue. If there is some existing code, I might be able to use it for the following situation: The specific motivation is for

Re: [galaxy-dev] Is there a way to dynamically assign Queues in Galaxy

2012-06-01 Thread Peter Cock
On Thu, May 31, 2012 at 12:53 AM, Anthonius deBoer thondeb...@me.com wrote: Hi, I often find my self having to battle with other users of the cluster, even for the one-off jobs that someone runs interactively... We have setup Fast/short queues and Long queues... Is there a way for a tool to

Re: [galaxy-dev] Galaxy at Bioinformatics Interoperability SIG, ISMB 2012

2012-06-01 Thread Peter Cock
On Fri, Jun 1, 2012 at 6:25 PM, Michael Reich micha...@broadinstitute.org wrote: Dear Galaxy developer community, Galaxy tool developers attending ISMB are encouraged to sign up for the Bioinformatics Interoperability SIG to be held on Friday, June 13. The SIG will include presentations on

Re: [galaxy-dev] Galaxy not killing split cluster jobs

2012-05-31 Thread Peter Cock
On Tue, May 1, 2012 at 3:46 PM, Dannon Baker dannonba...@me.com wrote: On Tue, May 1, 2012 at 3:10 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On May 1, 2012, at 10:03 AM, Dannon Baker dannonba...@me.com wrote: On May 1, 2012, at 9:51 AM, Peter Cock wrote: I'm a little confused about

Re: [galaxy-dev] HMMER wrappers

2012-05-29 Thread Peter Cock
On Fri, May 25, 2012 at 4:54 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Edward, It has taken me a while but I'm how trying to use HMMER3, and do so from within Galaxy. I've realised that the per sequence and per domain tables from hmmscan and hmmsearch (via the --tblout

[galaxy-dev] http://toolshed.g2.bx.psu.edu/ 502 Bad Gateway

2012-05-25 Thread Peter Cock
Hi, Could someone take a look at the Tool Shed server please? I'm getting this for http://toolshed.g2.bx.psu.edu/ 502 Bad Gateway nginx/1.1.4 Thanks, Peter ___ Please keep all replies on the list by using reply all in your mail client.

Re: [galaxy-dev] http://toolshed.g2.bx.psu.edu/ 502 Bad Gateway

2012-05-25 Thread Peter Cock
On Fri, May 25, 2012 at 12:03 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, The tool shed should be functional now.  Thanks for reporting this! Greg It was working again when I got your email, but while I was out to lunch it has gone back to Error 502 Bad Gateway :( Peter

Re: [galaxy-dev] HMMER wrappers

2012-05-25 Thread Peter Cock
On Fri, May 25, 2012 at 4:54 PM, Peter Cock p.j.a.c...@googlemail.comwrote: I ran into some similar problems wrapping other tools giving table based output, and used a wrapper script to make them into tab separated tables for use in Galaxy. e.g. SignalP 3 (spaces), EffectiveT4 (semi-colons

Re: [galaxy-dev] Avoid to download output files

2012-05-24 Thread Peter Cock
On Thu, May 24, 2012 at 1:54 PM, Jeremy Goecks jeremy.goe...@emory.edu wrote: Julie wrote: Thank you Jeremy, you answered partially. I would like insert a command, to launch my script, in the script or in the xml files of my tools to force it to change the extension of output files. But it

Re: [galaxy-dev] Avoid to download output files

2012-05-24 Thread Peter Cock
On Thu, May 24, 2012 at 3:27 PM, julie dubois dubju...@gmail.com wrote: Ok I've tried to add my script perl in the command line with semi-colon. It's work But this script sets in entry the variable $wigoutput of my file xml. But when I show it, it's not what I want; It's just the value : 1 .

Re: [galaxy-dev] AttributeError: 'module' object has no attribute 'FakeSocket'

2012-05-22 Thread Peter Cock
On Tue, May 22, 2012 at 4:14 PM, Sarah Diehl di...@immunbio.mpg.de wrote: Thanks Peter! It's very likely that I have the exact same problem, because I also have CentOS and needed to compile Python. In that case make sure you've done this: sudo yum install openssl openssl-dev and then repeat

Re: [galaxy-dev] Using a variable in data from_work_dir=

2012-05-17 Thread Peter Cock
On Thu, May 17, 2012 at 12:13 PM, Peter van Heusden p...@sanbi.ac.za wrote: Hi there I'm trying to wrap Bioperl's bp_genbank2gff script. This script, by default, produces an output file named the same as the basename of the input file, with .gff added. So e.g. bp_genbank2gff data.gbk

[galaxy-dev] Compressed data files in Galaxy? (e.g. GZIP or BGZF)

2012-05-16 Thread Peter Cock
Hello all, What is the current status in Galaxy for supporting compressed files? We've talked about this before, for example in addition to FASTQ, many of us have expressed a wish to work with gzipped FASTQ. I understand that some have customized their local Galaxy installations to use gzipped

Re: [galaxy-dev] user data upload directory structure

2012-05-15 Thread Peter Cock
On Mon, May 14, 2012 at 10:22 PM, Jean-Christophe Ducom jcdu...@scripps.edu wrote: All- Is there a way to change the upload default directory structure (/database/files) to organize files per user_id instead? something along the following lines ~galaxy-dist/database/files/postgresql_user_id0

Re: [galaxy-dev] parameters going into quality scores fastq Illumina

2012-05-15 Thread Peter Cock
On Mon, May 14, 2012 at 9:03 PM, Claudia Gottstein gottst...@cnsi.ucsb.edu wrote: Hi, I would like to understand which parameters, ideally which algorithm go into the scores one obtains from the quality statistics out of fastQ illumina data. What is this score composed of? And how come that

Re: [galaxy-dev] Sub-task splitting

2012-05-09 Thread Peter Cock
On 12-05-03 9:51 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hello all, Currently the Galaxy experimental task splitting code allows splitting into N chunks, e.g. 8 parts, ... Or, into chunks of at most size N (units dependent on the file type, e.g. lines in a tabular file or number

Re: [galaxy-dev] job terminating on warnings from software

2012-05-05 Thread Peter Cock
On Friday, May 4, 2012, Michael Moore wrote: I have a long string of steps dying on the last step because GNUPLOT sends a warning to the log which galaxy faithfully records 11 faithful times in my history files. The 10 other files have downloadable content, and in fact, outside of galaxy the

[galaxy-dev] Galaxy Admin - Browsing any user's histories?

2012-05-03 Thread Peter Cock
Hi all, Is it possible for a Galaxy Admin to browse any user's history? Sadly the Admin page on the wiki is still rather lean: http://wiki.g2.bx.psu.edu/Admin/Interface For instance, I've monitored a large Galaxy job on our cluster, and now that it has finished I'd like to see the results (in

Re: [galaxy-dev] Galaxy Admin - Browsing any user's histories?

2012-05-03 Thread Peter Cock
On Thu, May 3, 2012 at 11:40 AM, Hans-Rudolf Hotz h...@fmi.ch wrote: On 05/03/2012 12:10 PM, Peter Cock wrote: Hi all, Is it possible for a Galaxy Admin to browse any user's history? Sadly the Admin page on the wiki is still rather lean: http://wiki.g2.bx.psu.edu/Admin/Interface

Re: [galaxy-dev] Galaxy not killing split cluster jobs

2012-05-03 Thread Peter Cock
On Tue, May 1, 2012 at 3:46 PM, Dannon Baker dannonba...@me.com wrote: I'll take care of it.  Thanks for reminding me about the TODO! On a related point, I've noticed sometimes one child job from a split task can fail, yet the rest of the child jobs continue to run on the cluster wasting CPU

Re: [galaxy-dev] Galaxy not killing split cluster jobs

2012-05-03 Thread Peter Cock
On Thu, May 3, 2012 at 3:54 PM, Dannon Baker dannonba...@me.com wrote: On a related point, I've noticed sometimes one child job from a split task can fail, yet the rest of the child jobs continue to run on the cluster wasting CPU time. As soon as one child job dies (assuming there are no

[galaxy-dev] Sub-task splitting

2012-05-03 Thread Peter Cock
Hello all, Currently the Galaxy experimental task splitting code allows splitting into N chunks, e.g. 8 parts, with: parallelism method=multi split_mode=number_of_parts split_size=8 ... / Or, into chunks of at most size N (units dependent on the file type, e.g. lines in a tabular file or number

Re: [galaxy-dev] galaxy cluster integration

2012-05-02 Thread Peter Cock
On Wed, May 2, 2012 at 7:18 PM, Anne Pajon anne.pa...@cancer.org.uk wrote: Hi, I am struggling at integrating galaxy with our cluster. I am currently running galaxy on a separate server machine from our LSF cluster. The DRMAA library has been installed on the cluster and I've re-installed

Re: [galaxy-dev] galaxy cluster integration

2012-05-02 Thread Peter Cock
On Wed, May 2, 2012 at 9:58 PM, Anne Pajon anne.pa...@cancer.org.uk wrote: Hi Peter, Thanks for the clarification, it is really helpful. Would you mind sharing what needs to be done in term of configuration and instalation to transform the galaxy server into a submitting job to the cluster?

[galaxy-dev] Job scheduling: FIFO, or fairer to multiple users?

2012-05-01 Thread Peter Cock
Hello all, I'm curious if there is any way to manipulate the Galaxy job queuing in order to be 'fairer' to multiple simultaneous users. My impression is that Galaxy uses a simple FIFO queue itself, with for cluster jobs offloaded to the cluster queue immediately. In our case, I'm looking at

[galaxy-dev] Galaxy not killing split cluster jobs

2012-05-01 Thread Peter Cock
Hi all, We're running our Galaxy with an SGE cluster, using the DRMAA support in Galaxy, and job splitting. I've noticed if the user cancels a job (that was running or queued on the cluster) while the job is shows as deleted in Galaxy, looking at the queue on the cluster with qstat shows it

Re: [galaxy-dev] Merging BLAST database support into Galaxy?

2012-04-30 Thread Peter Cock
On Thu, Apr 26, 2012 at 10:40 PM, Edward Kirton eskir...@lbl.gov wrote: your suggestion for blastdbn and blastdbp sounds fine. it's okay if a few of our users need to edit the metadata of the dbs in their history. thanks for asking and doing this. Great. Perhaps you can throw some light on

[galaxy-dev] update_metadata.sh references missing get_python.sh

2012-04-26 Thread Peter Cock
Hello all, The shell script scripts/cleanup_datasets/update_metadata.sh tries to call non-existent file scripts/get_python.sh What is this intended to do? Thanks, Peter ___ Please keep all replies on the list by using reply all in your

Re: [galaxy-dev] Merging BLAST database support into Galaxy?

2012-04-26 Thread Peter Cock
Hi Edward, I've started work on this in earnest now. I see you only defined one new datatype, blastdb, which worked for nucleotide databases. I want to handle protein databases too, so I think two datatypes makes sense - which I am currently calling blastdbn and blastdbp. That won't be

[galaxy-dev] Composite datatypes and peek and main display

2012-04-26 Thread Peter Cock
Hello all, I'm looking at 'basic' composite datatypes in Galaxy, based on how Edward did BLAST databases in his BLAST+ fork on the toolshed. I've extended his work to handle both protein and nucleotide databases, and have got this working for tools to create a database (i.e. wrapped makeblastdb)

Re: [galaxy-dev] Stalled upload jobs under Admin, Manage jobs

2012-04-26 Thread Peter Cock
On Fri, Mar 16, 2012 at 11:00 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Feb 13, 2012 at 5:02 PM, Nate Coraor n...@bx.psu.edu wrote: On Feb 10, 2012, at 6:47 AM, Peter Cock wrote: Hello all, I've noticed we have about a dozen stalled upload jobs on our server from several users

Re: [galaxy-dev] About an additional tool

2012-04-24 Thread Peter Cock
On Tue, Apr 24, 2012 at 10:03 AM, Ciara Ledero lede...@gmail.com wrote: Hi there, I know  Galaxy already has a SAM-to-BAM converter, but part of my exercise/task is to incorporate a script that uses samtools' view command. I get this error: [samopen] SAM header is present: 66338 sequences.

Re: [galaxy-dev] JobManager object has no attribute dispatcher

2012-04-24 Thread Peter Cock
On Mon, Apr 23, 2012 at 7:08 PM, Nate Coraor n...@bx.psu.edu wrote: Not quite shortly, but it's been committed as 476ce0b78713.  Sorry for the wait. --nate Thanks Nate - that did the trick :) Peter ___ Please keep all replies on the

Re: [galaxy-dev] BAM to BigWig (and tool ID clashes)

2012-04-20 Thread Peter Cock
On Fri, Apr 20, 2012 at 2:17 AM, Brad Chapman chapm...@50mail.com wrote: Lance and Peter; Peter, thanks for noticing the problem and duplicate tools. Lance, I'm happy to merge these so there are not two different versions out there. I prefer your use for genomeCoverageBed over my custom

Re: [galaxy-dev] BAM to BigWig (and tool ID clashes)

2012-04-20 Thread Peter Cock
On Fri, Apr 20, 2012 at 4:12 PM, Brad Chapman chapm...@50mail.com wrote: Peter and Lance; I've made the update to Brad's script from the Tool Shed (attached), switching to using genomeCoverageBed and bedGraphToBigWig (based on the approach used in Lance's script), although in doing so I

Re: [galaxy-dev] Mira-Assembler: DOESN'T WORK ON GALAXY

2012-04-19 Thread Peter Cock
On Thu, Apr 19, 2012 at 12:40 AM, JIE CHEN jiechenable1...@gmail.com wrote: The version I installed is : mira_3.4.0_prod_linux-gnu_x86_64_static OK, good. The other key question I asked was did you get anything in the MIRA log file (it should be in your history with text data even though it

[galaxy-dev] BAM to BigWig (and tool ID clashes)

2012-04-19 Thread Peter Cock
Hi Brad Lance, I've been using Brad's bam_to_bigwig tool in Galaxy but realized today (with a new dataset using a splice-aware mapper) that it doesn't seem to be ignoring CIGAR N operators where a read is split over an intron. Looking over Brad's Python script which calculates the coverage to

Re: [galaxy-dev] BAM to BigWig (and tool ID clashes)

2012-04-19 Thread Peter Cock
On Thu, Apr 19, 2012 at 1:55 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Brad Lance, I've been using Brad's bam_to_bigwig tool in Galaxy but realized today (with a new dataset using a splice-aware mapper) that it doesn't seem to be ignoring CIGAR N operators where a read is split

Re: [galaxy-dev] BAM to BigWig (and tool ID clashes)

2012-04-19 Thread Peter Cock
On Thu, Apr 19, 2012 at 2:32 PM, Greg Von Kuster g...@bx.psu.edu wrote: The tool shed forces unique repository names per user account, allowing for uniqueness with that combination.  All tools uploaded into a tool shed repository are assigned a unique id called a guid, which is unique for all

Re: [galaxy-dev] JobManager object has no attribute dispatcher

2012-04-19 Thread Peter Cock
On Thu, Apr 19, 2012 at 2:53 PM, Nate Coraor n...@bx.psu.edu wrote: On Apr 17, 2012, at 9:04 AM, Peter Cock wrote: Hi all, Does anyone know what might have introduced this problem running galaxy-dist when using the task splitting functionality? I'm using the latest code from the default

Re: [galaxy-dev] Mira-Assembler: DOESN'T WORK ON GALAXY

2012-04-19 Thread Peter Cock
On Thu, Apr 19, 2012 at 6:35 PM, JIE CHEN jiechenable1...@gmail.com wrote: Hi Peter, Thank you for your patience. I checked the error message in the history. They all give exactly the same error-- the one i gave in the first thread. Are you saying this is the entire contents of the MIRA log

Re: [galaxy-dev] Mira-Assembler: DOESN'T WORK ON GALAXY

2012-04-19 Thread Peter Cock
On Thu, Apr 19, 2012 at 7:13 PM, JIE CHEN jiechenable1...@gmail.com wrote: Hi Peter, Here is the full log: Excellent :) The good news is MIRA seems to be installed and running fine - it just didn't like your test data, and I understand why: ... Sanger will load 1 reads. Longest Sanger:

[galaxy-dev] Merging BLAST database support into Galaxy?

2012-04-18 Thread Peter Cock
Hi Edward, We're now running BLAST+ searches on our local Galaxy via our cluster, and some of the cluster nodes have relatively small amounts of RAM. This means I've become more aware of limitations in the NCBI BLAST+ tools' support for using a subject FASTA file (instead of making a local BLAST

Re: [galaxy-dev] Merging BLAST database support into Galaxy?

2012-04-18 Thread Peter Cock
On Wed, Apr 18, 2012 at 10:53 AM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Edward, We're now running BLAST+ searches on our local Galaxy via our cluster, and some of the cluster nodes have relatively small amounts of RAM. This means I've become more aware of limitations in the NCBI

[galaxy-dev] Conditional tool options based on input file format?

2012-04-18 Thread Peter Cock
Dear all, I've looked over the wiki and as far as I can see, 'when' tags used inside 'conditional' tags only work on another input variable's values. I would like to be able to do is a conditional based on the file format of an input file. For instance, a tool might take XML or tabular files,

Re: [galaxy-dev] Merging BLAST database support into Galaxy?

2012-04-18 Thread Peter Cock
On Wed, Apr 18, 2012 at 6:10 PM, Edward Kirton eskir...@lbl.gov wrote: sounds great, thanks peter.  i granted you access to my toolshed repo, but perhaps we want only one tool in the toolshed when all done. Thanks - but given the tools currently live in the main repository with Galaxy itself,

Re: [galaxy-dev] Mira-Assembler: DOESN'T WORK ON GALAXY

2012-04-18 Thread Peter Cock
On Wed, Apr 18, 2012 at 11:21 PM, JIE CHEN jiechenable1...@gmail.com wrote: Dear all, I just installed Mira-Assembler via Galaxy tool sheds. As expected, it downloaded the wrappers into the shed-tools directory. Then I downloaded and installed Mira binaries on the system and add it to $PATH.

[galaxy-dev] JobManager object has no attribute dispatcher

2012-04-17 Thread Peter Cock
Hi all, Does anyone know what might have introduced this problem running galaxy-dist when using the task splitting functionality? I'm using the latest code from the default branch, changeset: 7027:f6e790d94282 galaxy.jobs.manager DEBUG 2012-04-17 13:55:03,610 (4) Job assigned to handler 'main'

Re: [galaxy-dev] Workflow Tags

2012-04-16 Thread Peter Cock
On Mon, Apr 16, 2012 at 8:12 AM, Frank Sørensen frank.soren...@ki.au.dk wrote: Hi dev-list, I have noticed, that when I clone a workflow, the new workflow apparently inherits all tags from all my workflows or something similar. The result is that when ever I clone a workflow the number of

Re: [galaxy-dev] Creating a workflow of workflows

2012-04-13 Thread Peter Cock
On Fri, Apr 13, 2012 at 1:00 AM, Aaron Gallagher agall...@fhcrc.org wrote: Hi again, I'm trying to set up a workflow for Galaxy (6799:40f1816d6857), but I can't seem to find a way to make a workflow that invokes another workflow. It seems like this wouldn't be unreasonable, since workflows

Re: [galaxy-dev] Auto delete dataset after workflow run?

2012-04-04 Thread Peter Cock
On Wed, Apr 4, 2012 at 12:30 PM, Praveen Raj Somarajan pravee...@ocimumbio.com wrote: Hello All, Is there a way to set auto delete some datafiles at the end of workflow run? I'm asking this because it is sometimes useful to delete some outputs (say intermediate files, .logs, etc), for

Re: [galaxy-dev] Fastq Groomer

2012-04-03 Thread Peter Cock
On Tue, Apr 3, 2012 at 1:57 AM, Xu, Jianpeng jianpeng...@emory.edu wrote: Hi, I have installed our local galaxy. I have a question for Fastq Groomer: Is it an interior tool within Galaxy ? I mean you already incorporate it into Galaxy?  Or, you incorporate an external tool into Galaxy ? Do

Re: [galaxy-dev] [galaxy-user] Blast2GO local instance Re: Table with gene count reads

2012-04-03 Thread Peter Cock
On Mon, Apr 2, 2012 at 6:41 PM, Greg Von Kuster g...@bx.psu.edu wrote: On Mar 24, 2012, at 7:30 AM, Peter Cock wrote: Have you seen the README file that comes with the Blast2GO wrapper? Perhaps the 'install from toolshed' could be tweaked to make this kind of documentation more visible

Re: [galaxy-dev] Missing tool previews in tool shed, e.g. hmmer.xml

2012-04-03 Thread Peter Cock
On Tue, Apr 3, 2012 at 7:25 PM, Greg Von Kuster g...@bx.psu.edu wrote: Invalid tools are now displayed in the Preview tools and inspect metadata by tool version section - here is the list for the hmmer repository. Lovely - that will help with the existing tools which are not all fully

Re: [galaxy-dev] Local display application questions

2012-04-02 Thread Peter Cock
On Mon, Apr 2, 2012 at 2:00 PM, Lukasse, Pieter pieter.luka...@wur.nl wrote: Hi Greg, I made a tool for Galaxy which produces a normal XML output file and next to that it also produces a HTML report with some quick summarized details for the user. I could solve this by using 2 output files,

Re: [galaxy-dev] [galaxy-user] Blast2GO local instance Re: Table with gene count reads

2012-03-24 Thread Peter Cock
On Fri, Mar 23, 2012 at 10:45 PM, Luciano Cosme cosme.sim...@gmail.com wrote: Howdy,    Thanks Jen, I will try it tomorrow.    I installed Blast2Go from the Toolshed in my local instance of Galaxy and when I try to run it I get the following error: Index file named 'blast2go.loc' is

Re: [galaxy-dev] Update on Megablast?

2012-03-23 Thread Peter Cock
On Fri, Mar 23, 2012 at 4:22 PM, Luobin Yang yangl...@isu.edu wrote: Hi, all, Since megablast has been replaced by blastn in newer NCBI BLAST tools, I am wondering if there is an update on megablast web interface and wrapper in Galaxy? Yes, there are wrappers for the main tools from NCBI

Re: [galaxy-dev] Fixed Thread number in certain tools

2012-03-22 Thread Peter Cock
On Thu, Mar 22, 2012 at 3:34 PM, Daniel Sobral dsob...@igc.gulbenkian.pt wrote: Hello, For certain tools, like bwa or blast, there is a non-configurable number of threads to use inside the tool xml. This is inconvenient for certain cluster configurations, where what we want is division of a

[galaxy-dev] Missing tool previews in tool shed, e.g. hmmer.xml

2012-03-19 Thread Peter Cock
Hello all, I was looking at Edward's updated HMMER wrapper on the toolshed, http://toolshed.g2.bx.psu.edu/ There is a preview offered for the (simple) hmmpress.xml only, giving the impression that Edward's repository isn't very useful. Why isn't anything shown for the more complex hmmer.xml? If

Re: [galaxy-dev] Missing tool previews in tool shed, e.g. hmmer.xml

2012-03-19 Thread Peter Cock
On Mon, Mar 19, 2012 at 1:28 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Peter, Here is the error produced by attempting to load the hmmr.xml tool config into the tool shed.  With regard to tool validity, the definition of a valid tool in the tool shed has always been restricted to the tool

Re: [galaxy-dev] Stalled upload jobs under Admin, Manage jobs

2012-03-16 Thread Peter Cock
On Mon, Feb 13, 2012 at 5:02 PM, Nate Coraor n...@bx.psu.edu wrote: On Feb 10, 2012, at 6:47 AM, Peter Cock wrote: Hello all, I've noticed we have about a dozen stalled upload jobs on our server from several users. e.g. Job ID        User    Last Update     Tool    State   Command Line

Re: [galaxy-dev] Bugs in FASTQ Quality Trimmer (version 1.0.0) on-line window

2012-03-12 Thread Peter Cock
I have a guess: In the history (right hand column) entry for your FASTQ file, click on the pencil icon to edit the attributes. Try explicitly setting the type of FASTQ, e.g. for Sanger FASTQ Galaxy uses the filetype 'fastqsanger'. This controls how the quality string is interpreted (which is

Re: [galaxy-dev] [galaxy-user] Metagenomics

2012-03-12 Thread Peter Cock
On Mon, Mar 12, 2012 at 6:28 PM, John Major john.e.major...@gmail.com wrote: A small warning re-the current cloud-Blast+ config. To properly use the metagenomic tools, if you use the blast+ galaxy tool, make sure to export in blast.XML, then you'll need a script to parse out the readID and

Re: [galaxy-dev] unable to add tools in tool_conf.xml

2012-03-08 Thread Peter Cock
On Thu, Mar 8, 2012 at 3:54 PM, Sivasangari NANDY nsi...@hotmail.com wrote: Hi friends, I've got a problem, i'm not able to add a tool in ttol_conf.xml. Actually i can see one tool that i've added but when i put the second Galaxy doesn't take it into account :   section name=My_tools

[galaxy-dev] Problem with filter tool accepting all rows

2012-03-07 Thread Peter Cock
Hi all, I just debugged a problem one of our users was having filtering BLAST tabular output using the Filter tool (version 1.1.0), where it was accepting 100% of the data. He had entered the filter condition as: 'c8-c7=0.8*c23' If you know how the tool works internally (essentially evaluating

Re: [galaxy-dev] HMMER wrappers

2012-03-06 Thread Peter Cock
On Tuesday, March 6, 2012, Edward Kirton eskir...@lbl.gov wrote: i gave it a bad name previously and peter didn't find it in a search at first, but i couldn't change the name, so i created a new repository. i put a note in the old repo (hmmscan), referring to the new repo (hmmer). that may

[galaxy-dev] Problems with Internet Explorer 9?

2012-03-02 Thread Peter Cock
Hello all, Are there (or were there) any known problems with Galaxy under Microsoft Internet Explorer 9? One of our biologists is now having problems with our local Galaxy server, probably after updating his browser. Specifically new history entries do not seem to automatically update from

Re: [galaxy-dev] HMMER wrappers

2012-02-29 Thread Peter Cock
On Wed, Feb 29, 2012 at 5:56 PM, Edward Kirton eskir...@lbl.gov wrote: hi, peter - i will fix the description and upload hmmsearch and infernal today. Great. Assuming hmmscan and hmmsearch have (almost) the same command line API, there is something to be said for presenting them as one tool in

[galaxy-dev] HMMER wrappers

2012-02-28 Thread Peter Cock
Hello all, Has anyone done a wrapper for HMMER2 or HMMER3? http://hmmer.janelia.org/software I can't find anything in the Tool Shed http://toolshed.g2.bx.psu.edu/ so we are likely to look into doing this ourselves - starting with hmmerscan. Thanks, Peter

Re: [galaxy-dev] corrupted binary file

2012-02-25 Thread Peter Cock
On Saturday, February 25, 2012, Yingwei HU wrote: Dear Peter, Thank you for your response. Yes. it is a custom binary file type. I attached the sample binary files. humanK562C_d2.splib is the original one. temp.splib is the uploaded and corrupted one. For definition in Galaxy, I just

Re: [galaxy-dev] corrupted binary file

2012-02-25 Thread Peter Cock
On Sat, Feb 25, 2012 at 9:40 AM, Yingwei HU husi...@gmail.com wrote: Dear Peter, I am sorry I do not know the version of my galaxy by using hg. I directly downloaded the zip file from the address: http://dist.g2.bx.psu.edu/galaxy-dist.tip.zip That would be fine for a one-off test

Re: [galaxy-dev] corrupted binary file

2012-02-24 Thread Peter Cock
On Fri, Feb 24, 2012 at 4:03 PM, Yingwei HU husi...@gmail.com wrote: Dear all, I found one of my binary files was corrupted after its uploading through Get data'. I used editplus to check its hex code and found some 0D is changed to 0A and also one byte is lost. May I ask how to solve this

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-23 Thread Peter Cock
On Wed, Feb 22, 2012 at 7:07 PM, dannonba...@me.com wrote: Awesome, I'll take a look.  And, if you're able to pull it together easily enough, clean branches are always nice. -Dannon It is all on one new branch, but this covers FASTA splitting (ready), splitting in the BLAST+ wrapper (ready

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-22 Thread Peter Cock
On Thu, Feb 16, 2012 at 9:02 PM, Peter wrote: On Thu, Feb 16, 2012 at 6:42 PM, Chris wrote: On Feb 16, 2012, at 12:24 PM, Peter wrote: I also need to look at merging multiple BLAST XML outputs, but this is looking promising. Yep, that's definitely one where a simple concatenation wouldn't

Re: [galaxy-dev] FastX tools

2012-02-22 Thread Peter Cock
On Tue, Feb 21, 2012 at 5:49 PM, Borrone, James james.w.borr...@okstate.edu wrote: hello, I am trying to use the FastX tools on the Main server to trim and manipulate FASTQ files extracted directly from an sff file (again using the main galaxy web server).  In using Clip and the Reverse

Re: [galaxy-dev] Using Galaxy as a Framework

2012-02-20 Thread Peter Cock
On Sat, Feb 18, 2012 at 10:40 PM, Mikal Ustad mikal.e.us...@gmail.com wrote: Hello, I am part of a senior project group at Northern Arizona University.  We are interested in adapting Galaxy for a project related to building image processing workflows.  The existing system that we are working

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-20 Thread Peter Cock
On Mon, Feb 20, 2012 at 8:08 AM, Bram Slabbinck br...@psb.vib-ugent.be wrote: Hi Dannon, If I may further elaborate on this issue, I would like to mention that this kind of functionality is also supported by the Sun Grid Engine in the form of 'array jobs'. With this functionality you can

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-17 Thread Peter Cock
On Thu, Feb 16, 2012 at 9:02 PM, Peter wrote: On Thu, Feb 16, 2012 at 6:42 PM, Chris wrote: Cool!  Seems like a perfectly fine start.  I guess you could grab the # of sequences from the dataset somehow (I'm guessing that is set somehow upon import into Galaxy). Yes, I should be able to get

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-16 Thread Peter Cock
On Wed, Feb 15, 2012 at 6:07 PM, Dannon Baker dannonba...@me.com wrote: Main still runs these jobs in the standard non-split fashion, and as a resource that is occasionally saturated (and thus doesn't necessarily have extra resources to parallelize to) will probably continue doing so as long

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-16 Thread Peter Cock
On Wed, Feb 15, 2012 at 6:07 PM, Dannon Baker dannonba...@me.com wrote: Good luck, let me know how it goes, and again - contributions are certainly welcome :) I think I found the first bug, method split in lib/galaxy/datatypes/sequence.py for class Sequence assumes four lines per sequence. This

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-16 Thread Peter Cock
On Thu, Feb 16, 2012 at 10:47 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Wed, Feb 15, 2012 at 6:07 PM, Dannon Baker dannonba...@me.com wrote: Good luck, let me know how it goes, and again - contributions are certainly welcome :) I think I found the first bug, method split in lib

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-16 Thread Peter Cock
On Thu, Feb 16, 2012 at 1:53 PM, Fields, Christopher J cjfie...@illinois.edu wrote: Makes sense from my perspective; splits have to be defined based on data type.  It could be as low-level as defining a simple iterator per record, then a wrapper that allows a specific chunk-size.  The split

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-16 Thread Peter Cock
Hi Dan, I think I need a little more advice - what is the role of the script scripts/extract_dataset_part.py and the JSON files created when splitting FASTQ files in lib/galaxy/datatypes/sequence.py, and then used by the class' process_split_file method? Why is there no JSON file created by the

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-16 Thread Peter Cock
On Thu, Feb 16, 2012 at 4:28 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Dan, I think I need a little more advice - what is the role of the script scripts/extract_dataset_part.py and the JSON files created when splitting FASTQ files in lib/galaxy/datatypes/sequence.py, and then used

[galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-15 Thread Peter Cock
Hi all, The comments on this issue suggest that the Galaxy team is/were working on splitting large jobs over multiple nodes/CPUs: https://bitbucket.org/galaxy/galaxy-central/issue/79/split-large-jobs Is there any relevant page on the wiki I should be aware of? Specifically I am hoping for a

Re: [galaxy-dev] Splitting large jobs over multiple nodes/CPUs?

2012-02-15 Thread Peter Cock
On Wed, Feb 15, 2012 at 5:08 PM, Dannon Baker dannonba...@me.com wrote: It's definitely an experimental feature at this point, and there's no wiki, but basic support for breaking jobs into tasks does exist.  It needs a lot more work and can go in a few different directions to make it better,

Re: [galaxy-dev] advanced parameters: all or nothing?

2012-02-10 Thread Peter Cock
On Thu, Feb 9, 2012 at 11:27 PM, Andrew Warren anwar...@vbi.vt.edu wrote: Hello, I was recently adjusting advanced parameters when running Tophat in Galaxy and noticed that when advanced parameters are used, every field is converted and submitted as command line parameter to the tool at

[galaxy-dev] Stalled upload jobs under Admin, Manage jobs

2012-02-10 Thread Peter Cock
Hello all, I've noticed we have about a dozen stalled upload jobs on our server from several users. e.g. Job ID UserLast Update ToolState Command LineJob Runner PID/Cluster ID 235221 hours agoupload1 upload NoneNoneNone ... 2339

[galaxy-dev] Problem with self.object_store.exists in set_dataset_sizes.py

2012-02-10 Thread Peter Cock
Hi all, I've just updated our Galaxy to the current release 26920e20157f which now includes quota support. This means I need to run set_dataset_sizes.py in order to record the current usage, as explained here: http://wiki.g2.bx.psu.edu/Admin/Disk%20Quotas However, I have run into a problem: $

Re: [galaxy-dev] advanced parameters: all or nothing?

2012-02-10 Thread Peter Cock
On Fri, Feb 10, 2012 at 4:18 PM, Jeremy Goecks jeremy.goe...@emory.edu wrote: On Thu, Feb 9, 2012 at 11:27 PM, Andrew Warren anwar...@vbi.vt.edu wrote: Hello, I was recently adjusting advanced parameters when running Tophat in Galaxy and noticed that when advanced parameters are used, every

Re: [galaxy-dev] Status on importing BAM file into Library does not update

2012-02-09 Thread Peter Cock
On Thu, Feb 9, 2012 at 2:57 AM, Fields, Christopher J cjfie...@illinois.edu wrote: Forgot to add, but this also seems tied to the same problem Ryan's describing.  IIRC Galaxy also runs 'samtools sort' after certain jobs, correct? chris This sounds like part of the BAM grooming (assuming

Re: [galaxy-dev] More helpful cluster job names

2012-02-07 Thread Peter Cock
On Mon, Feb 6, 2012 at 6:17 PM, Nate Coraor n...@bx.psu.edu wrote: On Feb 6, 2012, at 1:05 PM, Nate Coraor wrote: On Feb 3, 2012, at 7:30 AM, Peter Cock wrote: On Fri, Feb 3, 2012 at 11:55 AM, Ryan Golhar ngsbioinformat...@gmail.com wrote: 'qstat -f | grep jobname' will give you the full

Re: [galaxy-dev] Missing stdout and stderr files from SGE cluster

2012-02-03 Thread Peter Cock
On Thu, Feb 2, 2012 at 6:29 PM, Nate Coraor n...@bx.psu.edu wrote: On Feb 2, 2012, at 12:22 PM, Nate Coraor wrote: Ahh, I see what's going on here.  The drm out/err files are now being written to the job working directory, which is cleaned up at the end of the job wrapper's finish method,

[galaxy-dev] More helpful cluster job names

2012-02-03 Thread Peter Cock
Hello all, We're not planning to try the new run cluster jobs as user functionality (at least, not just yet), so for now all our SGE jobs show on the cluster via qstat or any other monitoring tool as belonging to the user galaxy. Currently the jobs are submitted as shell scripts galaxy_NNN.sh

[galaxy-dev] Missing stdout and stderr files from SGE cluster

2012-02-02 Thread Peter Cock
Hi all, I'm trying to configure Galaxy to talk to our SGE cluster via DRMAA. I've got jobs submitting, and can verify they are in the queue and then run via qstat (from the cluster). However, something isn't right. I'm testing with blastp using the following in universe_wsgi.ini

Re: [galaxy-dev] Missing stdout and stderr files from SGE cluster

2012-02-02 Thread Peter Cock
On Thu, Feb 2, 2012 at 1:54 PM, Langhorst, Brad langho...@neb.com wrote: Peter: I have a working sge setup. Two ideas: 1 - look in the sge logs on the worker node (on my system they are in /var/spool...) No sign of any SGE logs there - could be system configuration dependent, but thanks.

[galaxy-dev] Missing eggs for DRMMA?

2012-02-01 Thread Peter Cock
Hi all, I'm working on a test setup using an SGE cluster. During start of run.sh up I see: galaxy.jobs ERROR 2012-02-01 15:03:14,164 Job runner is not loadable: galaxy.jobs.runners.drmma Traceback (most recent call last): File /data/galaxy/galaxy-central/lib/galaxy/jobs/__init__.py, line 1187,

<    5   6   7   8   9   10   11   12   >