Re: [galaxy-dev] Getting binary programs into Galaxy distribution?
John, I'll take a look at the program. There isn't a great way to do this until the dependency installation system is working. A thin python wrapper (using Cython) would be the usual trick we would use. However: have you considered just using cat? This should be completely valid for gzip (at the cost of an extra 15 bytes per source file or so for duplicate headers). It looks like gzjoin does require decompression of all input data so this trade off may be worthwhile. On May 24, 2011, at 3:13 PM, Duddy, John wrote: There is a C program for merging Gzip files (gzjoin) that I’d love to rely on for a core Galaxy capability. Is there a standard way to get things like this included in Galaxy? Recoding it in Python would be a bit of a pain, and might be a lot slower due to the IO layer not allowing the reuse of buffers. Thanks - John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jdu...@illumina.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] new galaxy-dist fixes multiple IE UI bugs
Thanks Galaxy Dev Team, the new galaxy-dist fixes multiple UI bugs with IE, for e.g. the workflow editor works now works and the history expand/collapse works! :) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy BLAST XML to tabular fix
On Wed, May 18, 2011 at 6:40 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Kanwei, Could you merge this branch or transplant this fix please? https://bitbucket.org/peterjc/galaxy-central/changeset/3c0b903c86a9 https://bitbucket.org/peterjc/galaxy-central/src/blast_2011_05_18 New branch, blast_2011_05_18 There is no change to the logic, just to the assert statement where the sanity test didn't consider doubly masked sequences. Should I also update the XML wrapper's version number as well? Thanks, Peter Hi Kanwei, Did this email reach you? Let me know if you want a new unit test to go with it and/or to bump the tool's version number. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Problem with file's extension
Hi All, I have some problem with extension of file.some executables file will recognize the only the specific extension like autodock . it takes one input file ( extension with .dpf ) and one outpt file (extension with .dlg). how can we resolve such type of problem . Regards Shashi shekhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy BLAST XML to tabular fix
Sorry, it slipped by. Committed! On Wed, May 25, 2011 at 12:48 PM, Peter Cock p.j.a.c...@googlemail.comwrote: On Wed, May 18, 2011 at 6:40 PM, Peter Cock p.j.a.c...@googlemail.com wrote: Hi Kanwei, Could you merge this branch or transplant this fix please? https://bitbucket.org/peterjc/galaxy-central/changeset/3c0b903c86a9 https://bitbucket.org/peterjc/galaxy-central/src/blast_2011_05_18 New branch, blast_2011_05_18 There is no change to the logic, just to the assert statement where the sanity test didn't consider doubly masked sequences. Should I also update the XML wrapper's version number as well? Thanks, Peter Hi Kanwei, Did this email reach you? Let me know if you want a new unit test to go with it and/or to bump the tool's version number. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] new galaxy-dist fixes multiple IE UI bugs
Thank YOU for reporting them ;) On Wed, May 25, 2011 at 11:33 AM, Leandro Hermida soft...@leandrohermida.com wrote: Thanks Galaxy Dev Team, the new galaxy-dist fixes multiple UI bugs with IE, for e.g. the workflow editor works now works and the history expand/collapse works! :) ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem with file's extension
Hi Shashi, You can add your own datatypes using the instructions here: https://bitbucket.org/galaxy/galaxy-central/wiki/AddingDatatypes Thanks, K On Wed, May 25, 2011 at 1:05 PM, shashi shekhar meshash...@gmail.comwrote: Hi All, I have some problem with extension of file.some executables file will recognize the only the specific extension like autodock . it takes one input file ( extension with .dpf ) and one outpt file (extension with .dlg). how can we resolve such type of problem . Regards Shashi shekhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Problem with file's extension
Shashi, All of Galaxy's datasets have the extension .dat so your problem is not uncommon. The general solution to this is to create temp files that have the the required extensions for the tool and point them to/ from the actual Galaxy input and output. There are several ways to do this. If you have a wrapper script, you can do it there. See the BWA wrapper for an example in a Python wrapper (tools/sr_mapping/ bwa_wrapper.py, especially lines 223-8). If you have only the XML config file, you can do it there. For an example of this, see the Freebayes tool (tools/human_genome_variation/freebayes.xml). This links two input files to specifically named files. Let us know if you have further questions. Kelly On Wed May 25, at 1:05 PM, shashi shekhar wrote: Hi All, I have some problem with extension of file.some executables file will recognize the only the specific extension like autodock . it takes one input file ( extension with .dpf ) and one outpt file (extension with .dlg). how can we resolve such type of problem . Regards Shashi shekhar ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy BLAST XML to tabular fix
On Wed, May 25, 2011 at 1:10 PM, Kanwei Li kan...@gmail.com wrote: Sorry, it slipped by. Committed! No problem - thank you, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] multiple cluster job runners?
Shantanu Pavgi wrote:
On May 24, 2011, at 4:49 AM, Nate Coraor wrote:
Shantanu Pavgi wrote:
I am looking at options to configure settings such as wall-time, memory,
and number of cores etc. for each tool and wondering if this can be done
using multiple cluster job runner configurations. Is it possible to
define multiple cluster job runner configurations for each tool? In the
universe_wsgi.ini file it is mentioned that individual per-tool job
runners can be specified, however I don't see any example other than
'default_cluster_job_runner'. So is there any key-value syntax for
defining it in the ini file? I appreciate any comments or suggestions
regarding this issue.
Hi Shantanu,
Please have a look at the examples in the [galaxy:tool_runners] section
at the bottom of universe_wsgi.ini and let us know if this suits your
needs.
Yes, that should work for me.
I was wondering if I need to define job runners first and then specify it for
each tool: e.g.
{{{
$job_runner_one = drmaa:// -V /
$job_runner_two = drmaa:// -V -l h_rt=00:10:00 /
...
...
$galaxy_tool_one = $job_runner_one
...
}}}
The syntax is different, but this should be possible. See the
documentation on interpolation here:
http://docs.python.org/library/configparser.html
--nate
But current configuration syntax should work. Thanks for the help.
--
Shantanu.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] cluster files directory configuration
Shantanu Pavgi wrote: On May 19, 2011, at 4:05 PM, Shantanu Pavgi wrote: I have configured custom cluster_files_directory in universe_wsgi.ini file. The cluster jobs are running fine, however job scripts, stdout and stderr are being written in default database/pbs location. Am I missing something in the configuration here? Any help on what might be wrong here? Is this option getting overridden by some other config? SGE or DRMAA? I think this was broken in the dist version of Galaxy prior to the one just released on Friday - try the latest and let us know if there's still an issue. --nate -- Thanks, Shantanu. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Getting binary programs into Galaxy distribution?
I had considered cat, but from what I have read, not all readers understand it, or understand it as a single compressed stream. Since the resulting file would differ from a block gzipped file (embedded gzip headers with filenames, embedded per-file trailers, and the final CRC/size not matching the entire contents) I worry cat would be a brittle solution. Also, single AFAIK gzjoin is not part of any installable package (more of an example program) I don't know if it's something that can be addressed by a dependency installation system, unless you host the installations. There is stuff that happens on first start, such as copying *example files. Would it be far-fetched to compile the program at that stage? From: James Taylor [ja...@jamestaylor.org] Sent: Tuesday, May 24, 2011 11:51 PM To: Duddy, John Cc: galaxy-...@bx.psu.edu Dev Subject: Re: [galaxy-dev] Getting binary programs into Galaxy distribution? John, I'll take a look at the program. There isn't a great way to do this until the dependency installation system is working. A thin python wrapper (using Cython) would be the usual trick we would use. However: have you considered just using cat? This should be completely valid for gzip (at the cost of an extra 15 bytes per source file or so for duplicate headers). It looks like gzjoin does require decompression of all input data so this trade off may be worthwhile. On May 24, 2011, at 3:13 PM, Duddy, John wrote: There is a C program for merging Gzip files (gzjoin) that I’d love to rely on for a core Galaxy capability. Is there a standard way to get things like this included in Galaxy? Recoding it in Python would be a bit of a pain, and might be a lot slower due to the IO layer not allowing the reuse of buffers. Thanks - John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jdu...@illumina.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] NGS index files not showing up
Hi, I've installed galaxy locally on my Mac, I've got everything running fine except Bowtie/Tophat/etc. indexes. I tried following the steps here: https://bitbucket.org/galaxy/galaxy-central/wiki/NGSLocalSetup So my bowtie_indices.loc file looks like this: e_coli /Users/jel65/galaxy-central/tool-data/e_coli (no quotes) (I was just testing the e_coli index that came with bowtie) So I saved it, restarted the galaxy server, but still nothing shows up in the drop down. Did I miss a step? I've read the documentation multiple times. Thanks Jeremy ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Getting binary programs into Galaxy distribution?
Sounds like we'll have to do a phased solution. I'll see if cat works well enough for tool interoperability for now and see what we can do to make it better (if needed) later. Thanks, guys! John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jdu...@illumina.com -Original Message- From: Nate Coraor [mailto:n...@bx.psu.edu] Sent: Wednesday, May 25, 2011 9:24 AM To: Duddy, John Cc: James Taylor; galaxy-...@bx.psu.edu Dev Subject: Re: [galaxy-dev] Getting binary programs into Galaxy distribution? Duddy, John wrote: I had considered cat, but from what I have read, not all readers understand it, or understand it as a single compressed stream. Since the resulting file would differ from a block gzipped file (embedded gzip headers with filenames, embedded per-file trailers, and the final CRC/size not matching the entire contents) I worry cat would be a brittle solution. Also, single AFAIK gzjoin is not part of any installable package (more of an example program) I don't know if it's something that can be addressed by a dependency installation system, unless you host the installations. There is stuff that happens on first start, such as copying *example files. Would it be far-fetched to compile the program at that stage? We probably can't make the assumption that the user has the zlib dev package with the headers, or even a C compiler, but this is a general problem that we will need to solve as we implement tight dependency control. My initial thought on this is automatic fetching of precompiled binaries, and if not available for your platform then we can try to automatically compile. Although really, I would be surprised if anyone is running tools on anything other than Mac or Linux. For the immediate solution, we could probably fetch the binary much like we do for eggs. --nate From: James Taylor [ja...@jamestaylor.org] Sent: Tuesday, May 24, 2011 11:51 PM To: Duddy, John Cc: galaxy-...@bx.psu.edu Dev Subject: Re: [galaxy-dev] Getting binary programs into Galaxy distribution? John, I'll take a look at the program. There isn't a great way to do this until the dependency installation system is working. A thin python wrapper (using Cython) would be the usual trick we would use. However: have you considered just using cat? This should be completely valid for gzip (at the cost of an extra 15 bytes per source file or so for duplicate headers). It looks like gzjoin does require decompression of all input data so this trade off may be worthwhile. On May 24, 2011, at 3:13 PM, Duddy, John wrote: There is a C program for merging Gzip files (gzjoin) that I’d love to rely on for a core Galaxy capability. Is there a standard way to get things like this included in Galaxy? Recoding it in Python would be a bit of a pain, and might be a lot slower due to the IO layer not allowing the reuse of buffers. Thanks - John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jdu...@illumina.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Is dynamic associated information per dataset possible?
We'd like to be able to associate fixed things (project, Sample, sequencer used) with user's FASTQ files, and we'd also like to allow users to associate dynamic, site-specific stuff with the sequencing run. Currently, users track their runs using a CSV sample sheet, and often they add columns to that sample sheet for their own information. Is it possible to associate that information with the FASTQ file when it is placed in Galaxy? I know about metadata, but the supported fields look like they are fixed in the code. I was hoping for a solution where the users do not need to modify the Galaxy code to pull this off. John Duddy Sr. Staff Software Engineer Illumina, Inc. 9885 Towne Centre Drive San Diego, CA 92121 Tel: 858-736-3584 E-mail: jdu...@illumina.commailto:jdu...@illumina.com ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Error getting to http://galaxy/user/create on local instance
Greg. We do not use the template_cache_path in our config. I deleted all the files in ~/database/compiled_templates and restarted my server. I still get the error (even with a shift reload of my browser). I've tried it (on the Mac) with Firefox 3.5.2, Safari 5.0.5 and Google Chrome 11.0.696.71 We run two totally separate instances of Galaxy, on different virtual machines. One for testing, development of tools and deploying new versions of galaxy, and one for production, where our scientists do their analysis. Each of these servers run 4 instances of the galaxy process. 3 Web applications and 1 job runner. We get the error on both servers, but I'm doing all of this troubleshooting on our development server. This behavior is occuring for any user who tries to create a new galaxy account (so the original user who reported the problem, and myself as I test it. I can ask others to try if you think that will help). I mentioned the browsers I've tried it with above. I've not tried it from a windows box, but can, again, if you think that will help. We are running whatever version of galaxy-dist was available about a week and a half ago, which is when Glen did the upgrade. Thanks, Dave On 5/24/11 11:29 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hi Dave, If you do not have a template_cache_path config setting in universe_wsgi.ini, then your cached templates are stored in the default directory of ~/database/compiled_templates. Deleting your cached templates in ~/database/compiled_templates should have corrected the problem, assuming there are no other issues within your Galaxy environment, so let's begin down the path of determining the cause. You mention your development server in your response. Are you running only 1 Galaxy instance, or more than 1? Is the behavior occurring for all of your users or just a few? Does it occur with different browsers, or is it related to a certain browser brand? What version of Galaxy are your running? Thanks Dave, Greg Von Kuster On May 24, 2011, at 6:06 PM, Dave Walton wrote: Greg, We've tried just reloading the browser and that didn't do anything. As for deleting the cached templates, on our development server I tried deleting everything under the database/compiled_templates directory (I even restarted my server) and it appeared to have no affect. We still get the error message. Maybe I misunderstood you. The only cache related variable in my universe_wsgi files is static_cache_time = 360. Thanks, Dave On 5/24/11 4:27 PM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Dave, The index method signature of the user controller changed, so this is a template cache issue. It may work to simply have your users refresh their Galaxy masthead ( the top menu banner that includes the login in the User pop-up ). However, sometimes this isn't enough to clear the cache, so you can fix things for sure by deleting the cached templates in ~//database/compiled_templates/user assuming you have not changed the setting for the template cache in your universe_wsgi.ini file. If you have, delete the files in the user directory where you have them stored. Greg Von Kuster On May 24, 2011, at 4:02 PM, Dave Walton wrote: We upgraded our Galaxy instance to the current Dist build at the end of last week. Since then we've had a couple new users attempt to create accounts. When they go to the create user page (http://galaxy.jax.org/user/create) they get a 500 Server Error. The message in the error log is: 10.30.34.254 - - [24/May/2011:15:54:24 -0400] GET /user/create HTTP/1.0 500 - http://galaxy/user/login; Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 Error - type 'exceptions.TypeError': create() takes at least 3 arguments (2 given) Glen is traveling to the Galaxy Community Conference, he's the one that did the upgrade, but as far as I know, he did a pretty straight forward upgrade. Did something change, in configuration of the create user page? Any thoughts what might be causing this? Thanks, Dave ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu Greg Von Kuster Galaxy Development Team g...@bx.psu.edu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
