Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread James Taylor
open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch robert.baert...@gmail.com wrote: Peter and Dan, I like the idea of replacing all open() with

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
Peter and Dan, I like the idea of replacing all open() with galaxy_open() in all tools. You can tell the format by looking at the first 4 byes (see C code below from the UCSC browser team). Is there some pythonic way of overriding open? You need to read the first four bytes of the file to see

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Peter Cock
On Tue, Jul 9, 2013 at 5:53 PM, Robert Baertsch rbaer...@ucsc.edu wrote: On Jul 8, 2013, at 3:33 PM, Peter Cock wrote: The tools available in Galaxy are written in a range of languages including C, Perl, R, etc. Yes, some are in Python, but of those most are independent of Galaxy and can be

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
great. Let's put the bx-python calls in a galaxy_open helper function. On Jul 8, 2013, at 3:20 PM, James Taylor wrote: open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Jul 8, 2013 at 10:24 PM,

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
great. Let's put the bx-python calls in a galaxy_open helper function. On Jul 8, 2013, at 3:20 PM, James Taylor wrote: open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock p.j.a.c...@googlemail.com wrote: On Mon, Jul 8, 2013 at 10:24 PM,

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
I will implement this if the galaxy team likes the approach. We did this in ucsc genome browser code years ago: a single open_helper call handles, gzip, http, ftp and pipes. No need to care about how the data is compressed or where it data resides. wouldn't it be great to be able to pipe

Re: [galaxy-dev] gzipped fastq reader

2013-07-08 Thread Peter Cock
On Thu, Jul 4, 2013 at 9:49 PM, Robert Baertsch robert.baert...@gmail.com wrote: Dan, Do these readers support gzip files? reader = fastqVerboseErrorReader reader = fastqReader Presumably you are writing a Python script using this library? The answer is a qualified yes.

Re: [galaxy-dev] gzipped fastq reader

2013-07-08 Thread Peter Cock
On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch robert.baert...@gmail.com wrote: Peter and Dan, I like the idea of replacing all open() with galaxy_open() in all tools. You can tell the format by looking at the first 4 byes (see C code below from the UCSC browser team). Is there some pythonic

Re: [galaxy-dev] gzipped fastq reader

2013-07-08 Thread Peter Cock
On Mon, Jul 8, 2013 at 11:21 PM, Robert Baertsch rbaer...@ucsc.edu wrote: I respectfully disagree, If you want an extensible system, you should always wrap primitive system level calls. Any tools that opens a file that could be compressed would be affected. That is a huge number of tools. Do