Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
On Jul 8, 2013, at 3:33 PM, Peter Cock wrote: > On Mon, Jul 8, 2013 at 11:21 PM, Robert Baertsch wrote: >> I respectfully disagree, If you want an extensible system, you should >> always wrap primitive system level calls. >> >> Any tools that opens a file that could be compressed would be affe

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
On Jul 8, 2013, at 3:33 PM, Peter Cock wrote: > On Mon, Jul 8, 2013 at 11:21 PM, Robert Baertsch wrote: >> I respectfully disagree, If you want an extensible system, you should >> always wrap primitive system level calls. >> >> Any tools that opens a file that could be compressed would be affe

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
I will implement this if the galaxy team likes the approach. We did this in ucsc genome browser code years ago: a single open_helper call handles, gzip, http, ftp and pipes. No need to care about how the data is compressed or where it data resides. wouldn't it be great to be able to pipe data

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
great. Let's put the bx-python calls in a galaxy_open helper function. On Jul 8, 2013, at 3:20 PM, James Taylor wrote: > open_compressed in bx-python does this already (for bz2 as well). > > On Jul 8, 2013, at 5:58 PM, Peter Cock wrote: > >> On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch >>

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
great. Let's put the bx-python calls in a galaxy_open helper function. On Jul 8, 2013, at 3:20 PM, James Taylor wrote: > open_compressed in bx-python does this already (for bz2 as well). > > On Jul 8, 2013, at 5:58 PM, Peter Cock wrote: > >> On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch >>

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Peter Cock
On Tue, Jul 9, 2013 at 5:53 PM, Robert Baertsch wrote: > On Jul 8, 2013, at 3:33 PM, Peter Cock wrote: >> The tools available in Galaxy are written in a range >> of languages including C, Perl, R, etc. Yes, some are in Python, >> but of those most are independent of Galaxy and can be used >> separ

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
I respectfully disagree, If you want an extensible system, you should always wrap primitive system level calls. Any tools that opens a file that could be compressed would be affected. That is a huge number of tools. Do you really want a cottage industry of tools that have different methods of

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread Robert Baertsch
Peter and Dan, I like the idea of replacing all open() with galaxy_open() in all tools. You can tell the format by looking at the first 4 byes (see C code below from the UCSC browser team). Is there some pythonic way of overriding open? You need to read the first four bytes of the file to see if

Re: [galaxy-dev] gzipped fastq reader

2013-07-09 Thread James Taylor
open_compressed in bx-python does this already (for bz2 as well). On Jul 8, 2013, at 5:58 PM, Peter Cock wrote: > On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch > wrote: >> Peter and Dan, >> I like the idea of replacing all open() with galaxy_open() in all tools. You >> can tell the format by

Re: [galaxy-dev] gzipped fastq reader

2013-07-08 Thread Peter Cock
On Mon, Jul 8, 2013 at 11:21 PM, Robert Baertsch wrote: > I respectfully disagree, If you want an extensible system, you should > always wrap primitive system level calls. > > Any tools that opens a file that could be compressed would be affected. > That is a huge number of tools. Do you really w

Re: [galaxy-dev] gzipped fastq reader

2013-07-08 Thread Peter Cock
On Mon, Jul 8, 2013 at 10:24 PM, Robert Baertsch wrote: > Peter and Dan, > I like the idea of replacing all open() with galaxy_open() in all tools. You > can tell the format by looking at the first 4 byes (see C code below from > the UCSC browser team). Is there some pythonic way of overriding ope

Re: [galaxy-dev] gzipped fastq reader

2013-07-08 Thread Peter Cock
On Thu, Jul 4, 2013 at 9:49 PM, Robert Baertsch wrote: > Dan, > Do these readers support gzip files? > >reader = fastqVerboseErrorReader > reader = fastqReader Presumably you are writing a Python script using this library? The answer is a qualified yes. Instead of passing them a n

[galaxy-dev] gzipped fastq reader

2013-07-07 Thread Robert Baertsch
Dan, Do these readers support gzip files? reader = fastqVerboseErrorReader reader = fastqReader Do I have to define a special type in galaxy for gzipped files or will the fastq type be ok? Ideally, I would like to keep my files zipped and not have galaxy unzip them, since they t