Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Ralf Gommers
On Wed, Dec 14, 2011 at 4:11 PM, Bruce Southey bsout...@gmail.com wrote: ** On 12/14/2011 01:03 AM, Chris Barker wrote: On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: genfromtxt sure looks close for an API This I don't agree with. It has a huge

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Benjamin Root
On Wed, Dec 14, 2011 at 1:22 PM, Ralf Gommers ralf.gomm...@googlemail.comwrote: On Wed, Dec 14, 2011 at 4:11 PM, Bruce Southey bsout...@gmail.com wrote: ** On 12/14/2011 01:03 AM, Chris Barker wrote: On Tue, Dec 13, 2011 at 1:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote:

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Chris Barker
On Wed, Dec 14, 2011 at 11:36 AM, Benjamin Root ben.r...@ou.edu wrote: well, yes, though it does do a lot -- do you have a smpler one in mind? Just looking at what I normally wouldn't need for simple data files and/or what a beginning user won't understand at once, the `unpack` and `ndmin`

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-14 Thread Ralf Gommers
On Wed, Dec 14, 2011 at 9:54 PM, Chris Barker chris.bar...@noaa.gov wrote: On Wed, Dec 14, 2011 at 11:36 AM, Benjamin Root ben.r...@ou.edu wrote: well, yes, though it does do a lot -- do you have a smpler one in mind? Just looking at what I normally wouldn't need for simple data files

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Chris Barker
NOTE: Let's keep this on the list. On Tue, Dec 13, 2011 at 9:19 AM, denis denis-bz...@t-online.de wrote: Chris, unified, consistent save / load is a nice goal 1) header lines with date, pwd etc.: where'd this come from ? # (5, 5) svm.py bz/py/ml/svm 2011-12-13 Dec 11:56 --

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Bruce Southey
On 12/13/2011 12:08 PM, Chris Barker wrote: NOTE: Let's keep this on the list. On Tue, Dec 13, 2011 at 9:19 AM, denis denis-bz...@t-online.de mailto:denis-bz...@t-online.de wrote: Chris, unified, consistent save / load is a nice goal 1) header lines with date, pwd etc.:

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Chris Barker
On Tue, Dec 13, 2011 at 11:29 AM, Bruce Southey bsout...@gmail.com wrote: ** Reading data is hard and writing code that suits the diversity in the Numerical Python community is even harder! yup Both loadtxt and genfromtxt functions (other functions are perhaps less important) perhaps need

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Ralf Gommers
On Tue, Dec 13, 2011 at 10:07 PM, Chris Barker chris.bar...@noaa.govwrote: On Tue, Dec 13, 2011 at 11:29 AM, Bruce Southey bsout...@gmail.comwrote: ** Reading data is hard and writing code that suits the diversity in the Numerical Python community is even harder! yup Both loadtxt and

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-13 Thread Wes McKinney
On Mon, Dec 12, 2011 at 12:34 PM, Warren Weckesser warren.weckes...@enthought.com wrote: On Mon, Dec 12, 2011 at 10:22 AM, Chris.Barker chris.bar...@noaa.gov wrote: On 12/11/11 8:40 AM, Ralf Gommers wrote: On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker chris.bar...@noaa.gov     * If we

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-12 Thread Chris.Barker
On 12/11/11 8:40 AM, Ralf Gommers wrote: On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker chris.bar...@noaa.gov * If we have a good, fast ascii (or unicode?) to array reader, hopefully it could be leveraged for use in the more complex cases. So that rather than genfromtxt() being

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-12 Thread Warren Weckesser
On Mon, Dec 12, 2011 at 10:22 AM, Chris.Barker chris.bar...@noaa.govwrote: On 12/11/11 8:40 AM, Ralf Gommers wrote: On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker chris.bar...@noaa.gov * If we have a good, fast ascii (or unicode?) to array reader, hopefully it could be leveraged for

Re: [Numpy-discussion] Fast Reading of ASCII files

2011-12-11 Thread Ralf Gommers
On Wed, Dec 7, 2011 at 7:50 PM, Chris.Barker chris.bar...@noaa.gov wrote: Hi folks, This is a continuation of a conversation already started, but i gave it a new, more appropriate, thread and subject. On 12/6/11 2:13 PM, Wes McKinney wrote: we should start talking about building a *high

[Numpy-discussion] Fast Reading of ASCII files

2011-12-07 Thread Chris.Barker
Hi folks, This is a continuation of a conversation already started, but i gave it a new, more appropriate, thread and subject. On 12/6/11 2:13 PM, Wes McKinney wrote: we should start talking about building a *high performance* flat file loading solution with good column type inference and