Re: [Numpy-discussion] deprecate fromstring() for text reading?

On 3 Nov 2015, at 6:03 pm, Chris Barker - NOAA Federal wrote: > > I was more aiming to point out a situation where the NumPy's text file reader > was significantly better than the Pandas version, so we would want to make > sure that we properly benchmark any significant

Re: [Numpy-discussion] deprecate fromstring() for text reading?

Correct, there were entries that would sometimes take up their entire width. The delimited text readers could not read this particular dataset. The dataset I am referring to is the processed ISD data: https://www.ncdc.noaa.gov/isd As for fromstring() not being able to help there, I didn't mean to

Re: [Numpy-discussion] deprecate fromstring() for text reading?

I was more aiming to point out a situation where the NumPy's text file reader was significantly better than the Pandas version, so we would want to make sure that we properly benchmark any significant changes to NumPy's text reading code. Who knows where else NumPy beats Pandas? Indeed. For this

Re: [Numpy-discussion] deprecate fromstring() for text reading?

On Tue, Oct 27, 2015 at 7:30 AM, Benjamin Root wrote: > FWIW, when I needed a fast Fixed Width reader > was there potentially no whitespace between fields in that case? In which case, it really isn a different use-case than delimited text -- if it's at all common, a

Re: [Numpy-discussion] deprecate fromstring() for text reading?

FWIW, when I needed a fast Fixed Width reader for a very large dataset last year, I found that np.genfromtext() was faster than pandas' read_fwf(). IIRC, pandas' text reading code fell back to pure python for fixed width scenarios. On Fri, Oct 23, 2015 at 8:22 PM, Chris Barker - NOAA Federal <

Re: [Numpy-discussion] deprecate fromstring() for text reading?

On Thu, Oct 22, 2015 at 5:47 PM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: > > I think it would be good to keep the usage to read binary data at least. > > > Agreed -- it's only the text file reading I'm proposing to deprecate. It > was kind of weird to cram it in there in the

Re: [Numpy-discussion] deprecate fromstring() for text reading?

Grabbing the pandas csv reader would be great, and I hope it happens sooner than later, though alas, I haven't the spare cycles for it either. In the meantime though, can we put a deprecation Warning in when using fromstring() on text files? It's really pretty broken. -Chris On Oct 23, 2015, at

Re: [Numpy-discussion] deprecate fromstring() for text reading?

> On Oct 23, 2015, at 6:13 PM, Charles R Harris > wrote: > > > >> On Thu, Oct 22, 2015 at 5:47 PM, Chris Barker - NOAA Federal >> wrote: >> >>> I think it would be good to keep the usage to read binary data at least. >> >> Agreed --

Re: [Numpy-discussion] deprecate fromstring() for text reading?

On Oct 23, 2015 3:30 PM, "Jeff Reback" wrote: > > On Oct 23, 2015, at 6:13 PM, Charles R Harris wrote: > >> >> >> On Thu, Oct 22, 2015 at 5:47 PM, Chris Barker - NOAA Federal < chris.bar...@noaa.gov> wrote: >>> >>> I think it would be good to

Re: [Numpy-discussion] deprecate fromstring() for text reading?

> On Oct 23, 2015, at 6:49 PM, Nathaniel Smith wrote: > > On Oct 23, 2015 3:30 PM, "Jeff Reback" wrote: > > > > On Oct 23, 2015, at 6:13 PM, Charles R Harris > > wrote: > > > >> > >> > >> On Thu, Oct 22, 2015 at 5:47 PM, Chris

Re: [Numpy-discussion] deprecate fromstring() for text reading?

I think it would be good to keep the usage to read binary data at least. Or is there a good alternative to `np.fromstring(, dtype=...)`? -- Marten On Thu, Oct 22, 2015 at 1:03 PM, Chris Barker wrote: > There was just a question about a bug/issue with scipy.fromstring

Re: [Numpy-discussion] deprecate fromstring() for text reading?

I think it would be good to keep the usage to read binary data at least. Agreed -- it's only the text file reading I'm proposing to deprecate. It was kind of weird to cram it in there in the first place. Oh, fromfile() has the same issues. Chris Or is there a good alternative to

[Numpy-discussion] deprecate fromstring() for text reading?

There was just a question about a bug/issue with scipy.fromstring (which is numpy.fromstring) when used to read integers from a text file. https://mail.scipy.org/pipermail/scipy-user/2015-October/036746.html fromstring() is bugging and inflexible for reading text files -- and it is a very, very