Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-03 Thread Chris Barker
On Sat, Nov 1, 2014 at 7:31 AM, Warren Weckesser warren.weckes...@gmail.com wrote: (2) Multiple arrays in a single file: ... The file contains multiple arrays. Each array is preceded by a line containing the number of rows and columns in that array. The `max_rows` argument would make it

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-02 Thread Warren Weckesser
On Sat, Nov 1, 2014 at 4:41 PM, Alexander Belopolsky ndar...@mac.com wrote: On Sat, Nov 1, 2014 at 3:15 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Is there wider interest in such an argument to `genfromtxt`? For my use-cases, `max_rows` is sufficient. I can't recall ever

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-02 Thread Alexander Belopolsky
On Sun, Nov 2, 2014 at 1:56 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Or you could just call genfromtxt() once with `max_rows=1` to skip a row. (I'm assuming that the first argument to genfromtxt is the open file object--or some other iterator--and not the filename.) That's

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-02 Thread Alexander Belopolsky
Sorry, I meant names=True, not name=None. On Sun, Nov 2, 2014 at 2:18 PM, Alexander Belopolsky ndar...@mac.com wrote: On Sun, Nov 2, 2014 at 1:56 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Or you could just call genfromtxt() once with `max_rows=1` to skip a row. (I'm assuming

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-02 Thread Warren Weckesser
On Sun, Nov 2, 2014 at 2:18 PM, Alexander Belopolsky ndar...@mac.com wrote: On Sun, Nov 2, 2014 at 1:56 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Or you could just call genfromtxt() once with `max_rows=1` to skip a row. (I'm assuming that the first argument to genfromtxt is

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-02 Thread Alexander Belopolsky
On Sun, Nov 2, 2014 at 2:32 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Still, the case of dtype=None, name=None is problematic. Suppose I want genfromtxt() to detect the column names from the 1-st row and data types from the 3-rd. How would you do that? This may sound

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-02 Thread Warren Weckesser
On 11/2/14, Alexander Belopolsky ndar...@mac.com wrote: On Sun, Nov 2, 2014 at 2:32 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Still, the case of dtype=None, name=None is problematic. Suppose I want genfromtxt() to detect the column names from the 1-st row and data types

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Warren Weckesser
On 9/24/14, Alan G Isaac alan.is...@gmail.com wrote: On 9/24/2014 2:52 PM, Jaime Fernández del Río wrote: There is a PR in github that adds a new keyword to the genfromtxt function, to limit the number of rows that actually get read in: https://github.com/numpy/numpy/pull/5103 Sorry to come

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Alan G Isaac
On 11/1/2014 10:31 AM, Warren Weckesser wrote: Alan's suggestion to use a slice is interesting, but I'd like to see a more concrete proposal for the API. For example, how does it interact with `skip_header` and `skip_footer`? How would one use it to read a file in batches? I'm probably

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Warren Weckesser
On Sat, Nov 1, 2014 at 10:54 AM, Alan G Isaac alan.is...@gmail.com wrote: On 11/1/2014 10:31 AM, Warren Weckesser wrote: Alan's suggestion to use a slice is interesting, but I'd like to see a more concrete proposal for the API. For example, how does it interact with `skip_header` and

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Alexander Belopolsky
On Sat, Nov 1, 2014 at 3:15 PM, Warren Weckesser warren.weckes...@gmail.com wrote: Is there wider interest in such an argument to `genfromtxt`? For my use-cases, `max_rows` is sufficient. I can't recall ever needing the full generality of a slice for pulling apart a text file. Does anyone

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Alan G Isaac
On 11/1/2014 4:41 PM, Alexander Belopolsky wrote: I cannot think of a situation where I would need more generality such as reading every 3rd row or rows with the given numbers. Such processing is normally done after the text data is loaded into an array. I have done this as cheaper than

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Alan G Isaac
On 11/1/2014 3:15 PM, Warren Weckesser wrote: I intended the result of `genfromtxt(..., max_rows=n)` to produce the same array as produced by `genfromtxt(...)[:n]`. I find that counterintuitive. I would first honor skip_header. Cheers, Alan ___

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Warren Weckesser
On 11/1/14, Alan G Isaac alan.is...@gmail.com wrote: On 11/1/2014 4:41 PM, Alexander Belopolsky wrote: I cannot think of a situation where I would need more generality such as reading every 3rd row or rows with the given numbers. Such processing is normally done after the text data is loaded

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-11-01 Thread Warren Weckesser
On 11/1/14, Alan G Isaac alan.is...@gmail.com wrote: On 11/1/2014 3:15 PM, Warren Weckesser wrote: I intended the result of `genfromtxt(..., max_rows=n)` to produce the same array as produced by `genfromtxt(...)[:n]`. I find that counterintuitive. I would first honor skip_header. Sorry for

Re: [Numpy-discussion] Add `nrows` to `genfromtxt`

2014-09-24 Thread Alan G Isaac
On 9/24/2014 2:52 PM, Jaime Fernández del Río wrote: There is a PR in github that adds a new keyword to the genfromtxt function, to limit the number of rows that actually get read in: https://github.com/numpy/numpy/pull/5103 Sorry to come late to this party, but it seems to me that more