[Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Aldcroft, Thomas
The idea of a one-byte string dtype has been extensively discussed twice before, with a lot of good input and ideas, but no action [1, 2]. tl;dr: Perfect is the enemy of good. Can numpy just add a one-byte string dtype named 's' that uses latin-1 encoding as a bridge to enable Python 3 usage in

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Sturla Molden
On 22/02/15 19:21, Aldcroft, Thomas wrote: Problems like this are now showing up in the wild [3]. Workarounds are also showing up, like a way to easily convert from 'S' to 'U' within astropy Tables [4], but this is really not a desirable way to go. Gigabyte-sized string data arrays are not

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Nathaniel Smith
On Sun, Feb 22, 2015 at 11:29 AM, Sturla Molden sturla.mol...@gmail.com wrote: On 22/02/15 19:21, Aldcroft, Thomas wrote: Problems like this are now showing up in the wild [3]. Workarounds are also showing up, like a way to easily convert from 'S' to 'U' within astropy Tables [4], but this

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Robert Kern
On Sun, Feb 22, 2015 at 7:29 PM, Sturla Molden sturla.mol...@gmail.com wrote: On 22/02/15 19:21, Aldcroft, Thomas wrote: Problems like this are now showing up in the wild [3]. Workarounds are also showing up, like a way to easily convert from 'S' to 'U' within astropy Tables [4], but

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Sturla Molden
On 22/02/15 21:04, Robert Kern wrote: Python 3's `str` type is opaque, so it can freely choose how to represent the data in memory. numpy dtypes transparently describe how the data is represented in memory. Hm, yes, that is a good point. Sturla

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Nathaniel Smith
On Sun, Feb 22, 2015 at 2:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Feb 22, 2015 at 3:40 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at 2:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft,

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Aldcroft, Thomas
On Sun, Feb 22, 2015 at 5:56 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at 5:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Feb 22, 2015 at 3:40 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Nathaniel Smith
On Feb 22, 2015 3:39 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at 5:56 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at 5:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Feb 22, 2015 at 3:40 PM,

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Aldcroft, Thomas
On Sun, Feb 22, 2015 at 2:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: The idea of a one-byte string dtype has been extensively discussed twice before, with a lot of good input and ideas, but no action

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Charles R Harris
On Sun, Feb 22, 2015 at 3:40 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at 2:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: The idea of a one-byte string dtype has

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Nathaniel Smith
On Sun, Feb 22, 2015 at 2:42 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Feb 22, 2015 at 12:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: The idea of a one-byte string dtype has been

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Nathaniel Smith
On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: The idea of a one-byte string dtype has been extensively discussed twice before, with a lot of good input and ideas, but no action [1, 2]. tl;dr: Perfect is the enemy of good. Can numpy just add a one-byte

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Sturla Molden
On 22/02/15 20:57, Nathaniel Smith wrote: This is a discussion about how strings are represented as bit-patterns inside ndarrays; the internal storage representation used by 'str' is irrelevant. I thought it would be clever to just use the same internal representation as Python would choose.

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Charles R Harris
On Sun, Feb 22, 2015 at 12:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: The idea of a one-byte string dtype has been extensively discussed twice before, with a lot of good input and ideas, but no action

Re: [Numpy-discussion] One-byte string dtype: third time's the charm?

2015-02-22 Thread Aldcroft, Thomas
On Sun, Feb 22, 2015 at 5:46 PM, Charles R Harris charlesr.har...@gmail.com wrote: On Sun, Feb 22, 2015 at 3:40 PM, Aldcroft, Thomas aldcr...@head.cfa.harvard.edu wrote: On Sun, Feb 22, 2015 at 2:52 PM, Nathaniel Smith n...@pobox.com wrote: On Sun, Feb 22, 2015 at 10:21 AM, Aldcroft,

[Numpy-discussion] np.nonzero behavior with multidimensional arrays

2015-02-22 Thread Jaime Fernández del Río
This was raised in SO today: http://stackoverflow.com/questions/28663142/why-is-np-wheres-result-read-only-for-multi-dimensional-arrays/28664009 np.nonzero (and np.where for boolean arrays) behave differently for 1-D and higher dimensional arrays: In the first case, a tuple with a single