Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-22 Thread Paul Anton Letnes
On 21. apr. 2012, at 00:16, Drew Frank wrote: On Fri, Apr 20, 2012 at 11:45 AM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: Oh, right. I was thinking small as in fits in L2 cache, not small as in a few

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Frédéric Bastien
Hi, I just discovered that the NA mask will modify the base ndarray object. So I read about it to find the consequences on our c code. Up to now I have fully read: http://docs.scipy.org/doc/numpy/reference/arrays.maskna.html and partially read:

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Fernando Perez
On Fri, Apr 20, 2012 at 9:49 AM, Chris Barker chris.bar...@noaa.gov wrote: I recall discossion a couple times in the past of having some special-case numpy arrays for the simple, small cases -- perhaps 1-d or 2-d C-contiguous only, for instance. That might be a better way to address the

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Dag Sverre Seljebotn
Fernando Perez fperez@gmail.com wrote: On Fri, Apr 20, 2012 at 9:49 AM, Chris Barker chris.bar...@noaa.gov wrote: I recall discossion a couple times in the past of having some special-case numpy arrays for the simple, small cases -- perhaps 1-d or 2-d C-contiguous only, for instance.

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Fernando Perez
On Fri, Apr 20, 2012 at 11:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: I don't think you gain that much by using a different type though? Those optimized code paths could be plugged into NumPy as well. Could be: this was years ago, and the bottleneck for me was in the

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Dag Sverre Seljebotn
On 04/20/2012 08:35 PM, Fernando Perez wrote: On Fri, Apr 20, 2012 at 11:27 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: I don't think you gain that much by using a different type though? Those optimized code paths could be plugged into NumPy as well. Could be: this was

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Chris Barker
On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: Oh, right. I was thinking small as in fits in L2 cache, not small as in a few dozen entries. or even two or three entries. I often use a (2,) or (3,) numpy array to represent an (x,y) point (usually pulled

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Drew Frank
On Fri, Apr 20, 2012 at 11:45 AM, Chris Barker chris.bar...@noaa.gov wrote: On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: Oh, right. I was thinking small as in fits in L2 cache, not small as in a few dozen entries. Another example of a small array

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Gael Varoquaux
On Mon, Apr 16, 2012 at 10:40:53PM -0500, Travis Oliphant wrote: The objectors object to any binary ABI change, but not specifically three pointers rather than two or one? Adding pointers is not really an ABI change (but removing them after they were there would be...) It's really just the

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 6:44 AM, Travis Oliphant tra...@continuum.io wrote: Basically, there are two sets of changes as far as I understand right now:        1) ufunc infrastructure understands masked arrays        2) ndarray grew attributes to represent masked arrays I am proposing that we

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: Mark and I will have conversations about NumPy while he is in Austin.   There are many other active stake-holders whose opinions and

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Matthew Brett
Hi, On Tue, Apr 17, 2012 at 7:24 AM, Nathaniel Smith n...@pobox.com wrote: On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: Mark and I will have conversations about NumPy while he

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Eric Firing
On 04/17/2012 08:40 AM, Matthew Brett wrote: Hi, On Tue, Apr 17, 2012 at 7:24 AM, Nathaniel Smithn...@pobox.com wrote: On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brettmatthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphanttra...@continuum.io wrote: Mark and

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Fernando Perez
On Tue, Apr 17, 2012 at 11:40 AM, Matthew Brett matthew.br...@gmail.com wrote: I'm glad to hear that discussion is happening, but please do have it on list.   If it's off list it easy for people to feel they are being bypassed, and that the public discussion is not important. I'm afraid I have

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Matthew Brett
Hi, On Tue, Apr 17, 2012 at 12:04 PM, Fernando Perez fperez@gmail.com wrote: On Tue, Apr 17, 2012 at 11:40 AM, Matthew Brett matthew.br...@gmail.com wrote: I'm glad to hear that discussion is happening, but please do have it on list.   If it's off list it easy for people to feel they are

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Fernando Perez
On Tue, Apr 17, 2012 at 12:10 PM, Matthew Brett matthew.br...@gmail.com wrote: Right - but that would be an absurd overstatement of what I said. There's no point in addressing something I didn't say and no sensible person would think.   Indeed, it makes the discussion harder. Well, in that

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Matthew Brett
On Tue, Apr 17, 2012 at 12:32 PM, Fernando Perez fperez@gmail.com wrote: On Tue, Apr 17, 2012 at 12:10 PM, Matthew Brett matthew.br...@gmail.com wrote: Right - but that would be an absurd overstatement of what I said. There's no point in addressing something I didn't say and no sensible

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Tim Cera
I have never found mailing lists good places for discussion and consensus. I think the format itself does not lend itself to involvement, carefully considered (or the ability to change) positions, or voting since all of it can be so easily lost within all of the quoting, the back and forth,

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Ralf Gommers
On Tue, Apr 17, 2012 at 12:06 AM, Travis Oliphant tra...@continuum.iowrote: There is an issue with the NumPy 1.7 release that we all need to understand. Doesn't including the missing-data attributes in the NumPy structure in a released version of NumPy basically commit to including those

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Fernando Perez
On Mon, Apr 16, 2012 at 3:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: That's the first time I've heard this. Until now, we have talked a lot about adding bitmasks and API changes, not about complete removal. My assumption was that the experimental label was enough. From Nathaniel's

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
No off list discussions have been happening material to this point. I am basically stating my view for the first time. I have delayed because I realize it is not a pleasant view and I was hoping I could end up resolving it favorably. But, it needs to be discussed before 1.7 is released.

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 4:33 PM, Travis Oliphant tra...@continuum.iowrote: No off list discussions have been happening material to this point. I am basically stating my view for the first time. I have delayed because I realize it is not a pleasant view and I was hoping I could end up

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Ralf Gommers
On Tue, Apr 17, 2012 at 12:27 AM, Fernando Perez fperez@gmail.comwrote: On Mon, Apr 16, 2012 at 3:21 PM, Ralf Gommers ralf.gomm...@googlemail.com wrote: That's the first time I've heard this. Until now, we have talked a lot about adding bitmasks and API changes, not about complete

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
The comments I have heard have been from people who haven't wanted to make them on this list. I wish they would, but I understand that not everyone wants to be drawn into a long discussion.They have not been discussions. My bias is to just move forward with what is there. After a week

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 5:17 PM, Travis Oliphant tra...@continuum.iowrote: The comments I have heard have been from people who haven't wanted to make them on this list. I wish they would, but I understand that not everyone wants to be drawn into a long discussion.They have not been

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant tra...@continuum.io wrote: I have heard from a few people that they are not excited by the growth of the NumPy data-structure by the 3 pointers needed to hold the masked-array storage.   This is especially true when there is talk to

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 6:03 PM, Matthew Brett matthew.br...@gmail.com wrote: Hi, On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant tra...@continuum.io wrote: I have heard from a few people that they are not excited by the growth of the NumPy data-structure by the 3 pointers needed to

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 7:46 PM, Travis Oliphant tra...@continuum.io wrote: On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: Hi, On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant tra...@continuum.io wrote: I have heard from a few people that they are not excited by the growth of the

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
Ralf, I wouldn't change your plans just yet for NumPy 1.7. With Mark available full time for the next few weeks, I think we will be able to make rapid progress on whatever is decided -- in fact if people are available to help but just need resources let me know off list. I just want to

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
I think the answer to this is yes, but it could be as a feature-filled sub-class (like the current numpy.ma, except in C). I'd love to hear that argument fleshed out in more detail - do you have time? My proposal here is to basically take the current github NumPy data-structure and make

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 8:46 PM, Travis Oliphant tra...@continuum.iowrote: On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: Hi, On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant tra...@continuum.io wrote: I have heard from a few people that they are not excited by the growth of

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
On Apr 16, 2012, at 11:01 PM, Charles R Harris wrote: On Mon, Apr 16, 2012 at 8:46 PM, Travis Oliphant tra...@continuum.io wrote: On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: Hi, On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant tra...@continuum.io wrote: I have heard

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: I think the answer to this is yes, but it could be as a feature-filled sub-class (like the current numpy.ma, except in C). I'd love to hear that argument fleshed out in more detail - do you have time? My

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 10:38 PM, Travis Oliphant tra...@continuum.iowrote: On Apr 16, 2012, at 11:01 PM, Charles R Harris wrote: On Mon, Apr 16, 2012 at 8:46 PM, Travis Oliphant tra...@continuum.iowrote: On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: Hi, On Mon, Apr 16, 2012

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
On Apr 16, 2012, at 11:59 PM, Matthew Brett wrote: Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant tra...@continuum.io wrote: I think the answer to this is yes, but it could be as a feature-filled sub-class (like the current numpy.ma, except in C). I'd love to hear that argument