Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-22 Thread Paul Anton Letnes
On 21. apr. 2012, at 00:16, Drew Frank wrote: > On Fri, Apr 20, 2012 at 11:45 AM, Chris Barker wrote: >> >> On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn >> wrote: >>> Oh, right. I was thinking "small" as in "fits in L2 cache", not small as >>> in a few dozen entries. > > Another exa

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Drew Frank
On Fri, Apr 20, 2012 at 11:45 AM, Chris Barker wrote: > > On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn > wrote: > > Oh, right. I was thinking "small" as in "fits in L2 cache", not small as > > in a few dozen entries. Another example of a small array use-case: I've been using numpy for

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Chris Barker
On Fri, Apr 20, 2012 at 11:39 AM, Dag Sverre Seljebotn wrote: > Oh, right. I was thinking "small" as in "fits in L2 cache", not small as > in a few dozen entries. or even two or three entries. I often use a (2,) or (3,) numpy array to represent an (x,y) point (usually pulled out from a Nx2 array

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Dag Sverre Seljebotn
On 04/20/2012 08:35 PM, Fernando Perez wrote: > On Fri, Apr 20, 2012 at 11:27 AM, Dag Sverre Seljebotn > wrote: >> >> I don't think you gain that much by using a different type though? Those >> optimized code paths could be plugged into NumPy as well. > > Could be: this was years ago, and the bo

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Fernando Perez
On Fri, Apr 20, 2012 at 11:27 AM, Dag Sverre Seljebotn wrote: > > I don't think you gain that much by using a different type though? Those > optimized code paths could be plugged into NumPy as well. Could be: this was years ago, and the bottleneck for me was in the constructor and in basic arith

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Dag Sverre Seljebotn
Fernando Perez wrote: >On Fri, Apr 20, 2012 at 9:49 AM, Chris Barker >wrote: >> >> I recall discossion a couple times in the past of having some >> special-case numpy arrays for the simple, small cases -- perhaps 1-d >> or 2-d C-contiguous only, for instance. That might be a better way to >> a

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Fernando Perez
On Fri, Apr 20, 2012 at 9:49 AM, Chris Barker wrote: > > I recall discossion a couple times in the past of having some > special-case numpy arrays for the simple, small cases -- perhaps 1-d > or 2-d C-contiguous only, for instance. That might be a better way to > address the small-array performanc

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Chris Barker
On Mon, Apr 16, 2012 at 7:46 PM, Travis Oliphant wrote: > As Chuck points out, 3 more pointers is not necessarily that big of a deal if > you are talking about a large array (though for small arrays it could matter). yup -- for the most part, numpy arrays are best for workign with large data set

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-20 Thread Frédéric Bastien
Hi, I just discovered that the NA mask will modify the base ndarray object. So I read about it to find the consequences on our c code. Up to now I have fully read: http://docs.scipy.org/doc/numpy/reference/arrays.maskna.html and partially read: https://github.com/numpy/numpy/blob/master/doc/nep

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Tim Cera
I have never found mailing lists good places for discussion and consensus. I think the format itself does not lend itself to involvement, carefully considered (or the ability to change) positions, or voting since all of it can be so easily lost within all of the quoting, the back and forth, people

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Matthew Brett
On Tue, Apr 17, 2012 at 12:32 PM, Fernando Perez wrote: > On Tue, Apr 17, 2012 at 12:10 PM, Matthew Brett > wrote: >> Right - but that would be an absurd overstatement of what I said. >> There's no point in addressing something I didn't say and no sensible >> person would think.   Indeed, it mak

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Fernando Perez
On Tue, Apr 17, 2012 at 12:10 PM, Matthew Brett wrote: > Right - but that would be an absurd overstatement of what I said. > There's no point in addressing something I didn't say and no sensible > person would think.   Indeed, it makes the discussion harder. Well, in that case neither Eric Firing

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Matthew Brett
Hi, On Tue, Apr 17, 2012 at 12:04 PM, Fernando Perez wrote: > On Tue, Apr 17, 2012 at 11:40 AM, Matthew Brett > wrote: >> I'm glad to hear that discussion is happening, but please do have it >> on list.   If it's off list it easy for people to feel they are being >> bypassed, and that the publi

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Fernando Perez
On Tue, Apr 17, 2012 at 11:40 AM, Matthew Brett wrote: > I'm glad to hear that discussion is happening, but please do have it > on list.   If it's off list it easy for people to feel they are being > bypassed, and that the public discussion is not important. I'm afraid I have to disagree: you see

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Eric Firing
On 04/17/2012 08:40 AM, Matthew Brett wrote: > Hi, > > On Tue, Apr 17, 2012 at 7:24 AM, Nathaniel Smith wrote: >> On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett >> wrote: >>> Hi, >>> >>> On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant >>> wrote: Mark and I will have conversations about N

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Matthew Brett
Hi, On Tue, Apr 17, 2012 at 7:24 AM, Nathaniel Smith wrote: > On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett > wrote: >> Hi, >> >> On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant wrote: >>> Mark and I will have conversations about NumPy while he is in Austin.   >>> There are many other activ

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 5:59 AM, Matthew Brett wrote: > Hi, > > On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant wrote: >> Mark and I will have conversations about NumPy while he is in Austin.   >> There are many other active stake-holders whose opinions and views are >> essential for major cha

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-17 Thread Nathaniel Smith
On Tue, Apr 17, 2012 at 6:44 AM, Travis Oliphant wrote: > Basically, there are two sets of changes as far as I understand right now: > >        1) ufunc infrastructure understands masked arrays >        2) ndarray grew attributes to represent masked arrays > > I am proposing that we keep 1) but ch

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Gael Varoquaux
On Mon, Apr 16, 2012 at 10:40:53PM -0500, Travis Oliphant wrote: > > The objectors object to any binary ABI change, but not specifically > > three pointers rather than two or one? > Adding pointers is not really an ABI change (but removing them after > they were there would be...) It's really jus

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
On Apr 16, 2012, at 11:59 PM, Matthew Brett wrote: > Hi, > > On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant wrote: I think the answer to this is yes, but it could be as a feature-filled sub-class (like the current numpy.ma, except in C). >>> >>> I'd love to hear that argumen

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 10:38 PM, Travis Oliphant wrote: > > On Apr 16, 2012, at 11:01 PM, Charles R Harris wrote: > > > > On Mon, Apr 16, 2012 at 8:46 PM, Travis Oliphant wrote: > >> >> On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: >> >> > Hi, >> > >> > On Mon, Apr 16, 2012 at 3:06 PM, Travis

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 8:40 PM, Travis Oliphant wrote: >>> >>> I think the answer to this is yes, but it could be as a feature-filled >>> sub-class (like the current numpy.ma, except in C). >> >> I'd love to hear that argument fleshed out in more detail - do you have time? > > > My proposal

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
On Apr 16, 2012, at 11:01 PM, Charles R Harris wrote: > > > On Mon, Apr 16, 2012 at 8:46 PM, Travis Oliphant wrote: > > On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: > > > Hi, > > > > On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant > > wrote: > > > >> I have heard from a few people th

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 8:46 PM, Travis Oliphant wrote: > > On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: > > > Hi, > > > > On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant > wrote: > > > >> I have heard from a few people that they are not excited by the growth > of > >> the NumPy data-struct

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
>> >> I think the answer to this is yes, but it could be as a feature-filled >> sub-class (like the current numpy.ma, except in C). > > I'd love to hear that argument fleshed out in more detail - do you have time? My proposal here is to basically take the current github NumPy data-structure a

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
Ralf, I wouldn't change your plans just yet for NumPy 1.7. With Mark available full time for the next few weeks, I think we will be able to make rapid progress on whatever is decided -- in fact if people are available to help but just need resources let me know off list. I just want to ma

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 7:46 PM, Travis Oliphant wrote: > > On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: > >> Hi, >> >> On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant wrote: >> >>> I have heard from a few people that they are not excited by the growth of >>> the NumPy data-structure b

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
On Apr 16, 2012, at 8:03 PM, Matthew Brett wrote: > Hi, > > On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant wrote: > >> I have heard from a few people that they are not excited by the growth of >> the NumPy data-structure by the 3 pointers needed to hold the masked-array >> storage. This is

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 6:03 PM, Matthew Brett wrote: > Hi, > > On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant wrote: > >> I have heard from a few people that they are not excited by the growth of >> the NumPy data-structure by the 3 pointers needed to hold the masked-array >> storage.   T

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Matthew Brett
Hi, On Mon, Apr 16, 2012 at 3:06 PM, Travis Oliphant wrote: > I have heard from a few people that they are not excited by the growth of > the NumPy data-structure by the 3 pointers needed to hold the masked-array > storage.   This is especially true when there is talk to potentially add > additi

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 5:17 PM, Travis Oliphant wrote: > The comments I have heard have been from people who haven't wanted to make > them on this list. I wish they would, but I understand that not everyone > wants to be drawn into a long discussion.They have not been discussions. > > My bi

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
The comments I have heard have been from people who haven't wanted to make them on this list. I wish they would, but I understand that not everyone wants to be drawn into a long discussion.They have not been discussions. My bias is to just move forward with what is there. After a week or

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Ralf Gommers
On Tue, Apr 17, 2012 at 12:27 AM, Fernando Perez wrote: > On Mon, Apr 16, 2012 at 3:21 PM, Ralf Gommers > wrote: > > That's the first time I've heard this. Until now, we have talked a lot > about > > adding bitmasks and API changes, not about complete removal. My > assumption > > was that the exp

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Charles R Harris
On Mon, Apr 16, 2012 at 4:33 PM, Travis Oliphant wrote: > No off list discussions have been happening material to this point. I am > basically stating my view for the first time. I have delayed because I > realize it is not a pleasant view and I was hoping I could end up resolving > it favorabl

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
No off list discussions have been happening material to this point. I am basically stating my view for the first time. I have delayed because I realize it is not a pleasant view and I was hoping I could end up resolving it favorably. But, it needs to be discussed before 1.7 is released.

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Fernando Perez
On Mon, Apr 16, 2012 at 3:21 PM, Ralf Gommers wrote: > That's the first time I've heard this. Until now, we have talked a lot about > adding bitmasks and API changes, not about complete removal. My assumption > was that the experimental label was enough. From Nathaniel's reaction I > gathered the

Re: [Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Ralf Gommers
On Tue, Apr 17, 2012 at 12:06 AM, Travis Oliphant wrote: > There is an issue with the NumPy 1.7 release that we all need to > understand. Doesn't including the missing-data attributes in the NumPy > structure in a released version of NumPy basically commit to including > those attributes in NumP

[Numpy-discussion] Removing masked arrays for 1.7? (Was 1.7 blockers)

2012-04-16 Thread Travis Oliphant
There is an issue with the NumPy 1.7 release that we all need to understand. Doesn't including the missing-data attributes in the NumPy structure in a released version of NumPy basically commit to including those attributes in NumPy 1.8? I'm not comfortable with that, is everyone else?