Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-14 Thread Richard Hattersley
For what it's worth, I'd prefer ndmasked. As has been mentioned elsewhere, some algorithms can't really cope with missing data. I'd very much rather they fail than silently give incorrect results. Working in the climate prediction business (as with many other domains I'm sure), even the *potential

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-11 Thread Mark Wiebe
On Thu, May 10, 2012 at 10:28 PM, Matthew Brett wrote: > Hi, > > On Thu, May 10, 2012 at 2:43 AM, Nathaniel Smith wrote: > > Hi Matthew, > > > > On Thu, May 10, 2012 at 12:01 AM, Matthew Brett > wrote: > >>> The third proposal is certainly the best one from Cython's perspective; > >>> and I imag

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-11 Thread Travis Oliphant
On May 11, 2012, at 2:13 AM, Fernando Perez wrote: > On Thu, May 10, 2012 at 11:44 PM, Scott Sinclair > wrote: >> That's pretty much how things already work. The documentation is in >> the main source tree and built docs end up at http://docs.scipy.org. >> NEPs live at https://github.com/numpy/n

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-11 Thread Fernando Perez
On Thu, May 10, 2012 at 11:44 PM, Scott Sinclair wrote: > That's pretty much how things already work. The documentation is in > the main source tree and built docs end up at http://docs.scipy.org. > NEPs live at https://github.com/numpy/numpy/tree/master/doc/neps, but > don't get published outside

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Scott Sinclair
On 11 May 2012 08:12, Fernando Perez wrote: > On Thu, May 10, 2012 at 11:03 PM, Scott Sinclair > wrote: >> Having thought about it, a page on the website isn't a bad idea. I've >> added a note pointing to this discussion. The document now appears at >> http://numpy.scipy.org/NA-overview.html > >

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Fernando Perez
On Thu, May 10, 2012 at 11:03 PM, Scott Sinclair wrote: > Having thought about it, a page on the website isn't a bad idea. I've > added a note pointing to this discussion. The document now appears at > http://numpy.scipy.org/NA-overview.html Why not have a separate repo for neps/discussion docs?

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Scott Sinclair
On 11 May 2012 06:57, Travis Oliphant wrote: > > On May 10, 2012, at 3:40 AM, Scott Sinclair wrote: > >> On 9 May 2012 18:46, Travis Oliphant wrote: >>> The document is available here: >>>    https://github.com/numpy/numpy.scipy.org/blob/master/NA-overview.rst >> >> This is orthogonal to the disc

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Travis Oliphant
On May 10, 2012, at 12:21 AM, Charles R Harris wrote: > > > On Wed, May 9, 2012 at 11:05 PM, Benjamin Root wrote: > > > On Wednesday, May 9, 2012, Nathaniel Smith wrote: > > > My only objection to this proposal is that committing to this approach > seems premature. The existing masked arra

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Travis Oliphant
On May 10, 2012, at 3:40 AM, Scott Sinclair wrote: > On 9 May 2012 18:46, Travis Oliphant wrote: >> The document is available here: >>https://github.com/numpy/numpy.scipy.org/blob/master/NA-overview.rst > > This is orthogonal to the discussion, but I'm curious as to why this > discussion do

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Matthew Brett
Hi, On Thu, May 10, 2012 at 2:43 AM, Nathaniel Smith wrote: > Hi Matthew, > > On Thu, May 10, 2012 at 12:01 AM, Matthew Brett > wrote: >>> The third proposal is certainly the best one from Cython's perspective; >>> and I imagine for those writing C extensions against the C API too. >>> Having P

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Nathaniel Smith
Hi Matthew, On Thu, May 10, 2012 at 12:01 AM, Matthew Brett wrote: >> The third proposal is certainly the best one from Cython's perspective; >> and I imagine for those writing C extensions against the C API too. >> Having PyType_Check fail for ndmasked is a very good way of having code >> fail t

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Dag Sverre Seljebotn
On 05/10/2012 06:05 AM, Dag Sverre Seljebotn wrote: > On 05/10/2012 01:01 AM, Matthew Brett wrote: >> Hi, >> >> On Wed, May 9, 2012 at 12:44 PM, Dag Sverre Seljebotn >>wrote: >>> On 05/09/2012 06:46 PM, Travis Oliphant wrote: Hey all, Nathaniel and Mark have worked very hard on a

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Scott Sinclair
On 9 May 2012 18:46, Travis Oliphant wrote: > The document is available here: >    https://github.com/numpy/numpy.scipy.org/blob/master/NA-overview.rst This is orthogonal to the discussion, but I'm curious as to why this discussion document has landed in the website repo? I suppose it's not a re

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-10 Thread Gael Varoquaux
On Wed, May 09, 2012 at 02:35:26PM -0500, Travis Oliphant wrote: >  Basically it buys not forcing *all* NumPy users (on the C-API level) to >now deal with a masked array.    I know this push is a feature that is >part of Mark's intention (as it pushes downstream libraries to think about

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Charles R Harris
On Wed, May 9, 2012 at 11:05 PM, Benjamin Root wrote: > > > On Wednesday, May 9, 2012, Nathaniel Smith wrote: > >> >> >> My only objection to this proposal is that committing to this approach >> seems premature. The existing masked array objects act quite >> differently from numpy.ma, so why do y

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Benjamin Root
On Wednesday, May 9, 2012, Nathaniel Smith wrote: > > > My only objection to this proposal is that committing to this approach > seems premature. The existing masked array objects act quite > differently from numpy.ma, so why do you believe that they're a good > foundation for numpy.ma, and why wi

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Dag Sverre Seljebotn
On 05/10/2012 01:01 AM, Matthew Brett wrote: > Hi, > > On Wed, May 9, 2012 at 12:44 PM, Dag Sverre Seljebotn > wrote: >> On 05/09/2012 06:46 PM, Travis Oliphant wrote: >>> Hey all, >>> >>> Nathaniel and Mark have worked very hard on a joint document to try and >>> explain the current status of th

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Charles R Harris
On Wed, May 9, 2012 at 6:13 PM, Paul Ivanov wrote: > > > On Wed, May 9, 2012 at 3:12 PM, Travis Oliphant wrote: > >> On re-reading, I want to make a couple of things clear: >> >> 1) This "wrap-up" discussion is *only* for what to do for NumPy 1.7 in >> such a way that we don't tie our hands in th

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Paul Ivanov
On Wed, May 9, 2012 at 3:12 PM, Travis Oliphant wrote: > On re-reading, I want to make a couple of things clear: > > 1) This "wrap-up" discussion is *only* for what to do for NumPy 1.7 in > such a way that we don't tie our hands in the future.I do not believe > we can figure out what to do fo

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Nathaniel Smith
Hi Dag, On Wed, May 9, 2012 at 8:44 PM, Dag Sverre Seljebotn wrote: > I'm a heavy user of masks, which are used to make data NA in the > statistical sense. The setting is that we have to mask out the radiation > coming from the Milky Way in full-sky images of the Cosmic Microwave > Background. Th

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Matthew Brett
Hi, On Wed, May 9, 2012 at 12:44 PM, Dag Sverre Seljebotn wrote: > On 05/09/2012 06:46 PM, Travis Oliphant wrote: >> Hey all, >> >> Nathaniel and Mark have worked very hard on a joint document to try and >> explain the current status of the missing-data debate. I think they've >> done an amazing

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Nathaniel Smith
On Wed, May 9, 2012 at 5:46 PM, Travis Oliphant wrote: > Hey all, > > Nathaniel and Mark have worked very hard on a joint document to try and > explain the current status of the missing-data debate.   I think they've > done an amazing job at providing some context, articulating their views and > s

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Charles R Harris
On Wed, May 9, 2012 at 4:12 PM, Travis Oliphant wrote: > On re-reading, I want to make a couple of things clear: > > 1) This "wrap-up" discussion is *only* for what to do for NumPy 1.7 in > such a way that we don't tie our hands in the future.I do not believe > we can figure out what to do fo

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Travis Oliphant
On re-reading, I want to make a couple of things clear: 1) This "wrap-up" discussion is *only* for what to do for NumPy 1.7 in such a way that we don't tie our hands in the future.I do not believe we can figure out what to do for masked arrays in one short week. What happens be

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Charles R Harris
On Wed, May 9, 2012 at 1:35 PM, Travis Oliphant wrote: > My three proposals: >> >> * do nothing and leave things as is >> >> * add a global flag that turns off masked array support by default but >> otherwise leaves things unchanged (I'm still unclear how this would work >> exactly) >> >> * move

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Dag Sverre Seljebotn
On 05/09/2012 06:46 PM, Travis Oliphant wrote: > Hey all, > > Nathaniel and Mark have worked very hard on a joint document to try and > explain the current status of the missing-data debate. I think they've > done an amazing job at providing some context, articulating their views > and suggesting w

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Travis Oliphant
> Mark will you give more details about this proposal?How would the flag > work, what would it modify? > > The idea is inspired in part by the Chrome release cycle, which has a > presentation here: > > https://docs.google.com/present/view?id=dg63dpc6_4d7vkk6ch&pli=1 > > Some quotes: > Feat

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Travis Oliphant
> My three proposals: > > * do nothing and leave things as is > > * add a global flag that turns off masked array support by default but > otherwise leaves things unchanged (I'm still unclear how this would work > exactly) > > * move Mark's "masked ndarray objects" into a n

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Mark Wiebe
On Wed, May 9, 2012 at 2:15 PM, Travis Oliphant wrote: > > On May 9, 2012, at 2:07 PM, Mark Wiebe wrote: > > On Wed, May 9, 2012 at 11:46 AM, Travis Oliphant wrote: > >> Hey all, >> >> Nathaniel and Mark have worked very hard on a joint document to try and >> explain the current status of the mis

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Travis Oliphant
On May 9, 2012, at 2:07 PM, Mark Wiebe wrote: > On Wed, May 9, 2012 at 11:46 AM, Travis Oliphant wrote: > Hey all, > > Nathaniel and Mark have worked very hard on a joint document to try and > explain the current status of the missing-data debate. I think they've done > an amazing job at p

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Mark Wiebe
On Wed, May 9, 2012 at 11:46 AM, Travis Oliphant wrote: > Hey all, > > Nathaniel and Mark have worked very hard on a joint document to try and > explain the current status of the missing-data debate. I think they've > done an amazing job at providing some context, articulating their views and >

Re: [Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Charles R Harris
On Wed, May 9, 2012 at 10:46 AM, Travis Oliphant wrote: > Hey all, > > Nathaniel and Mark have worked very hard on a joint document to try and > explain the current status of the missing-data debate. I think they've > done an amazing job at providing some context, articulating their views and >

[Numpy-discussion] Missing data wrap-up and request for comments

2012-05-09 Thread Travis Oliphant
Hey all, Nathaniel and Mark have worked very hard on a joint document to try and explain the current status of the missing-data debate. I think they've done an amazing job at providing some context, articulating their views and suggesting ways forward in a mutually respectful manner. This