Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Thu, Jun 30, 2011 at 10:51 AM, Nathaniel Smith wrote: > On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett > wrote: >> In the interest of making the discussion as concrete as possible, here >> is my draft of an alternative proposal for NAs and masking, based on >> Nathaniel's comments.  Writing i

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Thu, Jun 30, 2011 at 6:02 PM, Matthew Brett wrote: > Hi, > > On Thu, Jun 30, 2011 at 9:01 PM, Lluís wrote: > > Matthew Brett writes: > > > >> Hi, > >> On Thu, Jun 30, 2011 at 7:27 PM, Lluís wrote: > >>> Matthew Brett writes: > >>> [...] > I'm afraid, like you, I'm a little lost in the wo

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

It seems to me, that what ``func`` should do, if it wants you to be able to unmask the NAs, is to make a masked array view of ``arr``, and return that.   And indeed the simplicity of the separated API immediately makes that clear - in my view at least. I agree on this example. My only concern

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Hi, On Thu, Jun 30, 2011 at 9:01 PM, Lluís wrote: > Matthew Brett writes: > >> Hi, >> On Thu, Jun 30, 2011 at 7:27 PM, Lluís wrote: >>> Matthew Brett writes: >>> [...] I'm afraid, like you, I'm a little lost in the world of masking, because I only need the NAs.  I was trying to see if

Re: [Numpy-discussion] broacasting question

On 30.06.2011, at 11:57PM, Thomas K Gamble wrote: >> np.add(b.reshape(2048,3136) * c, d, out=a[:,:3136]) >> >> But to say whether this is really the equivalent result to what IDL does, >> one would have to study the IDL manual in detail or directly compare the >> output (e.g. check what happens t

Re: [Numpy-discussion] broacasting question

> On 30.06.2011, at 7:32PM, Thomas K Gamble wrote: > > I'm trying to convert some IDL code to python/numpy and i'm having some > > trouble understanding the rules for boradcasting during some operations. > > example: > > > > given the following arrays: > > a = array((2048,3577), dtype=float) > > b

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Matthew Brett writes: > Hi, > On Thu, Jun 30, 2011 at 7:27 PM, Lluís wrote: >> Matthew Brett writes: >> [...] >>> I'm afraid, like you, I'm a little lost in the world of masking, >>> because I only need the NAs.  I was trying to see if I could come up >>> with an API that picked up some of the sy

Re: [Numpy-discussion] missing data discussion round 2

On Thu, Jun 30, 2011 at 12:27 PM, Eric Firing wrote: > On 06/30/2011 08:53 AM, Nathaniel Smith wrote: >> On Wed, Jun 29, 2011 at 2:21 PM, Eric Firing  wrote: >>> In addition, for new code, the full-blown masked array module may not be >>> needed.  A convenience it adds, however, is the automatic m

Re: [Numpy-discussion] missing data discussion round 2

On 06/30/2011 08:53 AM, Nathaniel Smith wrote: > On Wed, Jun 29, 2011 at 2:21 PM, Eric Firing wrote: >> In addition, for new code, the full-blown masked array module may not be >> needed. A convenience it adds, however, is the automatic masking of >> invalid values: >> >> In [1]: np.ma.log(-1) >>

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Nathaniel Smith writes: > On Thu, Jun 30, 2011 at 11:27 AM, Lluís wrote: >> As I tried to convey on my other mail, separating both will force you to >> either: >> >> * Make a copy of the array before passing it to another routine (because >>  the routine will assign np.NA but you still want the

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Hi, On Thu, Jun 30, 2011 at 7:27 PM, Lluís wrote: > Matthew Brett writes: > [...] >> I'm afraid, like you, I'm a little lost in the world of masking, >> because I only need the NAs.  I was trying to see if I could come up >> with an API that picked up some of the syntactic convenience of NAs, >>

Re: [Numpy-discussion] missing data discussion round 2

On Wed, Jun 29, 2011 at 2:21 PM, Eric Firing wrote: > In addition, for new code, the full-blown masked array module may not be > needed.  A convenience it adds, however, is the automatic masking of > invalid values: > > In [1]: np.ma.log(-1) > Out[1]: masked > > I'm sure this horrifies some, but t

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Thu, Jun 30, 2011 at 11:27 AM, Lluís wrote: > As I tried to convey on my other mail, separating both will force you to > either: > > * Make a copy of the array before passing it to another routine (because >  the routine will assign np.NA but you still want the original data) To help me unders

Re: [Numpy-discussion] missing data discussion round 2

On Thu, Jun 30, 2011 at 11:54 AM, Lluís wrote: > Mark Wiebe writes: > > Why is one "magic" and the other "real"? All of this is already > > sitting on 100 layers of abstraction above electrons and atoms. If > > we're talking about "real," maybe we should be programming in machine > > code or usin

Re: [Numpy-discussion] missing data discussion round 2

On Thu, Jun 30, 2011 at 11:42 AM, Matthew Brett wrote: > Hi, > > On Thu, Jun 30, 2011 at 5:13 PM, Mark Wiebe wrote: > > On Thu, Jun 30, 2011 at 11:04 AM, Gary Strangman > > wrote: > >> > >>> Clearly there are some overlaps between what masked arrays are > >>> trying to achieve and what

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Matthew Brett writes: [...] > I'm afraid, like you, I'm a little lost in the world of masking, > because I only need the NAs. I was trying to see if I could come up > with an API that picked up some of the syntactic convenience of NAs, > without conflating NAs with IGNOREs. I guess we need some

[Numpy-discussion] review request: introductory datetime documentation

https://github.com/numpy/numpy/pull/101 Thanks, Mark ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] missing data: semantics

On Thu, Jun 30, 2011 at 11:51 AM, Matthew Brett wrote: > Hi, > > On Thu, Jun 30, 2011 at 6:46 PM, Lluís wrote: > > Ok, I think it's time to step back and reformulate the problem by > > completely ignoring the implementation. > > > > Here we have 2 "generic" concepts (i.e., applicable to R), plus

Re: [Numpy-discussion] missing data: semantics

On Thu, Jun 30, 2011 at 11:46 AM, Lluís wrote: > Ok, I think it's time to step back and reformulate the problem by > completely ignoring the implementation. > > Here we have 2 "generic" concepts (i.e., applicable to R), plus another > extra concept that is exclusive to numpy: > > * Assigning np.N

Re: [Numpy-discussion] broacasting question

On 30.06.2011, at 7:32PM, Thomas K Gamble wrote: > I'm trying to convert some IDL code to python/numpy and i'm having some > trouble understanding the rules for boradcasting during some operations. > example: > > given the following arrays: > a = array((2048,3577), dtype=float) > b = array((256,

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Hi, On Thu, Jun 30, 2011 at 6:51 PM, Nathaniel Smith wrote: > On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett > wrote: >> In the interest of making the discussion as concrete as possible, here >> is my draft of an alternative proposal for NAs and masking, based on >> Nathaniel's comments.  Writi

Re: [Numpy-discussion] missing data: semantics

Hi, On Thu, Jun 30, 2011 at 6:46 PM, Lluís wrote: > Ok, I think it's time to step back and reformulate the problem by > completely ignoring the implementation. > > Here we have 2 "generic" concepts (i.e., applicable to R), plus another > extra concept that is exclusive to numpy: > > * Assigning n

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Thu, Jun 30, 2011 at 6:31 AM, Matthew Brett wrote: > In the interest of making the discussion as concrete as possible, here > is my draft of an alternative proposal for NAs and masking, based on > Nathaniel's comments.  Writing it, it seemed to me that Nathaniel is > right, that the ideas becom

[Numpy-discussion] missing data: semantics

Ok, I think it's time to step back and reformulate the problem by completely ignoring the implementation. Here we have 2 "generic" concepts (i.e., applicable to R), plus another extra concept that is exclusive to numpy: * Assigning np.NA to an array, cannot be undone unless through explicit ass

Re: [Numpy-discussion] broacasting question

2011/6/30 Thomas K Gamble > I'm trying to convert some IDL code to python/numpy and i'm having some > trouble understanding the rules for boradcasting during some operations. > example: > > given the following arrays: > a = array((2048,3577), dtype=float) > b = array((256,25088), dtype=float) > c

[Numpy-discussion] broacasting question

I'm trying to convert some IDL code to python/numpy and i'm having some trouble understanding the rules for boradcasting during some operations. example: given the following arrays: a = array((2048,3577), dtype=float) b = array((256,25088), dtype=float) c = array((2048,3136), dtype=float) d = arr

Re: [Numpy-discussion] missing data discussion round 2

Mark Wiebe writes: > Why is one "magic" and the other "real"? All of this is already > sitting on 100 layers of abstraction above electrons and atoms. If > we're talking about "real," maybe we should be programming in machine > code or using breadboards with individual transistors. M-x butterfly R

Re: [Numpy-discussion] missing data discussion round 2

Hi, On Thu, Jun 30, 2011 at 5:13 PM, Mark Wiebe wrote: > On Thu, Jun 30, 2011 at 11:04 AM, Gary Strangman > wrote: >> >>>      Clearly there are some overlaps between what masked arrays are >>>      trying to achieve and what Rs NA mechanisms are trying to achieve. >>>       Are they really simi

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Hi, On Thu, Jun 30, 2011 at 5:03 PM, Pierre GM wrote: > > On Jun 30, 2011, at 5:38 PM, Matthew Brett wrote: > >> Hi, >> >> On Thu, Jun 30, 2011 at 2:58 PM, Pierre GM wrote: >>> >>> On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: ### A alter

Re: [Numpy-discussion] missing data discussion round 2

On Thu, Jun 30, 2011 at 11:04 AM, Gary Strangman wrote: > > Clearly there are some overlaps between what masked arrays are >> trying to achieve and what Rs NA mechanisms are trying to achieve. >> Are they really similar enough that they should function using >> the same API?

Re: [Numpy-discussion] missing data discussion round 2

Mark Wiebe writes: > On Wed, Jun 29, 2011 at 1:20 PM, Lluís wrote: > [...] >> As far as I can tell, the only required difference between them is >> that NA bit patterns must destroy the data. Nothing else. Everything >> on top of that is a choice of API and interface mechanisms. I want >> the

Re: [Numpy-discussion] missing data discussion round 2

Clearly there are some overlaps between what masked arrays are trying to achieve and what Rs NA mechanisms are trying to achieve.  Are they really similar enough that they should function using the same API? Yes. And if so, won't that be confusing? No, I don't be

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Jun 30, 2011, at 5:38 PM, Matthew Brett wrote: > Hi, > > On Thu, Jun 30, 2011 at 2:58 PM, Pierre GM wrote: >> >> On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: >>> ### >>> A alternative-NEP on masking and missing values >>>

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

Hi, On Thu, Jun 30, 2011 at 2:58 PM, Pierre GM wrote: > > On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: >> ### >> A alternative-NEP on masking and missing values >> ### > > I like the idea of two different

Re: [Numpy-discussion] missing data discussion round 2

On Thu, Jun 30, 2011 at 1:49 AM, Chris Barker wrote: > On 6/27/11 9:53 AM, Charles R Harris wrote: > > Some discussion of disk storage might also help. I don't see how the > > rules can be enforced if two files are used, one for the mask and > > another for the data, but that may just be somethin

Re: [Numpy-discussion] missing data discussion round 2

On Wed, Jun 29, 2011 at 5:42 PM, Nathaniel Smith wrote: > On Wed, Jun 29, 2011 at 2:40 PM, Lluís wrote: > > I'm for the option of having a single API when you want to have NA > > elements, regardless of whether it's using masks or bit patterns. > > I understand the desire to avoid having two dif

Re: [Numpy-discussion] missing data discussion round 2

On Wed, Jun 29, 2011 at 4:21 PM, Eric Firing wrote: > On 06/29/2011 09:32 AM, Matthew Brett wrote: > > Hi, > > > [...] > > > > Clearly there are some overlaps between what masked arrays are trying > > to achieve and what Rs NA mechanisms are trying to achieve. Are they > > really similar enough

Re: [Numpy-discussion] missing data discussion round 2

On Wed, Jun 29, 2011 at 1:20 PM, Lluís wrote: > Mark Wiebe writes: > > > There seems to be a general idea that masks and NA bit patterns imply > > particular differing semantics, something which I think is simply > > false. > > Well, my example contained a difference (the need for the "skipna=Tru

Re: [Numpy-discussion] missing data discussion round 2

On Wed, Jun 29, 2011 at 2:32 PM, Matthew Brett wrote: > Hi, > > On Wed, Jun 29, 2011 at 6:22 PM, Mark Wiebe wrote: > > On Wed, Jun 29, 2011 at 8:20 AM, Lluís wrote: > >> > >> Matthew Brett writes: > >> > >> >> Maybe instead of np.NA, we could say np.IGNORE, which sort of conveys > >> >> the idea

Re: [Numpy-discussion] missing data discussion round 2

On Wed, Jun 29, 2011 at 1:07 PM, Dag Sverre Seljebotn < d.s.seljeb...@astro.uio.no> wrote: > On 06/29/2011 07:38 PM, Mark Wiebe wrote: > > On Wed, Jun 29, 2011 at 9:35 AM, Dag Sverre Seljebotn > > mailto:d.s.seljeb...@astro.uio.no>> wrote: > > > > On 06/29/2011 03:45 PM, Matthew Brett wrote: >

Re: [Numpy-discussion] feedback request: proposal to add masks to the core ndarray

On Wed, Jun 29, 2011 at 1:51 PM, Lluís wrote: > Mark Wiebe writes: > [...] > > I think that deciding on the value of NA signal values boils down to > > this question: should 3rd party code be able to interpret missing > data > > information stored in the separate mask array? > > > I'm

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Thu, Jun 30, 2011 at 8:17 AM, Charles R Harris wrote: > > > On Thu, Jun 30, 2011 at 7:31 AM, Matthew Brett wrote: > >> Hi, >> >> On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: >> > Anyway, it's pretty clear that in this particular case, there are two >> > distinct features that diffe

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On 06/30/2011 04:17 PM, Charles R Harris wrote: > > > On Thu, Jun 30, 2011 at 7:31 AM, Matthew Brett > wrote: > > Hi, > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith > wrote: > > Anyway, it's pretty clear that in this pa

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Thu, Jun 30, 2011 at 7:31 AM, Matthew Brett wrote: > Hi, > > On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > > Anyway, it's pretty clear that in this particular case, there are two > > distinct features that different people want: the missing data > > feature, and the masked array fe

Re: [Numpy-discussion] alterNEP - was: missing data discussion round 2

On Jun 30, 2011, at 3:31 PM, Matthew Brett wrote: > ### > A alternative-NEP on masking and missing values > ### I like the idea of two different special values, np.NA for missing values, np.IGNORE for masked

[Numpy-discussion] alterNEP - was: missing data discussion round 2

Hi, On Tue, Jun 28, 2011 at 4:06 PM, Nathaniel Smith wrote: > Anyway, it's pretty clear that in this particular case, there are two > distinct features that different people want: the missing data > feature, and the masked array feature. The more I think about it, the > less I see how they can be

Re: [Numpy-discussion] Fitting Log normal or truncated log normal distibution to three points

On Thu, Jun 30, 2011 at 5:25 AM, Christoph Deil wrote: > > On Jun 30, 2011, at 10:03 AM, Sachin Kumar Sharma wrote: > > Hi, > > I have three points 10800, 81100, 582000. > > What is the easiest way of fitting a log normal and truncated log normal > distribution to these three points using numpy. >

Re: [Numpy-discussion] Fitting Log normal or truncated log normal distibution to three points

On Jun 30, 2011, at 10:03 AM, Sachin Kumar Sharma wrote: > Hi, > > I have three points 10800, 81100, 582000. > > What is the easiest way of fitting a log normal and truncated log normal > distribution to these three points using numpy. The lognormal and maximum likelihood fit is available i

[Numpy-discussion] Fitting Log normal or truncated log normal distibution to three points

Hi, I have three points 10800, 81100, 582000. What is the easiest way of fitting a log normal and truncated log normal distribution to these three points using numpy. I would appreciate your reply for the same. Cheers Sachin ***