Re: [Numpy-discussion] new MaskedArray class

On Sat, Jun 22, 2019 at 6:50 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Hi Allan, > > I'm not sure I would go too much by what the old MaskedArray class did. It > indeed made an effort not to overwrite masked values with a new result, > even to the extend of copying back masked

Re: [Numpy-discussion] new MaskedArray class

On Thu, Jun 20, 2019 at 7:44 PM Allan Haldane wrote: > On 6/19/19 10:19 PM, Marten van Kerkwijk wrote: > > Hi Allan, > > > > This is very impressive! I could get the tests that I wrote for my class > > pass with yours using Quantity with what I would consider very minimal > > changes. I only

Re: [Numpy-discussion] new MaskedArray class

Hi Eric, On your other points: I remain unconvinced that Mask classes should behave differently on > different ufuncs. I don’t think np.minimum(ignore_na, b) is any different > to np.add(ignore_na, b) - either both should produce b, or both should > produce ignore_na. I would lean towards

Re: [Numpy-discussion] new MaskedArray class

Hi Stephan, Eric perhaps explained my concept better than I could! I do agree that, as written, your example would be clearer, but Allan's code and the current MaskedArray code do have not that much semblance to it, and mine even less, as they deal with operators as whole groups. For mine, it

Re: [Numpy-discussion] new MaskedArray class

I think we’d need to consider separately the operation on the mask and on the data. In my proposal, the data would always do np.sum(array, where=~mask), while how the mask would propagate might depend on the mask itself, I quite like this idea, and I think Stephan’s strawman design is actually

Re: [Numpy-discussion] new MaskedArray class

On Sun, Jun 23, 2019 at 11:55 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Your proposal would be something like np.sum(array, >> where=np.ones_like(array))? This seems rather verbose for a common >> operation. Perhaps np.sum(array, where=True) would work, making use of >>

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

I had not looked at any implementation (only remembered the nice idea of "importing from the future"), and looking at the links Eric shared, it seems that the only way this would work is, effectively, pre-compilation doing a `.replace('.T', '._T_from_the_future')`, where you'd be hoping that there

Re: [Numpy-discussion] new MaskedArray class

On Sat, Jun 22, 2019 at 11:51 AM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > Hi Allan, > > I'm not sure I would go too much by what the old MaskedArray class did. It > indeed made an effort not to overwrite masked values with a new result, > even to the extend of copying back masked

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

On Sun, 2019-06-23 at 19:51 +, Hameer Abbasi wrote: > +1 for this. I have often seen (and sometimes written) code that does > this automatically, and it is a common mistake. Yeah, likely worth a short. I doubt many uses for the n-dimensional axis transpose, so maybe a futurewarning approach

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

On Sun, Jun 23, 2019 at 10:37 PM Sebastian Berg wrote: > Yeah, likely worth a short. I doubt many uses for the n-dimensional > axis transpose, so maybe a futurewarning approach can work. If not, I > suppose the solution is the deprecation for ndim != 2. Any chance that the n-dimensional

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

On Sun, 2019-06-23 at 23:03 +0200, Andras Deak wrote: > On Sun, Jun 23, 2019 at 10:37 PM Sebastian Berg > wrote: > > Yeah, likely worth a short. I doubt many uses for the n-dimensional > > axis transpose, so maybe a futurewarning approach can work. If not, > > I > > suppose the solution is the

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

On Sun, 2019-06-23 at 17:12 -0400, Marten van Kerkwijk wrote: > Hi All, > > I'd love to have `.T` mean the right thing, and am happy that people > are suggesting it after I told Steward this was likely off-limits > (which, in fairness, did seem to be the conclusion when we visited > this

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

If I remember correctly, [numpy.future imports are] actually possible but hacky. So it would probably be nicer to not go there. There was some discussion of this at https://stackoverflow.com/q/29905278/102441. I agree with the conclusion we should not go there - in particular, note that every

Re: [Numpy-discussion] new MaskedArray class

On Sun, Jun 23, 2019 at 4:07 PM Marten van Kerkwijk < m.h.vankerkw...@gmail.com> wrote: > - If reductions/aggregations default to skipping missing elements, how is >> it be possible to express "NA propagating" versions, which are also useful, >> if slightly less common? >> > > I have been playing

[Numpy-discussion] Syntax Improvement for Array Transpose

Hi All, Based on discussion with Marten on github , I have a couple of suggestions on syntax improvements on array transpose operations. First, introducing a shorthand for the Hermitian Transpose operator. I thought "A.HT" might be a viable candidate.

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

+1 for this. I have often seen (and sometimes written) code that does this automatically, and it is a common mistake. However, we will need some way to filter for intent, as the people who write this code are the ones who didn’t read docs on it at the time, and so there might be a fair amount

Re: [Numpy-discussion] new MaskedArray class

> I think a sensible alternative mental model for the MaskedArray class is >> that all it does is forward any operations to the data it holds and >> separately propagate a mask, ORing elements together for binary operations, >> etc., and explicitly skipping masked elements in reductions (ideally

Re: [Numpy-discussion] new MaskedArray class

Hi Tom, I think a sensible alternative mental model for the MaskedArray class is >> that all it does is forward any operations to the data it holds and >> separately propagate a mask, >> > > I'm generally on-board with that mental picture, and agree that the > use-case described by Ben

Re: [Numpy-discussion] Syntax Improvement for Array Transpose

This might be contentious, but I wonder if, with a long enough deprecation cycle, we can change the meaning of .T. That would look like: * Emit a future warning on `more_than_2d.T` with a message like "in future .T will transpose just the last two dimensions, not all dimensions. Use

Re: [Numpy-discussion] new MaskedArray class

Hi Stephan, In slightly changed order: Let me try to make the API issue more concrete. Suppose we have a > MaskedArray with values [1, 2, NA]. How do I get: > 1. The sum ignoring masked values, i.e., 3. > 2. The sum that is tainted by masked values, i.e., NA. > > Here's how this works with