[Numpy-discussion] Are masked arrays slower for processing than ndarrays?

2009-05-09 Thread Eli Bressert
Hi, I'm using masked arrays to compute large-scale standard deviation, multiplication, gaussian, and weighted averages. At first I thought using the masked arrays would be a great way to sidestep looping (which it is), but it's still slower than expected. Here's a snippet of the code that I'm

Re: [Numpy-discussion] Are masked arrays slower for processing than ndarrays?

2009-05-09 Thread Eric Firing
Eli Bressert wrote: Hi, I'm using masked arrays to compute large-scale standard deviation, multiplication, gaussian, and weighted averages. At first I thought using the masked arrays would be a great way to sidestep looping (which it is), but it's still slower than expected. Here's a

Re: [Numpy-discussion] Are masked arrays slower for processing than ndarrays?

2009-05-09 Thread Eric Firing
Eli Bressert wrote: Hi, I'm using masked arrays to compute large-scale standard deviation, multiplication, gaussian, and weighted averages. At first I thought using the masked arrays would be a great way to sidestep looping (which it is), but it's still slower than expected. Here's a snippet of

Re: [Numpy-discussion] Are masked arrays slower for processing than ndarrays?

2009-05-09 Thread Pierre GM
Short answer to the subject: Oh yes. Basically, MaskedArrays in its current implementation is more of a convenience class than anything. Most of the functions manipulating masked arrays create a lot of temporaries. When performance is needed, I must advise you to work directly on the data

Re: [Numpy-discussion] Are masked arrays slower for processing than ndarrays?

2009-05-09 Thread Pierre GM
On May 9, 2009, at 8:17 PM, Eric Firing wrote: Eric Firing wrote: A part of the slowdown is what looks to me like unnecessary copying in _MaskedBinaryOperation.__call__. It is using getdata, which applies numpy.array to its input, forcing a copy. I think the copy is actually