Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-11 Thread Graeme B. Bell
From my previous mail: this has the same performance as your code: a = empty([3] list(A.shape) For anyone that is interested. I ran a benchmark on the code after Julian kindly provided me with a correction to the listing he posted. a = empty([3] + list(A.shape)) a[0] = A5; a[1] = B2; a[2]

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Graeme B. Bell
Hi Robert, Thanks for proposing an alternative implementation approach. However, did you test your proposal before you made the assertion about its behaviour? reduce(np.logical_or, inputs, False) reduce(np.logical_and, inputs, True) This code consistently benchmarks 20% slower than the

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Graeme B. Bell
Hi Julian, Thanks for the post. It's great to hear that the main numpy function is improving in 1.8, though I think there is still plenty of value here for performance junkies :-) I don't have 1.8beta installed (and I can't conveniently install it on my machines just now). If you have

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Chris Barker - NOAA Federal
This is good stuff, but I can't help thinking that if I needed to do an any/all test on a number of arrays with common and/or combos -- I'd probably write a Cython function to do it. It could be a bit tricky to make it really general, but not bad for a couple specific dtypes / use cases. -just a

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-05 Thread Julian Taylor
hi, its not np.any that is slow in this case its np.array([A, B, C]) np.dstack([A, B, C]) is better but writing it like this has the same performance as your code: a = empty([3] list(A.shape) a[0] = A5; a[1] = B2; a[2] = A10; np.any(a, 0) I'll check if creating an array from a sequence can be

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Robert Kern
On Wed, Sep 4, 2013 at 11:05 AM, Graeme B. Bell g...@skogoglandskap.no wrote: In my current GIS raster work I often have a situation where I generate code something like this: np.any([A4, A==2, B==5, ...]) However, np.any() is quite slow. It's possible to use np.logical_or to

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Graeme B. Bell
Sorry, I should have been more clear. As shown in the benchmark/example, the method is replacing the behaviour of np.any(inputs, 0) not the behaviour of np.any(inputs) Here, where I'm making decisions based on overlaying layers of raster data in the same shape, I don't want to map the

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Phil Elson
For the record, I started a discussion about 6 months ago about a find_first type function which avoided running the logic over the whole array (using lambdas instead). This spilled into a discussion about implementing a short-cutted any or all function:

Re: [Numpy-discussion] fast_any_all , a trivial but fast/useful helper function for numpy

2013-09-04 Thread Julian Taylor
On 04.09.2013 12:05, Graeme B. Bell wrote: In my current GIS raster work I often have a situation where I generate code something like this: np.any([A4, A==2, B==5, ...]) However, np.any() is quite slow. It's possible to use np.logical_or to solve the problem, but then you