On Wed, Feb 1, 2012 at 7:13 PM, Travis Oliphant <tra...@continuum.io> wrote:
> Hey Mark, > > I spent some quality time with your iterator docs tonight and look forward > to getting into the code a bit more soon. I wanted to get your general > impressions about what it would take to extend the iterator API to handle > iterating over "regions" of the inputs --- i.e. to support generalized > ufuncs. > Supposing this feature occurred to me, and I were to name it "subarray iteration," I would create a branch to start developing it here: https://github.com/m-paradox/numpy/tree/subarray_iter Brief documentation of how the interface could look is here: https://github.com/m-paradox/numpy/commit/610e7ae0bad66b95988bd2a933c08537b26898c5 > Also, on my todo list is to compare generalized ufuncs with "threading" in > PDL (Perl Data Language) and see if we can support that in NumPy. > Threading is the word that PDL uses to describe "broadcasting" --- but it > does more than ufuncs. > > Here is some information on it. > I've read this document before, and recall not seeing anything that was more than broadcasting. The only notable difference from broadcasting was that threading adds axis padding to the right instead of to the left. -Mark > > -Travis > > > > On Feb 1, 2012, at 8:31 PM, Mark Wiebe wrote: > > On Wed, Feb 1, 2012 at 6:14 PM, Travis Oliphant <tra...@continuum.io>wrote: > >> >> On Feb 1, 2012, at 7:04 PM, Mark Wiebe wrote: >> >> On Wed, Feb 1, 2012 at 3:29 PM, Charles R Harris < >> charlesr.har...@gmail.com> wrote: >> >>> The macro PyArray_RemoveLargest has been replaced by >>> PyArray_RemoveSmallest (which seems strange), but I wonder if this >>> documentation still makes sense. >>> >> >> My impression about this code is that it went through a number of rounds >> trying to choose an iteration order heuristic that has improved performance >> over C-order. The change of Largest to Smallest probably reflects one of >> these heuristic changes. I think it's safe to say that the nditer >> introduced in 1.6 completely removes the need for this functionality. I did >> a grep for this function in the master branch, and it is no longer used by >> NumPy internally. >> >> >> There is a common need to iterate over all but one dimension of a NumPy >> array. The final dimension is iterated over in an "internal" loop. This >> is the essence of how ufuncs work and avoid the possibly expensive overhead >> of a C-call during each iteration. >> > > This use-case is handled by the flag NPY_ITER_EXTERNAL_LOOP ( > http://docs.scipy.org/doc/numpy/reference/c-api.iterator.html#NPY_ITER_EXTERNAL_LOOP) > in the nditer. > > >> Initially, it seemed prudent to remove the dimension that had the largest >> size (so that the final inner-iteration was the largest number). Later, >> timings revealed that that the 'inner' dimension should be the one with the >> smallest *striding*. I have not looked at nditer in detail, but would >> appreciate seeing an explanation of how the nditer approach removes the >> need for this macro. When that is clear, then this macro can and should >> be deprecated. >> > > To see the full list of what to use in the nditer versus the older > iterators, I created a table: > > > http://docs.scipy.org/doc/numpy/reference/c-api.iterator.html#converting-from-previous-numpy-iterators > > Only PyArray_BroadcastToShape and PyArray_MultiIter_NEXTi don't have a > nice correspondence, because they refer to implementation details in the > previous iterators which are done differently in the nditer. > > -Mark > > >> >> -Travis >> >> >> >> >> >> -Mark >> >> >>> diff --git a/doc/source/user/c-info.beyond-basics.rst b/doc/source/user/ >>> c-info.beyond-basics.rs >>> index 9ed2ab3..3437985 100644 >>> --- a/doc/source/user/c-info.beyond-basics.rst >>> +++ b/doc/source/user/c-info.beyond-basics.rst >>> @@ -189,7 +189,7 @@ feature follows. >>> PyArray_MultiIter_NEXT(mobj); >>> } >>> >>> -The function :cfunc:`PyArray_RemoveLargest` ( ``multi`` ) can be used to >>> +The function :cfunc:`PyArray_RemoveSmallest` ( ``multi`` ) can be used >>> to >>> take a multi-iterator object and adjust all the iterators so that >>> iteration does not take place over the largest dimension (it makes >>> that dimension of size 1). The code being looped over that makes use >>> >>> Chuck >>> >>> >>> _______________________________________________ >>> NumPy-Discussion mailing list >>> NumPy-Discussion@scipy.org >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion >>> >>> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
