That's right, essentially all I've done is replaced the code that handled preparing the arrays and producing blocks of values for the inner loops. There are three new parameters to evaluate_iter as well. It has an "out=" parameter just like ufuncs do, an "order=" parameter which controls the layout of the output if it's created by the function, and a "casting=" parameter which controls what kind of data conversions are permitted.
-Mark On Sun, Jan 9, 2011 at 3:33 PM, John Salvatier <[email protected]>wrote: > Is evaluate_iter basically numpexpr but using your numpy branch or are > there other changes? > > On Sun, Jan 9, 2011 at 2:45 PM, Mark Wiebe <[email protected]> wrote: > >> As a benchmark of C-based iterator usage and to make it work properly in a >> multi-threaded context, I've updated numexpr to use the new iterator. In >> addition to some performance improvements, this also made it easy to add >> optional out= and order= parameters to the evaluate function. The numexpr >> repository with this update is available here: >> >> https://github.com/m-paradox/numexpr >> >> To use it, you need the new_iterator branch of NumPy from here: >> >> https://github.com/m-paradox/numpy >> >> In all cases tested, the iterator version of numexpr's evaluate function >> matches or beats the standard version. The timing results are below, with >> some explanatory comments placed inline: >> >> -Mark >> >> In [1]: import numexpr as ne >> >> # numexpr front page example >> >> In [2]: a = np.arange(1e6) >> In [3]: b = np.arange(1e6) >> >> In [4]: timeit a**2 + b**2 + 2*a*b >> 1 loops, best of 3: 121 ms per loop >> >> In [5]: ne.set_num_threads(1) >> >> # iterator version performance matches standard version >> >> In [6]: timeit ne.evaluate("a**2 + b**2 + 2*a*b") >> 10 loops, best of 3: 24.8 ms per loop >> In [7]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b") >> 10 loops, best of 3: 24.3 ms per loop >> >> In [8]: ne.set_num_threads(2) >> >> # iterator version performance matches standard version >> >> In [9]: timeit ne.evaluate("a**2 + b**2 + 2*a*b") >> 10 loops, best of 3: 21 ms per loop >> In [10]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b") >> 10 loops, best of 3: 20.5 ms per loop >> >> # numexpr front page example with a 10x bigger array >> >> In [11]: a = np.arange(1e7) >> In [12]: b = np.arange(1e7) >> >> In [13]: ne.set_num_threads(2) >> >> # the iterator version performance improvement is due to >> # a small task scheduler tweak >> >> In [14]: timeit ne.evaluate("a**2 + b**2 + 2*a*b") >> 1 loops, best of 3: 282 ms per loop >> In [15]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b") >> 1 loops, best of 3: 255 ms per loop >> >> # numexpr front page example with a Fortran contiguous array >> >> In [16]: a = np.arange(1e7).reshape(10,100,100,100).T >> In [17]: b = np.arange(1e7).reshape(10,100,100,100).T >> >> In [18]: timeit a**2 + b**2 + 2*a*b >> 1 loops, best of 3: 3.22 s per loop >> >> In [19]: ne.set_num_threads(1) >> >> # even with a C-ordered output, the iterator version performs better >> >> In [20]: timeit ne.evaluate("a**2 + b**2 + 2*a*b") >> 1 loops, best of 3: 3.74 s per loop >> In [21]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b") >> 1 loops, best of 3: 379 ms per loop >> In [22]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b", order='C') >> 1 loops, best of 3: 2.03 s per loop >> >> In [23]: ne.set_num_threads(2) >> >> # the standard version just uses 1 thread here, I believe >> # the iterator version performs the same as for the flat 1e7-sized array >> >> In [24]: timeit ne.evaluate("a**2 + b**2 + 2*a*b") >> 1 loops, best of 3: 3.92 s per loop >> In [25]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b") >> 1 loops, best of 3: 254 ms per loop >> In [26]: timeit ne.evaluate_iter("a**2 + b**2 + 2*a*b", order='C') >> 1 loops, best of 3: 1.74 s per loop >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> [email protected] >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
