On Tue, Jun 5, 2012 at 12:55 PM, mark florisson <markflorisso...@gmail.com> wrote: > It would be great if we implement the NEP listed above, but with a few > extensions. I think Numpy should handle the lazy evaluation part, and > determine when expressions should be evaluated, etc. However, for each > user operation, Numpy will call back a user-installed hook > implementing some interface, to allow various packages to provide > their own hooks to evaluate vector operations however they want. This > will include packages such as Theano, which could run things on the > GPU, Numexpr, and in the future > https://github.com/markflorisson88/minivect (which will likely have an > LLVM backend in the future, and possibly integrated with Numba to > allow inlining of numba ufuncs). The project above tries to bring > together all the different array expression compilers together in a > single framework, to provide efficient array expressions specialized > for any data layout (nditer on steroids if you will, with SIMD, > threaded and inlining capabilities).
A global hook sounds ugly and hard to control -- it's hard to tell which operations should be deferred and which should be forced, etc. While it would be less magical, I think a more explicit API would in the end be easier to use... something like a, b, c, d = deferred([a, b, c, d]) e = a + b * c # 'e' is a deferred object too f = np.dot(e, d) # so is 'f' g = force(f) # 'g' is an ndarray # or force(f, out=g) But at that point, this could easily be an external library, right? All we'd need from numpy would be some way for external types to override the evaluation of ufuncs, np.dot, etc.? We've recently seen several reasons to want that functionality, and it seems like developing these "improved numexpr" ideas would be much easier if they didn't require doing deep surgery to numpy itself... -N _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
