On Tue, Nov 25, 2008 at 07:21:27PM +0100, Dag Sverre Seljebotn wrote: > Gabriel Gellner wrote: >> But the only difference in the caller between the f2py version and this one >> is this file. So the speed difference must be from the function calls etc as >> you have said, but I don't understand how the overhead can be from the caller >> as the code for this is identical (and not compiled). >> >> the code is basically: >> >> for model.b1 in np.linspace(0, 2.6, 100): >> odeint(model, y0, t0) >> >> where model is defined in a separate file, in one case as a fortran >> object generated by f2py and in this case by cython. > > Ah, right. > > It's interesting (but not that surprising) to see that f2py does perform > better in this area, due to both the call to np.empty and that acquiring > the buffer from the NumPy array likely is slower than the NumPy-specific > stuff that f2py is doing. > > But that Python snippet is likely going to use the majority of the time > no matter how things are done, so it doesn't matter much. I'm interested > to hear how much do you gain over pure Python in this case then -- > probably not too much, perhaps 2-300%? (Moving the entire for-loop to > Cython would typically give you around 800% improvement). > The Cython callback version is 10 times faster than the pure python.
> Anyway, this is not likely to be an area where Cython is improved, so > there's nothing to do about it. I don't think it is a common usecase > either -- typically either one keep everything in Python, or one decide > that Python only is too slow, but then one doesn't want to only leverage > a small part of the speed increase that compiled code can give you... > I don't think this is true. Speeding up callbacks (that I loop in my code is not essential odeint needs to call the provided function a lot, even without a python loop) is very, very common. PyDSTool (a dynamical systems package) has even invented their own text format to do this. Unless I can expect that every python routine I use has a Cython interface that excepts cdef-like functions (which for scipy [as far as I know] is currently not true at all) than I am stuck with python callbacks or making my own wrappers (and not using the vast scipy ecosystem), which would mean I should just use C or Fortran alone. Using tools like f2py, weave, and recently cython is the only major draw to python I have to offer fellow researchers over matlab (as it makes the equivalent MEX construction seem extra painful). Heck using this I can significantly speed up solvers written in pure python that consume a callback. I find for most iterative solvers in python the callback is the most expensive part (even for toy examples like I have given), especially with Cython to get rid of the loop overhead. > Even if you move the for-loop into Cython or Fortran you don't have to > use vectorized calculations. Something like: > > cython_apply_odeint(model, 0, 2.6, 100) > I see what you mean, yes I can do this (and often do), but it doesn't get over the callback issue. > and then just code a very generic for-loop in Cython which calls odeint > and model, just to move the loop on the other side of the > Python/Cython-barrier. > > Enough ranting though. Thanks for bringing this to my attention! > And thanks for discussing it. Gabriel _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
