Well, I don't know for a fact that this is truly what is happening. It is just Ondrej's insight, plus the fact that when I comment out the decorator, the import time is the same as it is in the master branch without Cythonization, but when it is there, it is twice as slow for repeated imports.
Clearly something is slowing it down because the tests run a little slower. Maybe Cythonizing more code would produce speed increases that would overcome any slowdowns from imports, or maybe it would level out. I haven't tried much. Ondrej is the one who is doing most of this, so he will probably know more here. Aaron Meurer On Aug 19, 2009, at 1:32 AM, Robert Bradshaw wrote: > On Aug 18, 2009, at 11:46 PM, Aaron S. Meurer wrote: > >> Hi. I am a developer for the project SymPy [0], which is a computer >> algebra system written in Python. We have been considering moving >> part or all of the code base into Cython for speed reasons. Ondrej >> Certik, the project's leader, recently gave a tutorial at SciPy09 >> (there should be a video up soon), which included a bit about >> increasing the speed of an operation by about 20x by using the >> cython.locals() decorator to convert some of the variables into >> cython >> ints. You can pull from his github account [1]. If you run: >> >>>>> from sympy import * >>>>> d = divisors(5*10**7) >> >> The function will run about 20x to 30x faster, as I mentioned. But >> if >> you run the ntheory test, which uses divisors, using >> >> $./bin/test sympy/ntheory >> >> it runs slower or at least as slow as without cython. We figured out >> that the reason is that when the multiple imports in the tests are >> called, they run about 2x slower with cython, because it calls the >> decorator each time. >> >> You can see this by running >> >> In [1]: %timeit from sympy import * >> >> with and without cython in IPython. >> >> Is there any way around this? This could be a show stopper for us, >> because the increased import times levels out or makes worse the >> total >> times for things. This is not exclusive to the test suite. It comes >> from multiple imports in each file. >> >> We know that we can use .pxd files, but that would require to handle >> both .pxd files and .py files instead of maintaining one file with >> everything. > > Eventually I want to get type inference, which could help here. > However, I'm not following how decorating these functions makes > everything slower. Sure, it's a bit slower on the first import, but > the decorator is no re-called on each import. Clearly I'm missing > something here, could you explain why the decorator is being > repeatedly called? > > In the worst case, maintaining separate .pxd files seems a small > price to pay to get a 20-30x speedup. Also, if you want to have cdef > classes, especially across various modules, you'll have to do that > anyways right now. (Patches for fixing this, e.g. via class > decorators, would be highly appreciated). > > - Robert > > > ------------------------- foo.py ----------------------- > > def decorate(f): > print "here" > return f > > @decorate > def f(x): > return x > > ---------------------------------------------------------- > > sage: from foo import * > here > sage: from foo import * > sage: from foo import * > > _______________________________________________ > Cython-dev mailing list > [email protected] > http://codespeak.net/mailman/listinfo/cython-dev _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
