On Tue, Aug 25, 2009 at 10:19 AM, Adam
Ginsburg<[email protected]> wrote:
>> From: Robert Bradshaw <[email protected]>
>> Subject: Re: [Cython] Linking error - 64 bit problem?
>>
>> On Aug 22, 2009, at 7:40 PM, Adam Ginsburg wrote:
>>
>>> I reinstalled cython from source (instead of easy_install-ing it) and
>>> my code compiled successfully. Sorry about the previous e-mail.
>>
>> Great--I saw your previous email and started to get worried. I
>> thought easy_install was a source install (we didn't ship any
>> binaries until recently, and then just windows installers).
>
> I'm not entirely sure what happened, but I have at least 3 python
> installs that I use routinely, so it was probably just a confusion
> issue on my end.
>
>>> Unfortunately, my code runs slower with cython than without, but at
>>> least now I know I can compile. In case anyone wants to offer
>>> assistance, the code I'm trying to optimize is:
>>> http://code.google.com/p/agpy/source/browse/trunk/plfit.py
>>> http://code.google.com/p/agpy/source/browse/trunk/cplfit.pyx
>>
>> My first piece of advice is to run cython -a over your code. This
>> will produce an html file which is most helpful in diagnosing where
>> you're wasting time. Just in a quick glance at your code I noticed:
>>
>> - You're calling Python's log, sqrt, max, abs, sum. This would negate
>> any gains you may hope to see.
>> - You're calling len a lot, either store it as a int somewhere, or
>> use .shape[0].
>> - Unless n is huge, arange(n)/float(n) is probably way to expensive.
>> If the above don't help, try unrolling this.
>>
>> I bet there's a lot more performance that one could ring out of your
>> code, but the above should take you far.
>
>
> Much appreciated. I guess the various levels of yellow in the html
> file indicate the slow lines? I tried getting rid of all of my numpy
> calls in the loop by rewriting them as loops, but that hasn't improved
> speed at all, and in fact appears to have become slower. Right now a
> fortran (f2py) version goes ~75% faster and pure python goes ~25%
> faster, so I must be doing something wrong.
>
> Example:
> I changed this:
> dat[xm] = numpy.max(numpy.abs(cf-cx))
>
> to this:
> for i from 0 <= i < n:
> val = abs(cf[i]-cx[i])
> if dat[xm] < val:
> dat[xm] = val
>
> Is that the right approach for removing max/sum ?
>
> And is this:
> cdef extern from "math.h":
> double log(double theta)
> double sqrt(double theta)
> double abs(double theta)
>
> the right way to get rid of numpy log/abs/sqrt calls?
>
> If you'd rather see the whole code, it's been updated on the google code site.
Some possible speedups:
I'd add a mode='c' option to all the cnp.ndarray's -- this will speed up access.
A future optimization would be the @cython.boundscheck(False) directive.
In the inner loops get rid of *all* python operations. For example:
line 63: z = z[z>=xmin]
this is heavy on numpy operations (allocates & discards a temp
boolean array every time, etc) and might kill performance.
line 64: n = float(...) ==> n = <float>(...) # replace a Python
cast with a C-level cast
line 68: a += float(n) / (log(z[i]/xmin)) ==> a += <float>n / (log(z[i]/xmin))
line 76: dat = dat[:xm]
These slicing operations are done through the Python-level,
involving the creation & destruction of python objects, etc. You
might be better off here with a for loop.
line 79: ibid.
line 80: cf = 1-(xmin/z)**a
This is again heavy on Python -- you might do better with a loop
over z and use pow from math.h.
Hope this helps (and let us know if it doesn't).
Kurt
>
> Thanks,
> Adam
> _______________________________________________
> Cython-dev mailing list
> [email protected]
> http://codespeak.net/mailman/listinfo/cython-dev
>
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
