On Tue, Jul 14, 2009 at 12:32 PM, Stefan Behnel<[email protected]> wrote: > > Robert Bradshaw wrote: >> On Jul 12, 2009, at 8:24 PM, Kurt Smith wrote: >> >>> In the midst of GSoC, I wanted to float an idea about a Cython >>> profiler, similar to the current Python profiler. Specifically I >>> wanted to see if there's interest, what sorts of features would be >>> desired, and, broadly, some implementation ideas. > > +1, totally interested. > [snip] > > The main problem I always had here is the tool barrier. You can use > cProfile for profiling Python code, but it can't look into the Cython > functions. Profiling Cython itself will show you loads of time spent inside > the main parser function, without further detail. That's not quite the > right level of granularity for optimisations.
Yep. My thinking too. > > And you can use callgrind for C-level profiling, which works very well > thanks to kcachegrind, but is much too low-level for the Python stuff. So > all you see is the internal interpreter calls instead of the code you are > really looking at. And line-level profiling obviously means C-lines, not > Cython lines. Can be ok sometimes, but it's still more low-level than > necessary, and hard to follow in the code you write. > > Having something in-between, maybe something that interfaces with cProfile > and Python's stats module would make things a lot easier. There's already a > conversion script that allows you to look at Python profiling dumps with > kcachegrind, so it might not even be too hard to make it work with both, so > that you could get profiling information at the Python, Cython and C level. This is good -- thanks for the links. Once I get some time I'll take a closer look. > > http://ddaa.net/blog/python/lsprof-calltree > http://oubiwann.blogspot.com/2006/08/python-and-kcachegrind.html > http://www.gnome.org/~johan/lsprofcalltree.py > > [snip] > > Absolutely. After all, profiling is not more than collecting information > like "I'm here, time is XYZ". We emit tons of error handling code that > contains line information, so this would just be one line more for each > line of Cython source, and likely similar to the code generated for the > ref-nanny. > > I'd also certainly go for emit_marker() here (which simply needs to get > fixed wherever it's not accurate enough to mark each line). We need to make > sure that this only emits code inside of C function bodies, though. > Alright, so maybe my transform idea is going the way of the dodo. That's fine, especially if it's too much for this task. > > It would be good to look into the Python extensions to gdb first, I'd say. > Something like that might work well enough already. After all, when you > debug, you *must* see the C code, so that's different from profiling. > I had no idea there were python extensions to gdb. I've got a bit of reading to do for the CEP. Thanks, Kurt _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
