Hi, Python ships with a profiler module which, unfortunately, is almost useless in a multi-threaded environment. *
I've created an alternative profiler module which queries per-thread CPU usage via netlink/taskstats, which limits the applicability to Linux (which shouldn't be much of an issue, profiling is usually not done by end users). It implements two modes: a "sampling" (does CPU time accounting based on stack fraames 100 times per second, by default) and a "deterministic" profiler (does CPU time accounting on each function call/return, based on sys.profiler interface). The deterministic profiler is currently implemented in pure python (except for taskstats interface) and much slower than the sampling profiler. Usage (don't forget make to build the c module): python >> from Profiler import * >> def f(): do_something() >> sampling_profiler(f) or >> deterministic_profiler(f) Output is currently in the form of annotated source code (xyz.py.html, in the same directory where xyz.py resides). Before the *_profiler function returns, it iterates over all code objects it encountered and annotates the source files with 2 columns in front: - 1st column: real time - 2nd column: CPU time numbers are log2(time_in_ns), colors are green-to-yellow for below-average and yellow-to-red for above-average metrics (relative to the average metric for all lines of the code object with a metric > 0). Is there common need for such a module? Is it possible to have this included in the standard cPython distribution? Which functional changes (besides a modification of the annotation output which shouldn't spread its result all over the FS) would be required to get this included? Which non-functional changes would be required to get this included? Please direct traffic regarding this subject to pyprof-de...@lists.sourceforge.net (no I'm not subscribed to python-dev). SF project page: https://sourceforge.net/projects/pyprof/ git repository: git://pyprof.git.sourceforge.net/gitroot/pyprof Regards, Christian *) to be more exact there are at least three profiler modules: profile, cProfile, and hotshot, while I did only try (and failed) to use profile in a multi-threaded environment (by manually setting threading.profile to the profiling function), glancing at the source, I'm pretty sure that cProfile behaves similarly; I didn't test the hotshot module, but it does some other trade-offs (space-for-time), so I think that "pyprof" still adds some value
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
