2017-03-16 2:04 GMT+01:00 Wang, Peter Xihong <peter.xihong.w...@intel.com>: > Understood on the obsolete benchmark part. This was the work done before the > new benchmark was created on github.
I strongly advice you to move to performance. It also has a nice a API. It now produces a JSON file with *all* data, instead of just writing into summaries into stdout. > I thought this is related, and thus didn't open a new thread. The other thread was a discussion about statistics, how to summarize all timing into two numbers :-) > Maybe you could point me to one single micro-benchmark for the time being, > and then we could compare result across? The "new" performance project is a fork of the old "benchmark" project. Benchmark names are very close or even the same for many benchmarks. If you would like to validate that your benchmark runner is stable: run call_method and call_simple microbenchmarks on different revisions of CPython, reboot sometimes the computer used to run benchmarks, and make sure that results are stable. Compare them with results of speed.python.org. call_method: https://speed.python.org/timeline/#/?exe=5&ben=call_method&env=1&revs=50&equid=off&quarts=on&extr=on call_simple: https://speed.python.org/timeline/#/?exe=5&ben=call_simple&env=1&revs=50&equid=off&quarts=on&extr=on Around november and december 2016, you should notice a significant speedup on call_method. The best is to be able to avoid "temporary spikes" like this one: https://haypo.github.io/analysis-python-performance-issue.html The API of the perf project, PGO and LTO compilation, new performance using perf, "perf system tune" for system tuning, etc. helped to get more stable results. Victor _______________________________________________ Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed
