On Thu, Sep 15, 2016 at 5:57 PM Victor Stinner <victor.stin...@gmail.com>

> 2016-09-15 10:02 GMT+02:00 INADA Naoki <songofaca...@gmail.com>:
> > In my environ:
> >
> > ~/local/python-master/bin/python3 -m timeit -s "d =
> > dict.fromkeys(range(10**6))" 'list(d)'
> Stooooop! Please stop using timeit, it's lying!
> * You must not use the minimum but average or median
> * You must run a microbenchmark in multiple processes to test
> different randomized hash functions and different memory layouts
> In short: you should use my perf module.
> http://perf.readthedocs.io/en/latest/cli.html#timeit
I'm sorry.  Changing habit is bit difficult. I'll use it in next time.

I ran microbench 3~5 times and confirm the result is stable before posting
And when difference is smaller than 10%, I don't believe the result.

> The memory layout and the hash function have a major important on such
> microbenchmark:
> https://haypo.github.io/journey-to-stable-benchmark-average.html
In this microbench, hash randomization is not important, because key
of dict is int.
(It means iterating dict doesn't cause random memory access in old dict
implementation too.)

> > Both Python is built without neither `--with-optimizations` or `make
> > profile-opt`.
> That's bad :-) For most reliable benchmarks, it's better to use
> LTO+PGO compilation.

LTO+PGO  may make performance of `git pull && make` unstable.
PGO clean build takes tooo long time for such a quick benchmark.
So I don't want to use PGO in such a quick benchmark.

And Python doesn't provide way to use LTO without PGO....
Python-Dev mailing list

Reply via email to