Hi. Disabling ASLR means you get more repeatable benchmarks, of course, but also means that on another identical machine (or a bit different circumstances), you can get different results, hence you moved statistical error to a more systematic one. I don't think that's a win
On Fri, Feb 12, 2016 at 8:42 AM, Patrascu, Alecsandru <alecsandru.patra...@intel.com> wrote: > Hi, > > Some of the things we do here at Intel, in our Languages Performance Lab > [1,2], is to disable ASLR as you get more reliable results. This can be > achieved on Linux by running echo 0 > /proc/sys/kernel/randomize_va_space. > Also, setting the CPU frequency at a fixed frequency, disabling Turbo Boost > and Hyper Threading, also helps for benchmark stability. > > From my experience, the isolcpus feature is useful when you have a lot of > cores on your machine because the kernel will have other cores on which it > can schedule its work; furthermore, it is a best effort situation and it is > not an absolute guarantee that the kernel will not use the cores specified if > you have a lot of processes running (for example, if you benchmark on a > machine with 2 physical cores and you isolate one of the cores, there is a > big chance that the kernel will schedule processes on this core also, even it > is for a small amount of time). Nevertheless, for machines with more physical > cores, it can be good to have dedicated core(s) on which we do benchmarking. > > [1] http://languagesperformance.intel.com/ > [2] https://lists.01.org/pipermail/langperf/ > > Thank you, > Alecsandru > >> -----Original Message----- >> From: Speed [mailto:speed- >> bounces+alecsandru.patrascu=intel....@python.org] On Behalf Of Victor >> Stinner >> Sent: Friday, February 12, 2016 12:54 AM >> To: speed@python.org >> Subject: [Speed] Linux tip: use isolcpus to have (more) reliable benchmark >> >> Hi, >> >> I'm sharing with you my notes (tricks) to get more reliable benchmarks on >> Linux if your CPU have multiple cores: >> >> https://haypo-notes.readthedocs.org/microbenchmark.html#reliable-micro- >> benchmarks >> >> FYI perf.py recently got a new --affinity= optional parameter. I plan to >> send a patch to automatically use /sys/devices/system/cpu/isolated if it's >> not empty. >> >> What are your "tricks" to get reliable benchmarks? >> >> Victor >> _______________________________________________ >> Speed mailing list >> Speed@python.org >> https://mail.python.org/mailman/listinfo/speed > _______________________________________________ > Speed mailing list > Speed@python.org > https://mail.python.org/mailman/listinfo/speed _______________________________________________ Speed mailing list Speed@python.org https://mail.python.org/mailman/listinfo/speed
