Hi,
What is the status of speed.python.org? Where are the benchmarks? Does
anyone try to setup something to run regulary benchmarks and display
data on web pages?
How can I help?
Victor
___
Speed mailing list
Speed@python.org
http://mail.python.org/mai
Great!
2016-02-04 7:48 GMT+01:00 Zachary Ware :
> I'm happy to announce that speed.python.org is finally functional!
> There's not much there yet, as each benchmark builder has only sent
> one result so far (and one of those involved a bit of cheating on my
> part), but it's there.
>
> There are l
I don't think that using a fixed number of iterations is good to get
stable benchmark results. I opened the following issue to discussed
that:
https://bugs.python.org/issue26275
I proposed to calibrate the number of runs and the number of loops
using time. I'm not convinced myself yet that it's a
2016-02-11 19:36 GMT+01:00 Brett Cannon :
> Are we happy with the current benchmarks?
bm_regex8 looks unstable, but I don't know if it's an issue of the
benchmark itself or perf.py (see the other thread "[Speed] Any changes
we want to make to perf.py?").
I spent a lot of time (probably too much!)
Hi,
To run "micro"-benchmarks on "micro"-optimizations, I started to use
timeit, but in my experience timeit it far from reliable.
When I say micro: I'm talking about a test which takes less than 1000
ns, sometimes even a few nanoseconds!
You always have to run the same micro-benchmark when time
Hi,
I'm sharing with you my notes (tricks) to get more reliable benchmarks
on Linux if your CPU have multiple cores:
https://haypo-notes.readthedocs.org/microbenchmark.html#reliable-micro-benchmarks
FYI perf.py recently got a new --affinity= optional parameter. I plan
to send a patch to automati
2016-02-11 23:50 GMT+01:00 Yury Selivanov :
> That said, I'd like to find a better alternative to spectral-norm, something
> real, that stresses ints/floats and not using numpy.
Case Van Horsen mentioned mpmath test suite:
https://bugs.python.org/issue21955#msg259859
I extracted the slowest test
2016-02-12 0:16 GMT+01:00 Yury Selivanov :
> Unless 21955 makes numpy code slower, I'm not sure why we're discussing
> this.
Stefan Krah wrote that it makes the decimal module 6% slower:
https://bugs.python.org/issue21955#msg259571
Again in another message, "big slowdown for _decimal":
https://bu
Hi,
2016-02-12 16:42 GMT+01:00 Maciej Fijalkowski :
> timeit does two really terrible things - uses min(time) and disables
> the garbage collector, which makes it completely unreliable.
Can you please elaborate why using min(times) is a bad idea?
I'm also using min() in my tool, I expect that it
Hi,
Last months, I spent a lot of time on microbenchmarks. Probably too
much time :-) I found a great Linux config to get a much more stable
system to get reliable microbenchmarks:
https://haypo-notes.readthedocs.org/microbenchmark.html
* isolate some CPU cores
* force CPU to performance
* disabl
Hi,
The problem with not cheating on the statistical distribution is that
I would like to get a quick feedback on my changes to know if my
change is faster or not. Having to wait 1 hour to check a single
change is not really convenient. I prefer to get a feedback is less
than 5 minutes. Tune Linux
Hi,
2016-04-26 10:56 GMT+02:00 Armin Rigo :
> Hi,
>
> On 25 April 2016 at 08:25, Maciej Fijalkowski wrote:
>> The problem with disabled ASLR is that you change the measurment from
>> a statistical distribution, to one draw from a statistical
>> distribution repeatedly. There is no going around do
Hi,
2016-04-26 11:01 GMT+02:00 Antonio Cuni :
> On Mon, Apr 25, 2016 at 12:49 AM, Victor Stinner
> wrote:
>> Last months, I spent a lot of time on microbenchmarks. Probably too
>> much time :-) I found a great Linux config to get a much more stable
>> system to get
2016-04-26 18:36 GMT+02:00 Antoine Pitrou :
> The minimum is a reasonable metric for quick throwaway benchmarks as
> timeit is designed for, as it has a better hope of alleviating the
> impact of system load (as such throwaway benchmarks are often run on
> the developer's workstation).
IMHO we mus
amp; CPU
isolation). Again, deadcode1 looks slower in most cases, whereas
deadcode2 looks faster in most cases, whereas the difference is still
dead code...
Victor, disappointed
changeset: 101088:96628516a16b
user:Victor Stinner
date:Mon Apr 25 21:34:47 2016 +0200
files:
> For more benchmarks, see attached deadcode1.log and deadcode2.log:
> results of the CPython benchmark to compare deadcode1 VS reference,
> and deadcode2 VS reference run on my desktop PC (perf.py --fast & CPU
> isolation). Again, deadcode1 looks slower in most cases, whereas
> deadcode2 looks fas
Hi,
2016-04-27 20:30 GMT+02:00 Brett Cannon :
> My first intuition is some cache somewhere is unhappy w/ the varying sizes.
> Have you tried any of this on another machine to see if the results are
> consistent?
On my laptop, the performance when I add deadcode doesn't seem to
change much: the de
Hi,
I'm still having fun with microbenchmarks. I disabled Power States
(pstate) of my Intel CPU and forced the frequency for 3.4 GHz. I
isolated 2 physical cores on a total of 4. Timings are very stable
*but* sometimes, I get impressive slowdown: like 60% or 80% slower,
but only for a short time.
Hi,
I'm still (!) investigating the reasons why the benchmark call_simple
(ok, let's be honest: the *micro*benchmark) gets different results for
unknown reasons.
(*) Collisions in hash tables: perf.py already calls the benchmark
with PYTHONHASHSEED=1 to test the same hash function. A more generic
(the change can be a single MHz: 3400 => 3401). I didn't notice such
minor CPU frequency change, but I didn't check carefully.
Anyway, I disabled the Turbo Mode and Hyperthreading in the EFI. It
should avoid the strange performance "drop".
Victor
2016-05-17 16:44 GMT+02:00
2016-05-18 8:55 GMT+02:00 Maciej Fijalkowski :
> I think you misunderstand how caches work. The way caches work depends
> on the addresses of memory (their value) which even with ASLR disabled
> can differ between runs. Then you either do or don't have cache
> collisions. How about you just accept
2016-05-18 10:45 GMT+02:00 Armin Rigo :
> On 17 May 2016 at 23:11, Victor Stinner wrote:
>> with PYTHONHASHSEED=1 to test the same hash function. A more generic
>> solution is to use multiple processes to test multiple hash seeds to
>> get a better uniform distribution.
&g
2016-05-18 8:55 GMT+02:00 Maciej Fijalkowski :
> I think you misunderstand how caches work. The way caches work depends
> on the addresses of memory (their value) which even with ASLR disabled
> can differ between runs. Then you either do or don't have cache
> collisions.
Ok. I'm not sure yet that
2016-05-18 20:54 GMT+02:00 Maciej Fijalkowski :
>> Ok. I'm not sure yet that it's feasible to get exactly the same memory
>> addresses for "hot" objects allocated by Python between two versions
>> of the code (...)
>
> Well the answer is to do more statistics really in my opinion. That
> is, perf s
2016-05-18 23:05 GMT+02:00 Paul Graydon :
> Bear in mind that what you see by way of CPU Speed is based on *sampling*,
> and the CPU can be switched speeds very
> quickly. Far faster than you'd necessarily see in your periodic updates.
> Also note that if your cooling isn't up to
> scratch for
FYI I'm running the CPython Benchmark Suite with:
taskset -c 1,3 python3 -u perf.py --rigorous ../ref_python/pgo/python
../fastcall/pgo/python -b all
I was asked to use --rigorous and -b all when I worked on other patches, like:
https://bugs.python.org/issue21955#msg259431
2016-05-19 0:04 GMT+02
2016-05-17 23:11 GMT+02:00 Victor Stinner :
> (...)
>
> (*) System load => CPU isolation, disable ASLR, set CPU affinity on
> IRQs, etc. work around this issue --
> http://haypo-notes.readthedocs.io/microbenchmark.html
>
> (...
>
> (*) Locale, size of the command l
2016-05-30 10:14 GMT+02:00 Antoine Pitrou :
>> I'm still (!) investigating the reasons why the benchmark call_simple
>> (ok, let's be honest: the *micro*benchmark) gets different results for
>> unknown reasons.
>
> Try to define MCACHE_STATS in Objects/typeobject.c and observe the
> statistics from
Hi,
I started to write blog posts on stable benchmarks:
1) https://haypo.github.io/journey-to-stable-benchmark-system.html
2) https://haypo.github.io/journey-to-stable-benchmark-deadcode.html
3) https://haypo.github.io/journey-to-stable-benchmark-average.html
One important point is that minimum
2016-06-02 10:38 GMT+02:00 Armin Rigo :
> Looks good. IMHO the important bit is that `timeit` is simple to use,
> readily available, and gives just a number, which makes it very
> attractive for people.
By default min & max are hidden. You can show them using -v option.
To make the output even s
2016-06-02 9:17 GMT+02:00 Antoine Pitrou :
> Just use a simple JSON format.
Yeah, Python 2.7 includes are JSON parser and JSON is human readble
(but not really designed to be modified by a human).
I had a technical issue: I wanted to produce JSON output *and* keep
nice human output at the same ti
Hi,
I completed the API of my small perf module and released a version 0.2:
https://perf.readthedocs.io/
It is supposed to provide the basic tools to collect samples, compute
the average, display the result, etc. I started to work on JSON
serialization to "easily" run multiple processes. The idea
Hi,
I just released perf 0.3. Major changes:
- "python -m perf" CLI now has 3 commands: show, compare and
compare_to. Compare commands says if the difference is significant (I
copied the code from perf.py)
- TextRunner is now able to spawn child processes, parse command
arguments and more
- If Te
Hi,
performance/bm_pickle.py of the CPython benchmark suite uses the
pickle protocol 2 by default. Why not always testing the highest
protocol?
In Python 3.5, the highest protocol is 4 which is more efficient than
the protocol 2.
Is it a deliberate choice to test exactly the same thing between
P
Hi,
I modified the CPython benchmark suite to use my perf module:
https://hg.python.org/sandbox/benchmarks_perf
Changes:
* use statistics.median() rather than mean() to compute of "average"
of samples. Example:
Median +- Std dev: 256 ms +- 3 ms -> 262 ms +- 4 ms: 1.03x slower
* replace com
2016-07-04 16:17 GMT+02:00 Victor Stinner :
> I modified the CPython benchmark suite to use my perf module:
> https://hg.python.org/sandbox/benchmarks_perf
Hum, you need the development version of perf to test it:
git clone https://github.com/haypo/perf.git
> Changes:
>
> * r
2016-07-04 19:49 GMT+02:00 Antoine Pitrou :
>>Median +- Std dev: 256 ms +- 3 ms -> 262 ms +- 4 ms: 1.03x slower
>
> That doesn't sound like a terrific idea. Why do you think the median
> gives a more interesting figure here?
When the distribution is uniform, mean and median are the same. In my
2016-07-05 10:08 GMT+02:00 Antoine Pitrou :
>> When the system noise is high, the skewness is much larger. In this
>> case, median looks "more correct".
>
> It "looks" more correct?
My main worry is to get reproductible "stable" benchmark results. I
started to work on perf because most results of
2016-07-05 12:08 GMT+02:00 Antoine Pitrou :
> If it's a gaussian curve (not a curse, probably :-)), then you can
> summarize it with two values: the mean and the stddev. But it's
> probably not a gaussian, because of system noise and other factors, so
> your assumption is wrong :-)
What do you pr
Hi,
I'm pleased to announce the release of the Python perf modul version 0.6.
The main difference is that the JSON format and perf command line
tools support benchmark suites and not only individual benchmarks.
I added a new "convert" command which allows to modify a benchmark
file: remove bench
2016-07-04 16:17 GMT+02:00 Victor Stinner :
> I modified the CPython benchmark suite to use my perf module:
> https://hg.python.org/sandbox/benchmarks_perf
Updates with the release of perf 0.6.
runner.py now has 3 commands: run, compare, run_compare
* "run" runs benchmarks on
2016-07-06 18:41 GMT+02:00 Antoine Pitrou :
> I'm not sure this is meant to implement my suggestion from the other
> thread,
Yes, I implemented this after the discussion we had in the other thread.
> but if so, there is a misunderstanding: I did not suggest to
> remove the samples outside of the
2016-07-06 22:24 GMT+02:00 Antoine Pitrou :
>> 5% smallest/5% largest: do you mean something like sorting all
>> samples, remove items from the two tails?
>>
>> Something like sorted(samples)[3:-3] ?
>
> Yes.
Hum, it may work if the distribution is uniform (symmetric), but
usually the right tail i
Hi,
I released perf 0.7 (quickly followed by a 0.7.1 bugfix):
http://perf.readthedocs.io/
I wrote this new version to collect more data in each process. It now
reads (and stores) CPUs config, CPUs temperature, CPUs frequency,
system load average, etc. Later we can add for example the process RSS
Hi,
tl;dr Do you use "perf.py --track_memory"? If yes, for which purpose?
Are you using it on Windows or Linux?
I'm working on the CPython benchmark suite. It has a --track_memory
command option to measure the peak of the memory usage. A main process
runs worker processes and track their memory
Hi,
I updated all benchmarks of the CPython Benchmark Suite to use my perf
module. So you get timings of all individual run all *all* benchmarks
and store them in JSON to analyze them in detail. Each benchmark has a
full CLI, for example it gets a new --output option to store result as
JSON direct
I modified my perf module to add two new options: --tracemalloc and
--track-memory.
--tracemalloc enables tracemalloc and gets the peak of the traced
Python memory allocations: peak is computed per process.
--track-memory is similar but reads PeakPagefileUsage of
GetProcessMemoryInfo() on Windows
2016-07-29 19:03 GMT+02:00 Brett Cannon :
>> * Should it support --track_memory? it doesn't support it correctly
>> right now, it's less precise than before (memory is tracked directly
>> in worker processes, no more by the main process) => discuss this
>> point in my previous email
>
> I don't hav
2016-07-29 20:51 GMT+02:00 Zachary Ware :
> I think rather than using virtual environments which aren't truly
> supported by <3.3 anyway, ...
What do you mean? I'm building and destroying dozens of venv everyday
at work using tox on Python 2.7. The virtualenv command works well,
no? Do you have is
2016-07-30 19:48 GMT+02:00 Armin Rigo :
> Hi Victor,
>
> Fwiw, there is some per-OS (and even apparently
> per-Linux-distribution) solution mentioned here:
> http://stackoverflow.com/questions/774556/peak-memory-usage-of-a-linux-unix-process
>
> For me on Arch Linux, "/usr/bin/time -v CMD" returns
The Python perf module is a toolkit to write, run, analyze and modify
benchmarks:
http://perf.readthedocs.io/en/latest/
Version 0.7.3 (2016-08-17):
* add a new ``slowest`` command
* convert: add ``--extract-metadata=NAME``
* add ``--tracemalloc`` option: use the ``tracemalloc`` module to track
Ok, I just added the mem_max_rss metadata in the development version
of the perf module, just after the perf 0.7.3 release.
Victor
2016-08-08 3:38 GMT+02:00 Victor Stinner :
> 2016-07-30 19:48 GMT+02:00 Armin Rigo :
>> Hi Victor,
>>
>> Fwiw, there is some per-OS (and even ap
Hi,
After a few months of work, I created a "new" benchmark suite for Python:
https://github.com/python/benchmarks
It's based on:
https://hg.python.org/sandbox/benchmarks_perf
which is my fork the CPython benchmark suite:
https://hg.python.org/benchmarks
which is based on Unladen Swallo
2016-08-18 2:37 GMT+02:00 Victor Stinner :
> PyPy, Pyston, Pyjion, Numba, etc. : Hey! it's now time to start to
> take a look at my project and test it ;-) Tell me what is broken, what
> is missing, and I will try to help you to move your project to this
> new benchmark suite!
I
2016-08-18 6:47 GMT+02:00 Zachary Ware :
> Also, if you're interested in having your interpreter benchmarked on
> speed.python.org, contact me with clear instructions (preferably in
> the form of a shell or Python script) on how to build, test, install,
> and invoke your interpreter from a fresh Ub
2016-08-18 8:48 GMT+02:00 Armin Rigo :
> Indeed, bzr cannot be installed on PyPy because it uses Cython in a
> strange way: it declares and directly pokes inside PyListObjects from
> a .pyx file. But note that bzr (seems to) have systematically a pure
> Python version of all its .pyx files. (...)
Hi Maciej,
2016-08-18 9:17 GMT+02:00 Maciej Fijalkowski :
> Did you look into integrating the pypy benchmarks that we added over the
> years?
Not yet, but yes, I plan to collect benchmarks from PyPy, Pyston, Pyjion, etc.
Later we can discuss if some benchmarks should be disabled in the
default
2016-08-18 3:17 GMT+02:00 Victor Stinner :
> I made a quick test on PyPy: (...)
I ran a longer test last night. Some benchmarks were slower on PyPy.
In fact, the benchmark doesn't give enough time to PyPy to warm up its
JIT.
I just released perf 0.7.4 which has a better calibration
Le 21 août 2016 11:02 AM, "Maciej Fijalkowski" a écrit :
> Let's not have it called "bzr" though because it gives the wrong
> impression.
The benchmark was called "bzr_startup", bit I agree that "django" mus be
renamed to "django_template". There is an HTTP benchmark, tornado_http,
using the loca
Done: I renamed "django" benchmark" to "django_template":
https://github.com/python/benchmarks/commit/d674a99e3a9a10a29c44349b2916740680e936c8
Victor
2016-08-21 19:36 GMT+02:00 Victor Stinner :
> Le 21 août 2016 11:02 AM, "Maciej Fijalkowski" a écrit :
&g
Hi,
I released a first version of the Python benchmark suite. (Quickly
followed by a 0.1.1 bugfix ;-))
It is now possible to install it using pip:
python3 -m pip install performance
And run it using:
pyperformance run --python=python2 --rigorous -b all -o py2.json
pyperformance run --
2016-08-24 17:38 GMT+02:00 Victor Stinner :
> Now the development version always install performance 0.1.1 (see
> performance/requirements.txt). I should fix this to install the
> development version of performance/ when it is run from the source
> code (when setup.py is available i
Hi,
For the first release of the "new" benchmark suite, I chose the name
"performance", since "benchmark" and "benchmarks" names were already
reserved on PyPI. It's the name of the Python module, but also of the
command line tool: "pyperformance".
Since there is an "old" benchmark suite
(https://
Le vendredi 26 août 2016, Antoine Pitrou a écrit :
> On Fri, 26 Aug 2016 00:07:53 +0200
> Victor Stinner >
> wrote:
> >
> > By the way, I don't know if it's worth it to have a "pyperformance"
> > command line tool. You can already use "
Release early, release often: performance 0.1.2 has been released! The
first version supporting Windows. I renamed the GitHub project from
python/benchmarks to python/performance.
All changes:
* Windows is now supported
* Add a new ``venv`` command to show, create, recrete or remove the
virtual
Hi,
Would it be possible to run a new instance of CodeSpeed (the website
behing speed.python.org) which would run the "performance" benchmark
suite rather than the "benchmarks" benchmark suite? And would it be
possible to run it on CPython (2.7 and 3.5 branches) and PyPy (master
branch, maybe also
2016-09-01 19:53 GMT+02:00 Kevin Modzelewski :
> Just my two cents -- having a benchmark change underneath the benchmark
> runner is quite confusing to debug, because it looks indistinguishable from
> a non-reproducible regression that happens in the performance itself.
I agree. That's why I propo
Hi,
I released performance 0.2.2. Compared to performance 0.1:
* it fixes the --track-memory option, adds a new "show" command,
* enhance "compare" output (display python & performance versions, use
sample units: seconds or bytes),
* it isolates again environment variables (fix --inherit-environ
Hi,
I released perf 0.7.11. News since perf 0.7.3:
* Support PyPy
* Add units to samples: second, byte, integer. Benchmarks on the
memory usage (track memory) are now displayed correctly.
* Remove environment variables: add --inherit-environ cmdline option.
* Add more metadata: mem_max_rss, pytho
Hi,
While analyzing a performance regression (
http://bugs.python.org/issue28243 ) , I had a major issue with my
benchmark. Suddenly, for no reason, after 30 minutes of benchmarking,
the benchmark became 2x FASTER...
A similar issue occurred to me the last week when testing if PGO
compilation mak
2016-09-23 11:23 GMT+02:00 Antoine Pitrou :
> Did the benchmark really become 2x faster, or did the clock become 2x
> slower?
>
> If you found a magic knob in your CPU that suddently makes it 2x
> faster, many people would probably like to hear about it ;-)
He he. It's a matter of point of view :-
2016-09-23 12:19 GMT+02:00 Antoine Pitrou :
> On Fri, 23 Sep 2016 11:44:12 +0200
> Victor Stinner
> wrote:
>> I guess that for some reasons, the CPU frequency was 1.6 GHz (min
>> frequency) even if I configured the CPU frequency governor to
>> performance.
>
&
I wrote a long article explaing how I identified the bug by testing
Turbo Boost, CPU temperature, CPU frequency, etc.
https://haypo.github.io/intel-cpus-part2.html
Copy/paste of my conclusion:
To get stable benchmarks, the safest fix for all these issues is
probably to set the CPU frequency of th
Hi,
I made further tests and I understood better the issue.
In short, the intel_pstate driver doesn't support NOHZ_FULL, and so
the frequency of CPUs using NOHZ_FULL depends on the workload of other
CPUs. This is especially true when using the powersave (default) cpu
frequency governor. At least,
2016-09-29 17:11 GMT+02:00 Armin Rigo :
> On my laptop, the speed ranges between 500MHz and 2300MHz.
Oh I see, on this computer the difference can be up to 5x slower!
>> * (Use NOHZ_FULL but) Force frequency to the maximum
>> * Don't use NOHZ_FULL
>
> I think we should force the frequency to a s
Hi,
I always wanted to be able to compare the performance of two Python
versions using timeit *in a single command*. So I just implemented it!
I added --python and --compare-to options.
Real example to show the new "timeit --compared-to" feature:
---
$ export PYTHONPATH=~/prog/GIT/perf
$ ./pytho
Hi,
I just released performance 0.3, the Python benchmark suite, with 10
new benchmarks from the PyPy benchmark suite:
https://github.com/python/performance
Version 0.3.0 changelog.
New benchmarks:
* Add ``crypto_pyaes``: Benchmark a pure-Python implementation of the AES
block-cipher in CTR m
I didn't copy/paste code from PyPy benchmaks directly. I updated 3rd
party dependencies, I updated the code to use the perf API, and
sometimes I even fixed bugs in the benchmarks.
2016-10-11 2:22 GMT+02:00 Victor Stinner :
> * Add ``sqlalchemy_declarative`` and ``sqlalchemy_im
Hi,
Latest enhancements:
1) perf supports multiple benchmarks per script
2) perf calibrates the benchmark in a dedicated process
3) new --duplication option to perf timeit
(1)
I solved an old limitaton of my perf module: since perf 0.8, it's now
possible to run multiple benchmarks in a simple s
If you are not subscribed to the Python-Dev mailing list, here is the
copy of the email I just sent.
Victor
-- Forwarded message --
From: Victor Stinner
Date: 2016-10-20 12:56 GMT+02:00
Subject: Benchmarking Python and micro-optimizations
To: Python Dev
Hi,
Last months, I
Hi,
I removed all old benchmarks results and I started to run manually
benchmarks. The timeline view is interesting to investigate
performance regression:
https://speed.python.org/timeline/#/?exe=3&ben=grid&env=1&revs=50&equid=off&quarts=on&extr=on
For example, it seems like call_method became sl
2016-11-02 11:04 GMT+01:00 Armin Rigo :
> On 19 October 2016 at 18:55, Victor Stinner wrote:
>> 3) new --duplication option to perf timeit
>
> This is never a good idea on top of PyPy, so I wouldn't mind if using
> this option on top of PyPy threw an error.
Can you pl
Hum, so for an usability point of view, I think that the best to do is
to ignore the option if Python has a JIT.
On CPython, --duplicate makes sense (no?). So for example, the
following command should use duplicate on CPython but not on PyPy:
python2 -m perf timeit '[1,2]*1000' --duplicate=100
2016-11-02 15:20 GMT+01:00 Armin Rigo :
> Is that really the kind of examples you want to put forward?
I am not a big fan of timeit, but we must use it sometimes to
micro-optimizations in CPython to check if an optimize really makes
CPython faster or not. I am only trying to enhance timeit.
Unders
Hi,
Last months, I used various shell and Python scripts to tune a system
to run benchmarks.
There are many parameters to be set to "fully" configure a Linux:
* Turbo Boost of Intel CPU
* CPU scaling governor
* CPU speed
* Isolate CPU
* Disable kernel RCU on isolated CPUs
* etc.
I added a new "
Hi,
Good news, I regenerated all benchmark results of CPython using the
latest versions of perf and perfomance and the results look much more
reliable. Sadly, I didn't kept a screenshot of old benchmarks, so you
should trust me, I cannot show you the old unstable timeline.
--
I regenerated all b
2016-11-04 20:18 GMT+01:00 Miquel Torres :
> Nice! For the record, I'll be giving a talk in PyCon Ireland about
> Codespeed. Would you mind me citing those tweets and screenshots, to
> highlight usage on speed.python.org?
Sure. Keep me in touch in you publish your slides later.
> You mentioned n
2016-11-04 20:21 GMT+01:00 Yury Selivanov :
> I'm curious why call_* benchmarks became slower on 3.x?
It's almost the same between 2.7 and default. For 3.6, it looks like
an issue on the benchmark runner, not on Python itself:
>> Please ignore call_method, call_method_slots, call_method_unknown
>
Hi,
I noticed a temporary performance peak in the call_method:
https://speed.python.org/timeline/#/?exe=4&ben=call_method&env=1&revs=50&equid=off&quarts=on&extr=on
The difference is major: 17 ms => 29 ms, 70% slower!
I expected a temporary issue on the server used to run benchmarks,
but... I re
I found some interesting differences using the Linux perf tool.
# perf stat -e L1-icache-loads,L1-icache-load-misses ./python
performance/benchmarks/bm_call_method.py --inherit=PYTHONPATH -v
--worker -l1 -n 25 -w0
2016-11-04 23:35 GMT+01:00 Victor Stinner :
> (1) ce85a1f129e3: 17
I proposed a patch which fixes the issue:
http://bugs.python.org/issue28618
"Decorate hot functions using __attribute__((hot)) to optimize Python"
Victor
___
Speed mailing list
Speed@python.org
https://mail.python.org/mailman/listinfo/speed
2016-11-04 21:58 GMT+01:00 Victor Stinner :
> 2016-11-04 20:21 GMT+01:00 Yury Selivanov :
>> I'm curious why call_* benchmarks became slower on 3.x?
>
> It's almost the same between 2.7 and default. For 3.6, it looks like
> an issue on the benchmark runner, not on P
2016-11-05 15:56 GMT+01:00 Nick Coghlan :
> Since the use case for --duplicate is to reduce the relative overhead
> of the outer loop when testing a micro-optimisation within a *given*
> interpreter, perhaps the error should be for combining --duplicate and
> --compare-to at all? And then it would
Hello,
> The OpenStack-Ansible project has noticed that performance on Ubuntu 16.04 is
> quite significantly slower than on 14.04.
> At the moment it's looking like *possibly* a GCC related bug.
Is it exactly the same Python version? What is the full version?
Try to get compiler flags:
python2
Hi,
I'm happy because I just finished an article putting the most
important things that I learnt this year on the most silly issue with
Python performance: code placement.
https://haypo.github.io/analysis-python-performance-issue.html
I explain how to debug such issue and my attempt to fix it in
Le 19 nov. 2016 21:29, "serge guelton" a écrit :
> Thanks *a lot* victor for this great article. You not only very
> accurately describe the method you used to track the performance bug,
> but also give very convincing results.
You're welcome. I'm not 100% sure that adding the hot attrbute makes
Hi,
tl;dr I'm trying to run benchmarks with PGO, but I get new errors, so
speed.python.org is currently broken.
Last december, I upgraded the speed-python server (server used to run
benchmarks for CPython) from Ubuntu 14.04 (LTS) to 16.04 (LTS) to be
able to compile Python using PGO compilation.
2017-01-03 0:56 GMT+01:00 Victor Stinner :
> I'm now trying to run benchmarks one more time... ;-)
Good news: results are slowly being uploaded to https://speed.python.org/
It takes 50 minutes to benchmark one revision: first compilation, run
the test suite to train the compiler, re
Hi Peter,
Thank you for your bug report. The bug is a recent regression in the
perf module. I just fixed it with the newly released perf 0.9.3.
http://perf.readthedocs.io/en/latest/changelog.html#version-0-9-3-2017-01-16
I also released performance 0.5.1 which now uses perf 0.9.3. By the
way, I f
Hi,
I annoyed many of you on this mailing list with all my WTF performance
issues. I consider that I now succeeded to get benchmarks stable
enough to become "usable" (reliable). Two days ago, I gave a talk "How
to run a stable benchmark" at FOSDEM (Brussels, Belgium) on my
findings.
Slides and vi
1 - 100 of 188 matches
Mail list logo
