On 06/16/2012 09:57 PM Jan Decaluwe wrote:
On 06/16/2012 05:20 PM, Maciej Fijalkowski wrote:
Have a look at our current benchmarks. They're on
bitbucket.org/pypy/benchmarks<http://bitbucket.org/pypy/benchmarks>.
Look at the format. Ideally they would execute a few different kinds
of examples (say 5) each running for about a second?
At only a second for a benchmark, how do you account for
the JIT warming-up phase?
In MyHDL simulations, I have found the JIT warming-up phase
to be quite significant (which is OK, because I get such
good results afterwards).
I run the benchmarks for around 1 minute to remove
the influence of the JIT warming-up phase from the speed-up
factor. Probably you run the benchmarks on a much faster
machine than my i3 laptop, but still.
Re faster and slower machines, I wonder if it wouldn't be interesting
to have an option to report results in gigabogoinst instead of,
or in addition to, seconds, (by multiplying seconds by bogogips (==
bogomips/1000 ;-)), e.g.,
>>> bogogips=(lambda b=[float(line.split(':')[1].strip()) for line in open('/proc/cpuinfo') if
line.startswith('bogomips')]:sum(b)/len(b))()/1000.0
>>> bogogips
3.99242
would be the factor on my laptop, averaging the bogomips of all (two) CPUs.
If the jit processing had optional callback hooks returning ('<hook id>',
time.time()) that
a detailed benchmarker could use to see what activities were taking what time,
that would be cool too.
Maybe you already have something like that?
Regards,
Bengt Richter
_______________________________________________
pypy-dev mailing list
pypy-dev@python.org
http://mail.python.org/mailman/listinfo/pypy-dev
