On 12/06/2009 06:22 PM, chromatic wrote:
On Sunday 06 December 2009 at 08:06, Geoffrey Broadwell wrote:
While I would love for HEAD to be always the fastest, if you mentally
graph this you can see that trunk is in the main grouping of total times
in the 32 to 40 second range. It's the outliers in the 55 to 74 second
range that really aren't looking good.
To me there are two optimization goals:
1. Make sure HEAD is in the same grouping as the fastest time, for all
benchmarks we can find. (Don't ask me for an exact rule for grouping --
but if you need a programmatic rule of thumb for a meta-bench tool,
perhaps cut off at 30% or even 50% worse than the best time.)
I don't trust time-related benchmarkings for this reason. Yes, stochastic
analysis can give you some degree of probability that comparisons are all
like-to-like, but the output of a single Callgrind run is much easier to
compare (and gives a better explanation of *why* and *how* and *where*
performance has changed).
This is where the RTEMS community might be able to offer some
experience when the RTEMS port is complete. We regularly benchmark
on FOSS cycle accurate simulators where the load in the test
is constant across runs. We even benchmark interrupt dispatch
time with no variation. :)
When it comes time to worry more about performance, benchmark
results on various RTEMS target architectures would be good.
--joel sherrill
RTEMS
-- c
_______________________________________________
http://lists.parrot.org/mailman/listinfo/parrot-dev
_______________________________________________
http://lists.parrot.org/mailman/listinfo/parrot-dev
