On Sunday, 29 May 2016 at 12:22:23 UTC, qznc wrote:
I played around with the benchmark. Some more numbers:
The mean slowdown is 114, which means 14% slower than the
fastest one. The mean absolute deviation (MAD) is 23. More
precisely, the mean deviation above the mean slowdown of 103 is
100 and -13 below the mean slowdown. 1191 of the 10000 runs
were above the mean slowdown and 8770 below. The 39 missing
runs are equal to the mean slowdown.
A minor thing - you might consider also calculating the median
and median version of MAD (median of absolute deviations from the
median). The reason is that benchmarks often have outliers in the
max time dimension, median will do a job reducing the effect of
those outliers than mean. Your benchmark code could publish both
forms.
--Jon
