I've analyzed this quite a bit at work and the average and median are not very informative. You need the mode, but in most benchmarks the mode is very close to the minimum, so using the minimum is even better.
How is it better?
In speed measurements, all noise is additive (there's no noise that may make a benchmark appear to run faster). There are also quite a few outliers. Recording the average will include a fair amount of noise.
The benchmarked function itself could have a relevant variance, e.g. when using randomized algorithms. You already get that from the LRU table in the array append cache.
