Hi all, On 07 Feb 2013, at 10:44, Simon Marlow <marlo...@gmail.com> wrote:
> On 06/02/13 22:26, Andy Georges wrote: >> Quantifying performance changes with effect size confidence intervals - >> Tomas Kalibera and Richard Jones, 2012 (tech report) > > This is a good one - it was actually a talk by Richard Jones that highlighted > to me the problems with averaging over benchmarks (aside from the problem > with GM, which he didn't mention). The paper has a guide for practitioners that improves on what I did in part of my PhD. I think it could be fairly easy to wrap that around Criterion for comparing runs -- most of your . I should note that a number of people I know are involved in performance measurement think it is a bit too detailed, but if you can implement this in your testing framework, it could be a cool feature that other people start using too. > This paper mentions Criterion, incidentally. Yes :-) I mentioned it several times when we discussed performance measuring in the Evaluate workshops. Since I changed jobs, I am no longer very actively involved here, but some people seem to have picked things up, I guess. >> • [[1]] J.E., Smith. Characterizing computer performance with a single >> number. CACM 31(10), 1988. > > And I wish I'd read this a long time ago :) Thanks. No more geometric means > for me! You are very welcome. Regards, -- Andy _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
