Salam Muhaimin, So to answer my own question, the current practice seems to be to just eyeball
> Geometric Mean -0.0% -0.0% -0.3% -0.1% +0.1% and if the numbers are within historical epsilons of 0, that means no change. For a moment, I thought this was some erratically behaving VM. The compile times are across-the-board lower, nice! -- Kim-Ee On Sat, Sep 28, 2013 at 4:39 AM, Muhaimin Ahsan <[email protected]> wrote: > Kim-Ee, > > The updated fib-analyse report from a few hours ago is posted here: > https://gist.github.com/leroux/6725810#file-headvordnub-analysis-L2988 > Comment (http://ghc.haskell.org/trac/ghc/ticket/8173#comment:9) > > Sorry for the misunderstanding. > > Muhaimin > > On Sep 27, 2013, at 3:56 PM, Kim-Ee Yeoh <[email protected]> wrote: > > > On Fri, Sep 27, 2013 at 8:14 PM, GHC <[email protected]> wrote: > >> A 5% improvement in compile time is remarkable, if it's true. Great! But >> I'm always worried about the noise in compile times measured in seconds. >> > > Does anyone else think the noise in runtimes is alarming considering that > the following is the fib-analysis of /binary-identical/ programs? > > > Min -0.1% -0.0% -25.4% -32.2% -1.3% > > Max +0.1% +0.0% +19.0% +22.2% +10.0% > > Shouldn't we find an explanation for this before believing the compile > time numbers? What would cause these wide swings on the benchmarking > machine? > > p.s. For the record: Should do more rigorous statistical testing instead > of naive percentages, yes? > > -- Kim-Ee > _______________________________________________ > ghc-tickets mailing list > [email protected] > http://www.haskell.org/mailman/listinfo/ghc-tickets > > > > _______________________________________________ > ghc-devs mailing list > [email protected] > http://www.haskell.org/mailman/listinfo/ghc-devs > >
_______________________________________________ ghc-devs mailing list [email protected] http://www.haskell.org/mailman/listinfo/ghc-devs
