On 17 Mar 2021, at 16:16, Andreas Klebinger <klebinger.andr...@gmx.at> wrote: > > While I fully agree with this. We should *always* want to know if a small > syntetic benchmark regresses by a lot. > Or in other words we don't want CI to accept such a regression for us ever, > but the developer of a patch should need to explicitly ok it. > > Otherwise we just slow down a lot of seldom-used code paths by a lot. > > Now that isn't really an issue anyway I think. The question is rather is 2% a > large enough regression to worry about? 5%? 10%?
You probably want a sliding window anyway. Having N 1.8% regressions in a row can still slow things down a lot. While a 3% regression after a 5% improvement is probably fine. - Merijn
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
