On Fri, Apr 11, 2014 at 10:58 AM, Simon Marlow <marlo...@gmail.com> wrote:
> Let me second this. In particular, I think we regress on GHC performance > regularly, because the perf tests just aren't a good way to prevent > regressions. When we have a +/- 10% window, someone can commit a 9% > regression without triggering a perf failure, but the next patch to come > along with a 1% regression will be unfairly blamed. > Agreed. I think graphing results helps here. It's often easier to visualize identify which commit is the real culprit. Aside: I think moving completely to subrepos will generally help us track down regressions, both performance and correctness, faster. Being able to `git bisect` your way to the cause saves a lot of time. Furthermore, by the time we get to the perf tests we're nearly done and > just want to push and go home, not go back and profile GHC. Yet the perf > tests have an important purpose: the idea is to catch the problem when we > have the crucial piece of information: the patch that caused the > regression. Someone can try to optimise GHC later, but they have to start > from scratch without the information about what caused the regressions. > > Having an automated system to track GHC performance would help a lot with > this, I think. Agree a 100%. Automation is what's needed here.
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
