On 11/04/2014 11:55, Johan Tibell wrote:
On Fri, Apr 11, 2014 at 10:58 AM, Simon Marlow <marlo...@gmail.com <mailto:marlo...@gmail.com>> wrote:Let me second this. In particular, I think we regress on GHC performance regularly, because the perf tests just aren't a good way to prevent regressions. When we have a +/- 10% window, someone can commit a 9% regression without triggering a perf failure, but the next patch to come along with a 1% regression will be unfairly blamed. Agreed. I think graphing results helps here. It's often easier to visualize identify which commit is the real culprit. Aside: I think moving completely to subrepos will generally help us track down regressions, both performance and correctness, faster. Being able to `git bisect` your way to the cause saves a lot of time. Furthermore, by the time we get to the perf tests we're nearly done and just want to push and go home, not go back and profile GHC. Yet the perf tests have an important purpose: the idea is to catch the problem when we have the crucial piece of information: the patch that caused the regression. Someone can try to optimise GHC later, but they have to start from scratch without the information about what caused the regressions. Having an automated system to track GHC performance would help a lot with this, I think. Agree a 100%. Automation is what's needed here.
Just to add one thing: when I wrote that above I was thinking primarily about the performance of GHC itself, but of course it all applies to both GHC and the code that GHC generates. Since the latter also affects the former, if we track both together we'll be able to see when we have changes in GHC performance that aren't related to changes in compiled code performance.
I care a *lot* about the performance of GHC itself these days, the performance of GHC will directly impact how fast we can get code into production.
Cheers, Simon _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
