It seems to be that this thought is in the air right now. This was done just a few days ago: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7184
https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7231 also looks relevant. Richard > On Dec 22, 2021, at 7:19 AM, Joachim Breitner <m...@joachim-breitner.de> > wrote: > > Hi, > > the new (or “new”?) handling of perf numbers, where CI just magically > records and compares them, without us having to manually edit the > `all.T` files, is a big improvement, thanks! > > However, I found the choice of the base commit to compare against > unhelpful. Assume master is at commit M, and I start a feature branch > and MR with commit A. CI runs, and tells me about a performance > regressions, and CI is red. I now fix the issue and push commit B to > the branch. CI runs, but it picks A to compare against, and now it is > red because of an seemingly unexpected performance improvement! > > I would have expected that all CI runs for this MR to compare the > performance against the base branch on master, and to look for perf > change notices in all commit messages in between. > > I see these advantages: > > * The reported perf changes correspond to the changes shown on the MR > page > * Green CI = the MR is ready (after squashing) > * CI will have numbers for the base commit more reliably > (else, if I push commit C quickly after B, then the job for B might > be cancelled and Ci will report changes of C against A instead of B, > which is unexpected). > > I have used this logic of reporting perf changes (or any other > “differential CI”) against the base branch in the Motoko project and it > was quite natural. > > Would it be desirable and possible for us here, too? > > > (A possible rebuttal might be: we don’t push new commits to feature > branches, but always squash and rebase, as that’s what we have to do > before merging anyways. If that’s the case then ok, although I > generally lean to having chronological commits on feature branches and > a nice squashed commit on master.) > > Cheers, > Joachim > > > -- > Joachim Breitner > m...@joachim-breitner.de > http://www.joachim-breitner.de/ > > _______________________________________________ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs