Now that we're using gradle, perhaps we could be more intelligent
about only running the affected tests? E.g. when you touch Python (or
Go) you shouldn't need to run the Java precommit at all, which
would reduce the latency for those PRs and also the time spent in
queue. Presumably this could even be applied per-module for the Java
tests. (Maybe a large, shared build cache could help here as
well...)<br><br>I also wouldn't be opposed to a quicker immediate
signal, plus more extensive tests before actually merging. It's
also nice to not have to wait an hour to see that you have a lint
error; quick stuff like that could be signaled quickly before a
contributor looses context. <br>On Fri, May 18, 2018 at 5:55 AM
Kenneth Knowles <k...@google.com> wrote:<br>><br>> I like
the idea. I think it is a good time for the project to start tracking
this and keeping it usable.<br>><br>> Certainly 2 hours is more
than enough, is that not so? The Java precommit seems to take <=40
minutes while Python takes ~20 and Go is so fast it doesn't
matter. Do we have enough stragglers that we don't make it in the
95th percentile? Is the time spent in the Jenkins
queue?<br>><br>> For our current coverage, I'd be willing to
go for:<br>><br>> - 1 hr hard cap (someone better at stats
could choose %ile)<br>> - roll back or remove test from
precommit if fix looks like more than 1 week (roll back if it is perf
degradation, remove test from precommit if it is additional coverage
that just doesn't fit in the time)<br>><br>> There's a
longer-term issue that doing a full build each time is expected to
linearly scale up with the size of our repo (it is the monorepo
problem but for a minirepo) so there is no cap that is feasible until
we have effective cross-build caching. And my long-term goal would be
<30 minutes. At the latency of opening a pull request and then
checking your email that's not burdensome, but an hour
is.<br>><br>> Kenn<br>><br>> On Thu, May 17, 2018 at 6:54
PM Udi Meiri <eh...@google.com> wrote:<br>>><br>>>
HI,<br>>> I have a proposal to improve contributor experience by
keeping precommit times low.<br>>><br>>> I'm looking
to get community consensus and approval about:<br>>> 1. How long
should precommits take. 2 hours @95th percentile over the past 4 weeks
is the current proposal.<br>>> 2. The process for dealing with
slowness. Do we: fix, roll back, remove a test from
precommit?<br>>> Rolling back if a fix is estimated to take
longer than 2 weeks is the current proposal.<br>>><br>>>
https://docs.google.com/document/d/1udtvggmS2LTMmdwjEtZCcUQy6aQAiYTI3OrTP8CLfJM/edit?usp=sharing
