[somehow my email got garbled...] Now that we're using gradle, perhaps we could be more intelligent about only running the affected tests? E.g. when you touch Python (or Go) you shouldn't need to run the Java precommit at all, which would reduce the latency for those PRs and also the time spent in queue. Presumably this could even be applied per-module for the Java tests. (Maybe a large, shared build cache could help here as well...)
I also wouldn't be opposed to a quicker immediate signal, plus more extensive tests before actually merging. It's also nice to not have to wait an hour to see that you have a lint error; quick stuff like that could be signaled quickly before a contributor looses context. - Robert On Fri, May 18, 2018 at 5:55 AM Kenneth Knowles <[email protected]> wrote: > I like the idea. I think it is a good time for the project to start > tracking this and keeping it usable. > > Certainly 2 hours is more than enough, is that not so? The Java precommit > seems to take <=40 minutes while Python takes ~20 and Go is so fast it > doesn't matter. Do we have enough stragglers that we don't make it in the > 95th percentile? Is the time spent in the Jenkins queue? > > For our current coverage, I'd be willing to go for: > > - 1 hr hard cap (someone better at stats could choose %ile) > - roll back or remove test from precommit if fix looks like more than 1 > week (roll back if it is perf degradation, remove test from precommit if it > is additional coverage that just doesn't fit in the time) > > There's a longer-term issue that doing a full build each time is expected > to linearly scale up with the size of our repo (it is the monorepo problem > but for a minirepo) so there is no cap that is feasible until we have > effective cross-build caching. And my long-term goal would be <30 minutes. > At the latency of opening a pull request and then checking your email > that's not burdensome, but an hour is. > > Kenn > > On Thu, May 17, 2018 at 6:54 PM Udi Meiri <[email protected]> wrote: > >> HI, >> I have a proposal to improve contributor experience by keeping precommit >> times low. >> >> I'm looking to get community consensus and approval about: >> 1. How long should precommits take. 2 hours @95th percentile over the >> past 4 weeks is the current proposal. >> 2. The process for dealing with slowness. Do we: fix, roll back, remove a >> test from precommit? >> Rolling back if a fix is estimated to take longer than 2 weeks is the >> current proposal. >> >> >> https://docs.google.com/document/d/1udtvggmS2LTMmdwjEtZCcUQy6aQAiYTI3OrTP8CLfJM/edit?usp=sharing >> >
