The split did seemingly trim about 30 minutes off the Java precommit. Of course the difference between 50 and 80 minutes won't qualitatively change much. I don't see any other obvious and easy wins. I still like the split for the separation of signals.
Kenn On Tue, Oct 23, 2018 at 2:47 PM Robert Bradshaw <[email protected]> wrote: > On Tue, Oct 23, 2018 at 11:28 PM Kenneth Knowles <[email protected]> wrote: > >> Hi all, >> >> Java Precommit duration is about 1h15. That is quite a burden. Especially >> if something gets broken. >> > > I'm in favor of (simple!) build breaks going in before precommits finish, > on the promise that the offending test(s) passed locally. Short of that, we > can roll back. > > If it were cheap to get a fast "this is probably good" signal, that could > be useful as well, though once you hit the "I'm waiting long enough to go > do something else" the difference between 20 minutes and 80 minutes is not > that huge. > > >> We turned off parallel builds, which we really need to re-enable. >> > > +1 > > >> But beyond that, I see low-hanging fruit that would most appropriately be >> a separate Jenkins job. >> >> Here's a scan of a successful run: >> https://scans.gradle.com/s/2s4bd5hc45wuy/timeline >> >> * 17m :beam-runners-google-cloud-dataflow-java-examples:preCommit >> * 4m :beam-runners-google-cloud-dataflow-java-examples-streaming:preCommit >> These are integration tests that should have their own job & status >> anyhow. We lumped them in because Maven can't do separate tests. Gradle >> makes this cheap and easy. >> >> Then there are these which are the only other tasks over 1m: >> >> * 2m :beam-runners-google-cloud-dataflow-java-legacy-worker:test >> * 2m :beam-runners-google-cloud-dataflow-java-fn-api-worker:test >> * 2m :beam-sdks-java-nexmark:test >> * 1m :beam-sdks-java-io-google-cloud-platform:test >> * 1m :beam-sdks-java-io-hbase:test >> * 1m :beam-sdks-java-extensions-sql:test >> >> Maybe not worth messing with these. Also if we remove all the shadowJar >> and shadowTestJar tasks it actually looks like it would only save 5 >> minutes, so I was hasty in thinking that would solve things. It will make >> interactive work better (going from 30s to maybe <10s for rebuilds) but >> won't help that much for Jenkins. >> >> Kenn >> >
