> (1) The build is really a tree of modules. With our recent de-coupling from Storm, this has become all the more exaggerated...
I agree that the approach I followed in my POC suffers from the risk of a future refactor accidently causing some tests not to run. But I just did it this way because it was the simplest approach to see something work. I think we could easily find a way to mitigate or eliminate this risk in a final solution. > (2) I actually like the current limits because it has forced contributors and reviewers to consider the overall build time as they design and add new tests.... I agree with the goal. I want the tests to run as quickly as possible too. But spending time here worried about whether the builds will complete after the cache is invalidated seems like a waste of everyone's time. In addition, I've seen times where my own feature branches CI builds fail because the time limit is breached. I think this tends to happen when Travis is under greater load or when they have some resources down for maintenance or outages (or so I assume). Personally, I am in favor of anything that reduces the time for a developer to get feedback on changes they've made. I like the idea of parallelizing/splitting the integration tests for this reason. On Thu, May 23, 2019 at 11:33 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I think this is a neat idea, however I have a couple concerns with this: > > 1. The build is really a tree of modules. With our recent de-coupling > from Storm, this has become all the more exaggerated. This is a good > thing, > however the benefit of the parent pom approach we currently have is > that it > does a pretty good job of guaranteeing all of the leaf modules get > built as > well. Specifying the modules individually is going to turn this into a > very > manual process any time a new module is added or refactored, and is more > likely to result in bugs. One other option to aid in solving that > problem > is to use aggregator modules, and then have the full local build parent > pom > use those aggregators instead of referring to the individual modules the > way it does now. You still get the top-down tree, but there's less of a > gap > between Travis and our primary Maven structure. > 2. I actually like the current limits because it has forced contributors > and reviewers to consider the overall build time as they design and add > new > tests. Especially with the current set of upgrades and proposed > integration > test and Docker changes, I think we should be extra vigilant on getting > those build/test times *down*, rather than enabling them to increase. > Sure, > there are other options for solving that problem, but Travis is a simple > equalizer because it's impartial to whatever local hardware engineers > are > using. > > > On Wed, May 22, 2019 at 7:48 AM Nick Allen <n...@nickallen.org> wrote: > > > FYI - Here is a POC build of this concept running. This only runs the > > Parser and Enrichment integration tests, but as separate jobs in Travis. > > > > https://travis-ci.org/nickwallen/metron/builds/535791047 > > > > > > > > > > On Wed, May 22, 2019 at 9:20 AM Nick Allen <n...@nickallen.org> wrote: > > > > > > > > > Justin Leet said > > > <https://github.com/apache/metron/pull/1417#issuecomment-494464795> > > [1]: Looking > > > at our build times, I'm actually concerned that if we kill the caches > our > > > builds won't complete. The integration tests take >45 minutes and it's > > very > > > possible redownloading everything goes over our remaining time. > > > > > > To recap, our current Travis CI build is composed of multiple jobs. > > Travis' > > > time limit of 50 minutes > > > <https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts > > > > [2] > > > is per job, rather than the total build time. While our total build > time > > > is on the order of 2.5 hours, it is only our integration-test job which > > is > > > coming close to that 50 minute limit. > > > > > > We could try splitting our integration tests into multiple jobs. Each > of > > > these would have the opportunity to run in parallel (given whatever > > > resources Travis can allocate to us), but more importantly each one has > > its > > > own 50 minute limit. For example... > > > > > > - Parser Integration Tests: > > > - `time mvn surefire:test@integration-tests -pl > > > "metron-platform/metron-parsing/metron-parsing-storm, > > > metron-platform/metron-parsing/metron-parsers, > > > metron-platform/metron-parsing/metron-parsers-common/"` > > > - Enrichment Integration Tests > > > - `time mvn surefire:test@integration-tests -pl > > > > > > "metron-platform/metron-enrichment/metron-enrichment-common/,metron-platform/metron-enrichment/metron-enrichment-storm"` > > > - etc, etc > > > > > > > > > We would just need to determine how to logically split the tests up. > If > > > this sounds reasonable, I've already got a start on a POC. > > > > > > --- > > > [1] https://github.com/apache/metron/pull/1417#issuecomment-494464795 > > > [2] > > https://docs.travis-ci.com/user/customizing-the-build/#build-timeouts > > > > > >
