JB, Maybe I was not clear, when I talked about the tests I was thinking more about execute them in parallel in the same machine, this is not the case today for some test suites, and for these the tests need to be refined to support this, and configured via the pom to execute the tests in parallel per method, class, etc. Of course we need to check if this is worth, because I can imagine that the more expensive time for example in the IO case comes from starting the embedded versions of the IOs (e.g. HadoopMiniCluster, MongodExecutable, HBasetestingutility, etc) and not from the tests themselves but this has to be evaluated.
On Wed, Feb 15, 2017 at 5:46 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > On Jenkins it's possible to run several jobs in the same time but on > different executor. That's the easiest way. > > Regards > JB > > On Feb 15, 2017, 10:15, at 10:15, "Ismaël Mejía" <ieme...@gmail.com> > wrote: > >This question got lost in the discussion, but there is a small > >improvement > >that we can do: > > > >> Just to check, are we doing parallel builds? > > > >We are on jenkins, not in travis, there is an ongoing PR to fix this. > > > >What we can improve is to check if we can run some of the test suites > >in > >parallel to gain some extra time. For exemple most of the IOs and some > >runners don't execute tests in parallel. > > > >Ismael > > > >(slightly related), is there a way to get change the timeout of travis > >jobs). Because it is failing most of the time now because of this, and > >it > >is quite noisey to have so many false positives. > > > > > > > > > >On Fri, Feb 10, 2017 at 8:00 PM, Robert Bradshaw < > >rober...@google.com.invalid> wrote: > > > >> On Fri, Feb 10, 2017 at 8:45 AM, Dan Halperin > ><dhalp...@google.com.invalid > >> > > >> wrote: > >> > >> > On Fri, Feb 10, 2017 at 7:42 AM, Kenneth Knowles > ><k...@google.com.invalid > >> > > >> > wrote: > >> > > >> > > On Feb 10, 2017 07:36, "Dan Halperin" > ><dhalp...@google.com.invalid> > >> > wrote: > >> > > > >> > > Before we added checkstyle it was under a minute. Now it's over > >five? > >> > > That's awful IMO > >> > > > >> > > > >> > > Checkstyle didn't cause all that, did it? > >> > > > >> > > >> > The "5 minutes" was going with Aviem's numbers after this change. > >But > >> yes, > >> > Checkstyle alone substantially (>+50%) the time from what it was > >> previously > >> > to adding it back to the default build. > >> > >> > >> Just to check, are we doing parallel builds? > >> > >> > >> > > >> > Noting that findbugs takes quite a lot more time. Javadoc and jar > >are the > >> > > other two slow ones. > >> > > > >> > > RAT is fast. But it has very poor error messages, so we wouldn't > >want a > >> > new > >> > > contributor trying to figure out what is going on without our > >help. > >> > > > >> > > >> > There is a larger philosophical issue here: is there a point of > >Jenkins > >> > precommit testing? Why not just make `mvn install` run everything > >that > >> > Jenkins does? For that matter, why don't committers just push > >directly to > >> > master? Wouldn't that make everyone's life easier? > >> > > >> > I'd argue that's not true. > >> > > >> > 1. Developer productivity -- Jenkins should run many more checks > >than > >> > developers do. Especially time-, resource-, or setup- intensive > >tasks. > >> > 2. Automated enforcement -- Jenkins is better at running the right > >> commands > >> > than we are. > >> > 3. Lower the barrier to entry -- individual developers need not > >have a > >> > running Spark/Flink/Apex/Dataflow setup in order to contribute > >code. > >> > 4. Focus on the user -- someone checking out the code and using it > >for > >> the > >> > first time does not care whether the code style checks or has the > >right > >> > licenses -- that should have been enforced by the Beam team before > >> > committing. > >> > > >> > We should be *very* choosy about what we enforce on every developer > >every > >> > time they go to compile. I probably compile Beam 50x-100x a day. > >> Literally, > >> > the extra minutes you want to add here will cost me an hour daily. > >> > > >> > >> By the same token of having a different bar for the Jenkins presubmit > >vs. > >> what's run locally, I think it makes a lot of sense to run a > >different > >> command for iterative development than you run before creating a pull > >> request. E.g. during development I'll often run only one test rather > >than > >> the entire suite, but do run the entire suite occasionally (often > >before > >> commit, especially before pushing). > >> > >> The contributors guild should give a suggested command to run before > >> creating a PR, right in the docs of how to create a PR, which may be > >more > >> expensive than what you run during development. IMHO, this should be > >fairly > >> comprehensive (certainly tests and checkstyle, maybe javadoc and > >findbugs). > >> This should be the "default" command that the one-time-contributor > >should > >> know. For those compiling 50x or more a day, I think the burden of > >learning > >> a second (or more) cheaper commands is not high, and we could even > >put such > >> a thing in the docs (and hopefully a common maven convention like > >"mvn > >> test"). > >> > >> I've listed the fraction of commits I think will break one of the > >following > >> > if that property is not tested: > >> > > >> > * compiling (100%) > >> > * tests (100%) > >> > * checkstyle (90%) > >> > * javadoc (30%) > >> > * findbugs (5%) > >> > * rat (1%) > >> > > >> > So you can see where I stand and why. I'm sorry that 1/20 PRs has > >Apache > >> > RAT catch a licensing issue or Findbugs catch a threading issue -- > >you > >> can > >> > always get a larger set of the precommit checks using -Prelease, > >though > >> of > >> > course the integration tests and runnableonservice tests may catch > >more > >> > issues still. But I want my developer minutes back for the 95%+ of > >the > >> > rest. > >> > > >> > Dan > >> > > >> >
