Hello team, Success of that particular build was pure luck. Point of the commit that triggered the build and several previous commits was to get source code static analysis checks to work and consistently use custom settings from buildtools module. E.g. SE_BAD_FIELD rule/check is excluded from findbugs checks. Before these changes findbugs maven report on Jenkins used to list that as high prio findbugs issue<https://builds.apache.org/job/Mahout-Quality/2118/findbugsResult/HIGH/module.-1810137467/package.-1319752439/>in modules like mahout-core, now it doesn't<https://builds.apache.org/job/Mahout-Quality/2131/findbugsResult/HIGH/>. I wanted to work on eliminating those quality issues, and found lack of consistency between Jenkins and IDE annoying enough to go and fix it before 0.8 release, even while on vacation - island of Thassos, Greece, it's nice here btw, you should all come :-)
What did the trick (as of r1500216) for last two builds to be successful was serializing unit tests. At least some of them it seems are not designed to run in parallel (they very likely share some state), and they were running in parallel (1.5 per CPU core of Jenkins node on which build is running), causing each other to fail randomly. Now it's all sequential. Good thing is, that all Mahout build jobs are passing now. Downside is that build time has increased. It tripled for these two: https://builds.apache.org/job/Mahout-Quality/buildTimeTrend https://builds.apache.org/job/Mahout-Trunk/buildTimeTrend and nightly build took ~7x more time to complete: https://builds.apache.org/job/mahout-nightly/buildTimeTrend Mahout-Examples build jobs build times it seems didn't change. Fast feedback from unit tests IMO is very important, so we should improve the build time, better sooner than later, through more build script configuration, but also designing unit tests for parallel execution in mind. Kind regards, Stevo Slavić. On Sat, Jul 6, 2013 at 2:37 AM, Suneel Marthi <[email protected]>wrote: > Thanks Stevo. This could be a dumb question given my complete lack of > understanding of all things Maven - so what was it that was causing random > test failures? > > > > > ________________________________ > From: Ted Dunning <[email protected]> > To: Mahout Dev List <[email protected]> > Sent: Friday, July 5, 2013 8:22 PM > Subject: Re: Jenkins build is back to normal : Mahout-Quality #2128 > > > Thank you Stevo! > > > On Fri, Jul 5, 2013 at 4:45 PM, Apache Jenkins Server < > [email protected]> wrote: > > > See <https://builds.apache.org/job/Mahout-Quality/2128/changes> > > > > >
