For some reason i concluded it was related to big number PMD/style warnings. but i may be wrong and i don't remember why i concluded that. I certainly know the least about Jenkins and ci stuff.
On Mon, Mar 5, 2012 at 11:40 AM, Tom Pierce <t...@cloudera.com> wrote: > Hi folks, > > I spent a little time looking into our Jenkins failures. I am not > very familiar with Jenkins, so this is probably a little remedial - > pointers for other/better things to look at are appreciated! > > We've let this go for a long time: > * Last stable build (#1218), 3 mo 3 days ago > > Before that, it looks like we were having intermittent failures > (failures at 1196, 1208, 1217 and then 1219-). > > Our "workspace" disk usage spiked sometime around build 1370 - going > up over 800MB. Prior to this spike, workspace disk usage was > flat-lined at or very near 0 - going back to build 1218. Since the > spike, it looks like we've been oscillating between 800+0MB workspace > usage. This is likely due to failures occurring at different points > (before or after scratch space is used?). > > Between the 1218/1219 Mahout Jenkins reports, we got a report about > Mahout-Examples (#37) failing, with a shell trace, including this > evidence of a misconfiguration on the test box: > > [trunk] $ /home/hudson/tools/maven/apache-maven-2.2.1/bin/mvn > -DskipTests=true -U clean install > Error: JAVA_HOME is not defined correctly. > We cannot execute /home/hudson/tools/java/latest1.6/bin/java > Build step 'Invoke top-level Maven targets' marked build as failure > > The last-good and first-bad builds are too old to have a full Jenkins > report, but I think these are the intervening commits (between 1218 > and 1219): > > http://svn.apache.org/viewvc?rev=1209476&view=rev > http://svn.apache.org/viewvc?rev=1209480&view=rev > http://svn.apache.org/viewvc?rev=1209577&view=rev > http://svn.apache.org/viewvc?rev=1209794&view=rev > > Those are pretty small changes except for the last - which added a new > LDA implementation (MAHOUT-897). > > Looking through a full batch of Jenkins console output (30MB!) did not > let me pin down the reason Jenkins deems the build unstable (and > certainly nothing that indicates the above changes are to blame). I > see lots of tests run (a few skipped, but no errors/failures), lots of > checkstyle/findbugs problems, but nothing that screams "failure here". > It does seem that the cause is related to the checkstyle/pmd/findbugs > results, due to this line toward the end: > > Build step 'Report Violations' changed build result to UNSTABLE > > Does anyone know how to trace this back to a specific condition/fault? > > -tom
