About Jenkins:
Will it be good to create separate maven profiles executed using
separate Jenkins jobs for
a) normal build without findbugs/checkstyle/pmd etc
b) quality build with findbugs/checkstyle/pmd
So, the normal build will not fail due to formatting/findbugs etc issues
and quality build will keep a check on formatting etc. This will also
make the normal build very fast.
I can take up this task it if people agree.
About commits over formatting/static/final minor issues etc:
Since you are planning to do it for next few weeks, so, it might help in
reducing chaos if you pick up (some number of)packages/week or so and
inform others in advance( some info like package names and end time of
code change ).
Paritosh
On 07-03-2012 09:10, tom pierce wrote:
Thanks for the tips; maybe we should have a precommit target that
would run tests and checkstyle/pmd/findbugs and fail if counts go up?
Seems simpler than trying to synchronize IntelliJ/Eclipse/Emacs/VI
style rules.
I guess what I was really trying to ask was whether there was a
general commitment within the project to using Jenkins with the
current style/checkbugs/etc rules.
If there is, and folks will address upticks in error/warning counts as
they occur, it is worth investing a bit of time to get the build back
to stable. If not, we should either fix the Jenkins config to reflect
what we're willing to maintain, or spare the ASF resources and disable
it.
I guess I'm willing to play janitor once - does anyone object to some
omnibus commits over the next few weeks that deal with formatting and
other minor final/static/etc. issues? This will inevitably screw up
patches sitting around in JIRA, local changes, diffs against svn
history, etc. in the usual small, but annoying, ways. I'm willing to
fence off specific files and/or modules, if requested.
Fair warning: I'll treat silence as consent!
-tom
On 03/05/2012 08:41 PM, Dmitriy Lyubimov wrote:
so bottom line, i just do autoformat and then post the patch to review
board and keep adjusting it manually until there are no "reds" there
as well.
Also at some point i had a script that ran checkstyle and grep'd
warnings produced for files that are different from trunk (i.e.
current patch only) but it stopped working at some point for some
reason, perhaps something was done to maven config or something.
Bottom line, if you find a good process that works for you to squish
these little critters, please don't hesitate to share. I ended up only
50% there although i spent some time on trying to find an optimal
process with eclipse since this project seems to be sensitive about
these things.
On Mon, Mar 5, 2012 at 5:34 PM, Dmitriy Lyubimov<dlie...@gmail.com>
wrote:
i've been using an eclipse style format settings which i tuned as
close as i could to match intelliJ's that Sean patches produce.
Some of those formattings (mostly wrapping preference rules) don't
trigger checkstyle problems regardless of which tool is used but they
do trigger reformatting if i reformat after somebody who formatted
using Intellij and vice versa. Which really makes things difficult
from the point of view of auto formatting.
Some of the checkstyle problems inherently missed by my eclipse
autoformatting. (such as trailing spaces in comments). But I suspect
Intellij is probably not doing 100% either.
Bottom line i haven't figured a really good way to address all
formatting rule checks that Mahout's checkstyle settings employ. But i
have a first approximation for eclipse styles config which i can
share.
Grant also was suggesting a codestyle for eclipse from Lucene which is
probably not significantly better in that regard either.
If you like intelliJ, you may bug Sean for his styles on that side of
things. But if you run eclipse i'd really like to use one shared style
so autoformat doesn't act differently on something that has already
been formatted.
-d
On Mon, Mar 5, 2012 at 4:51 PM, tom pierce<t...@apache.org> wrote:
I am starting to agree that Jenkins is calling the build "unstable"
on style
grounds, but my evidence is shaky: the style plot has a "caution
triangle"
and the checkstyle count has a storm cloud.
I have made a couple of commits to clear up some style warnings
(which were
mostly of my own doing), and I was thinking about maybe trying to
knock off
a few small commits like this 1-2x/week.
Before I start playing code-janitor I want to make sure I'm not
going to be
shoving against the tide. Are the style rules we're checking with
these
tools an accurate representation of the project standards? Are we
willing
to do things like set up a JIRA trigger to test patches (I think
Hadoop does
this), and generally pay attention to Jenkins failures? Is there a
maven
target that runs these checks?
-tom
On 03/05/2012 02:50 PM, Dmitriy Lyubimov wrote:
For some reason i concluded it was related to big number PMD/style
warnings. but i may be wrong and i don't remember why i concluded
that. I certainly know the least about Jenkins and ci stuff.
On Mon, Mar 5, 2012 at 11:40 AM, Tom Pierce<t...@cloudera.com>
wrote:
Hi folks,
I spent a little time looking into our Jenkins failures. I am not
very familiar with Jenkins, so this is probably a little remedial -
pointers for other/better things to look at are appreciated!
We've let this go for a long time:
* Last stable build (#1218), 3 mo 3 days ago
Before that, it looks like we were having intermittent failures
(failures at 1196, 1208, 1217 and then 1219-).
Our "workspace" disk usage spiked sometime around build 1370 - going
up over 800MB. Prior to this spike, workspace disk usage was
flat-lined at or very near 0 - going back to build 1218. Since the
spike, it looks like we've been oscillating between 800+0MB
workspace
usage. This is likely due to failures occurring at different points
(before or after scratch space is used?).
Between the 1218/1219 Mahout Jenkins reports, we got a report about
Mahout-Examples (#37) failing, with a shell trace, including this
evidence of a misconfiguration on the test box:
[trunk] $ /home/hudson/tools/maven/apache-maven-2.2.1/bin/mvn
-DskipTests=true -U clean install
Error: JAVA_HOME is not defined correctly.
We cannot execute /home/hudson/tools/java/latest1.6/bin/java
Build step 'Invoke top-level Maven targets' marked build as failure
The last-good and first-bad builds are too old to have a full
Jenkins
report, but I think these are the intervening commits (between 1218
and 1219):
http://svn.apache.org/viewvc?rev=1209476&view=rev
http://svn.apache.org/viewvc?rev=1209480&view=rev
http://svn.apache.org/viewvc?rev=1209577&view=rev
http://svn.apache.org/viewvc?rev=1209794&view=rev
Those are pretty small changes except for the last - which added
a new
LDA implementation (MAHOUT-897).
Looking through a full batch of Jenkins console output (30MB!)
did not
let me pin down the reason Jenkins deems the build unstable (and
certainly nothing that indicates the above changes are to blame). I
see lots of tests run (a few skipped, but no errors/failures),
lots of
checkstyle/findbugs problems, but nothing that screams "failure
here".
It does seem that the cause is related to the
checkstyle/pmd/findbugs
results, due to this line toward the end:
Build step 'Report Violations' changed build result to UNSTABLE
Does anyone know how to trace this back to a specific
condition/fault?
-tom
