Hi,
To get an overview on what is going on with our Jenkins instances, what
value they provide and how much effort they generate, I broke down the
issues reported by them along various axis.
There where 327 issues reported between 8.12.16 and 28.2.17. With 82
days this amounts to almost 4 issues a day. Note that this number is
quite biased as that time period includes the Christmas break where we
didn't had much activity. The correct numbers are probably closer to 72
days and and 4.5 issues per day.
To me the most striking thing in below breakdowns are the high number of
duplicates (256 / 78%) and the high number of infrastructure relates
issues (84 / 26%). To me this means we are spending too much time in
triaging issues and hunting down infrastructure problems.
From the total of 25 fixed issues only 4 where actual regressions. Two
of which were caused by missing licenses headers, a problem that our
release process also would have caught.
Finally all numbers are further biased because the Jenkins Jira
notification plugin itself fails sometimes [1] (frequently?), which
causes build failures not to be reported.
Michael
Issues by resolution:
256 Duplicates (172 test failures / 84 infra issues)
27 Unresolved ( 21 test failures / 6 infra issues)
25 Fixed
15 CI and infra issue
4 Rare test artefacts
Infra issues (84):
32 Backing channel disconnected
20 JVM crash
12 File name too long
6 Failed silently
4 Artifact resolution error
4 Maven failure
3 Timeout (120 min.)
2 Disk full
1 Checksum mismatch
Fixed issues (25):
4 bug / regression (OAK-5339, OAK-5540, OAK-5241, OAK-5471)
7 timing
14 test artefact
[1]
ERROR: Build step failed with exception
java.lang.NullPointerException
at
hudson.plugins.jira.JiraCreateIssueNotifier.getStatus(JiraCreateIssueNotifier.java:218)
at
hudson.plugins.jira.JiraCreateIssueNotifier.currentBuildResultSuccess(JiraCreateIssueNotifier.java:387)
at
hudson.plugins.jira.JiraCreateIssueNotifier.perform(JiraCreateIssueNotifier.java:159)
at
hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
at hudson.model.Build$BuildExecution.post2(Build.java:185)
at
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
at hudson.model.Run.execute(Run.java:1753)
at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
at
hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Build step 'JIRA: Create issue' marked build as failure
Finished: FAILURE
