Hi,
The new Jenkins Job I configured [1] doesn't seem to send emails nor create issues when it fails. E.g. see build 78 [2]. Could someone else have a quick look at the configuration of that job as I cannot figure out what is wrong with it.
[1] https://builds.apache.org/view/J/job/Jackrabbit%20Oak/ [2] https://builds.apache.org/view/J/job/Jackrabbit%20Oak/78/ On 21.03.17 09:06, Michael Dürig wrote:
Hi, As a first reaction to this and to increase our benefit from Jenkins I disabled email notifications and Jira issue reporting for our Jenkins Matrix jobs [1, 2]. The jobs are still there and I suggest everyone has a look at them once in a while. At the same time I set up a new Jenkins jobs, which is much lighter as it only runs the unit tests on trunk [3]. The job is triggered at every commit and usually completes after about 25 minutes. Currently the job sends a notification to @oak-dev should it fail and I might experiment with adding Jira issue reporting to it (but might hit INFRA-13599 [4]). So far this jobs has proofed very stable (no failures in the past 20 builds). The stability together with the quick turnaround should give us fast feedback on regressions. Any failure reported by this job is thus a signal for immediate action. Michael [1] https://builds.apache.org/view/All/job/Apache%20Jackrabbit%20Oak%20matrix/ [2] https://builds.apache.org/view/All/job/Oak-Win/ [3] https://builds.apache.org/view/J/job/Jackrabbit%20Oak/ [4] https://issues.apache.org/jira/browse/INFRA-13599 On 28.02.17 12:31, Michael Dürig wrote:Hi, To get an overview on what is going on with our Jenkins instances, what value they provide and how much effort they generate, I broke down the issues reported by them along various axis. There where 327 issues reported between 8.12.16 and 28.2.17. With 82 days this amounts to almost 4 issues a day. Note that this number is quite biased as that time period includes the Christmas break where we didn't had much activity. The correct numbers are probably closer to 72 days and and 4.5 issues per day. To me the most striking thing in below breakdowns are the high number of duplicates (256 / 78%) and the high number of infrastructure relates issues (84 / 26%). To me this means we are spending too much time in triaging issues and hunting down infrastructure problems. From the total of 25 fixed issues only 4 where actual regressions. Two of which were caused by missing licenses headers, a problem that our release process also would have caught. Finally all numbers are further biased because the Jenkins Jira notification plugin itself fails sometimes [1] (frequently?), which causes build failures not to be reported. Michael Issues by resolution: 256 Duplicates (172 test failures / 84 infra issues) 27 Unresolved ( 21 test failures / 6 infra issues) 25 Fixed 15 CI and infra issue 4 Rare test artefacts Infra issues (84): 32 Backing channel disconnected 20 JVM crash 12 File name too long 6 Failed silently 4 Artifact resolution error 4 Maven failure 3 Timeout (120 min.) 2 Disk full 1 Checksum mismatch Fixed issues (25): 4 bug / regression (OAK-5339, OAK-5540, OAK-5241, OAK-5471) 7 timing 14 test artefact [1] ERROR: Build step failed with exception java.lang.NullPointerException at hudson.plugins.jira.JiraCreateIssueNotifier.getStatus(JiraCreateIssueNotifier.java:218) at hudson.plugins.jira.JiraCreateIssueNotifier.currentBuildResultSuccess(JiraCreateIssueNotifier.java:387) at hudson.plugins.jira.JiraCreateIssueNotifier.perform(JiraCreateIssueNotifier.java:159) at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720) at hudson.model.Build$BuildExecution.post2(Build.java:185) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665) at hudson.model.Run.execute(Run.java:1753) at hudson.matrix.MatrixRun.run(MatrixRun.java:146) at hudson.model.ResourceController.execute(ResourceController.java:98) at hudson.model.Executor.run(Executor.java:404) Build step 'JIRA: Create issue' marked build as failure Finished: FAILURE
