I've got a Jenkins job 'ivija-my360' that shows builds #1-#19 in the history sidebar, builds a build #20 every time I try to build it, and then forgets about it immediately.
It also says "last stable build" is #24. $ ls -l /var/lib/jenkins/jobs/ivija-my360/ total 28 drwxr-xr-x 26 jenkins jenkins 4096 Sep 16 07:23 builds -rw-r--r-- 1 jenkins jenkins 2412 Sep 15 22:38 config.xml -rw-r--r-- 1 jenkins jenkins 5130 Sep 16 07:23 disk-usage.xml lrwxrwxrwx 1 jenkins jenkins 22 Sep 15 23:08 lastStable -> builds/lastStableBuild lrwxrwxrwx 1 jenkins jenkins 26 Sep 15 23:08 lastSuccessful -> builds/lastSuccessfulBuild -rw-r--r-- 1 jenkins jenkins 3 Sep 15 23:08 nextBuildNumber drwxr-xr-x 4 jenkins jenkins 4096 Sep 15 19:49 outOfOrderBuilds -rw-r--r-- 1 jenkins jenkins 244 Sep 16 07:36 scm-polling.log $ cat /var/lib/jenkins/jobs/ivija-my360/nextBuildNumber 21 $ ls -l /var/lib/jenkins/jobs/ivija-my360/builds/ total 96 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 1 -> 2013-02-22_16-31-37 lrwxrwxrwx 1 jenkins jenkins 19 Feb 28 2013 10 -> 2013-02-28_08-21-38 lrwxrwxrwx 1 jenkins jenkins 19 Mar 20 2013 11 -> 2013-03-20_07-04-11 lrwxrwxrwx 1 jenkins jenkins 19 Apr 9 2013 12 -> 2013-04-09_14-22-59 lrwxrwxrwx 1 jenkins jenkins 19 Apr 9 2013 13 -> 2013-04-09_17-14-00 lrwxrwxrwx 1 jenkins jenkins 19 Apr 9 2013 14 -> 2013-04-09_19-02-26 lrwxrwxrwx 1 jenkins jenkins 19 Apr 9 2013 15 -> 2013-04-09_20-46-00 lrwxrwxrwx 1 jenkins jenkins 19 Apr 16 2013 16 -> 2013-04-16_15-40-00 lrwxrwxrwx 1 jenkins jenkins 19 Apr 19 2013 17 -> 2013-04-19_09-24-25 lrwxrwxrwx 1 jenkins jenkins 19 Sep 15 19:59 19 -> 2014-09-15_19-59-26 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 2 -> 2013-02-22_16-31-55 lrwxrwxrwx 1 jenkins jenkins 19 Sep 15 23:08 20 -> 2014-09-15_23-08-09 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_16-31-37 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_16-31-55 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_16-32-55 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_16-33-55 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_18-56-57 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_19-30-01 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_20-22-16 drwxr-xr-x 2 jenkins jenkins 4096 Feb 22 2013 2013-02-22_20-56-34 drwxr-xr-x 2 jenkins jenkins 4096 Feb 27 2013 2013-02-22_21-04-49 drwxr-xr-x 2 jenkins jenkins 4096 Mar 12 2013 2013-02-28_08-21-38 drwxr-xr-x 2 jenkins jenkins 4096 Apr 5 2013 2013-03-20_07-04-11 drwxr-xr-x 2 jenkins jenkins 4096 Apr 9 2013 2013-04-09_14-22-59 drwxr-xr-x 3 jenkins jenkins 4096 Apr 9 2013 2013-04-09_17-14-00 drwxr-xr-x 3 jenkins jenkins 4096 Apr 9 2013 2013-04-09_19-02-26 drwxr-xr-x 3 jenkins jenkins 4096 Apr 15 2013 2013-04-09_20-46-00 drwxr-xr-x 3 jenkins jenkins 4096 Apr 16 2013 2013-04-16_15-40-00 drwxr-xr-x 3 jenkins jenkins 4096 May 29 2013 2013-04-19_09-24-25 drwxr-xr-x 3 jenkins jenkins 4096 Jul 15 2013 2013-06-27_06-10-43 drwxr-xr-x 3 jenkins jenkins 4096 Jul 16 2013 2013-07-16_13-56-38 drwxr-xr-x 2 jenkins jenkins 4096 Jul 16 2013 2013-07-16_15-33-53 drwxr-xr-x 2 jenkins jenkins 4096 Jul 16 2013 2013-07-16_17-47-12 drwxr-xr-x 2 jenkins jenkins 4096 Mar 16 2014 2013-07-17_15-40-49 drwxr-xr-x 2 jenkins jenkins 4096 Sep 16 00:02 2014-09-15_19-59-26 drwxr-xr-x 2 jenkins jenkins 4096 Sep 16 07:23 2014-09-15_23-08-09 lrwxrwxrwx 1 jenkins jenkins 19 Jul 16 2013 21 -> 2013-07-16_15-33-53 lrwxrwxrwx 1 jenkins jenkins 19 Jul 16 2013 22 -> 2013-07-16_17-47-12 lrwxrwxrwx 1 jenkins jenkins 19 Jul 17 2013 24 -> 2013-07-17_15-40-49 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 3 -> 2013-02-22_16-32-55 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 4 -> 2013-02-22_16-33-55 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 5 -> 2013-02-22_18-56-57 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 6 -> 2013-02-22_19-30-01 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 7 -> 2013-02-22_20-22-16 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 8 -> 2013-02-22_20-56-34 lrwxrwxrwx 1 jenkins jenkins 19 Feb 22 2013 9 -> 2013-02-22_21-04-49 lrwxrwxrwx 1 jenkins jenkins 2 Sep 16 01:06 lastFailedBuild -> 19 lrwxrwxrwx 1 jenkins jenkins 2 Jul 17 2013 lastStableBuild -> 24 lrwxrwxrwx 1 jenkins jenkins 2 Jul 17 2013 lastSuccessfulBuild -> 24 lrwxrwxrwx 1 jenkins jenkins 2 Dec 11 2013 lastUnstableBuild -> -1 lrwxrwxrwx 1 jenkins jenkins 2 Sep 16 07:23 lastUnsuccessfulBuild -> 20 $ ls -l /var/lib/jenkins/jobs/ivija-my360/outOfOrderBuilds/ drwxr-xr-x 3 jenkins jenkins 4096 May 31 2013 2013-05-29_12-19-08 drwxr-xr-x 3 jenkins jenkins 4096 Jul 17 2013 2013-07-17_14-15-20 I have seen the "some builds are out of order, do you want to sweep them under the rug" warning in Manage Jenkins once or twice, and pushed the button. This warning doesn't show up now. I'm running Jenkins 1.580. This is similar to https://issues.jenkins-ci.org/browse/JENKINS-15156. What's different from the comments is that restarting Jenkins doesn't bring the missing builds back. I'm also pretty sure my on-disk data structures are messed up and I should fix them first -- or maybe the out-or-order build monitor thingy should be fixed to be able to do that, correctly? I found https://github.com/docwhat/jenkins-job-checker in the comments of that bug and ran it against my job directory. Its output: **** PROBLEMS **** ivija-my360: Problem: ORDER: The link builds/19 -> 2014-09-15_19-59-26 is out of order. Problem: ORDER: The link builds/20 -> 2014-09-15_23-08-09 is out of order. Problem: STOLEN: The date build builds/2013-06-27_06-10-43 had its number stolen by builds/19 -> 2014-09-15_19-59-26 Problem: STOLEN: The date build builds/2013-07-16_13-56-38 had its number stolen by builds/20 -> 2014-09-15_23-08-09 Problem: NEXT: The nextBuildNumber is set to 21 but I expected at least 25 Proposal: Archive out-of-order builds/19 -> 2014-09-15_19-59-26 Proposal: Archive out-of-order builds/20 -> 2014-09-15_23-08-09 Proposal: Relink 19 to builds/2013-06-27_06-10-43 Proposal: Archive newer build builds/2014-09-15_19-59-26 Proposal: Relink 20 to builds/2013-07-16_13-56-38 Proposal: Archive newer build builds/2014-09-15_23-08-09 Proposal: Reset nextBuildNumber The proposals seemed sane, so I ran jobber.rb --solve, stumbled upon some bug[*], reran it and now the build directory seems fine. [*] https://github.com/docwhat/jenkins-job-checker/issues/3 Trying to schedule a new build, of course, scheduled build #21 while completely ignoring the fact build #21 was now showing up in history. Asking Jenkins to reload configuration to disk fixed that (I canceled the scheduled build and scheduled a new one, which got number #25). I'll see whether the fix works once my Jenkins gets through the build queue, which will take a few hours (I'm moving to new slaves in fresh VMs and am rebuilding EVERYTHING to make sure they're set up correctly). It seems that OutOfOrderBuildMonitor ought to be a bit more stringent in its checks, especially with the incorrect nextBuildNumber setting. Marius Gedminas -- UNIX is user friendly. It's just selective about who its friends are.
signature.asc
Description: Digital signature
