[JIRA] (JENKINS-61687) Exceptions due to race condition during build deletion
Title: Message Title Kalle Niemitalo commented on JENKINS-61687 Re: Exceptions due to race condition during build deletion Would it be reasonable to specifically catch and swallow java.nio.file.NoSuchFileException from the Files.move call? Or would that just surface more concurrency problems in other parts of hudson.model.Run.delete? Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.205510.1585173096000.22508.1588761180537%40Atlassian.JIRA.
[JIRA] (JENKINS-61687) Exceptions due to race condition during build deletion
Title: Message Title Daniel Beck commented on JENKINS-61687 Re: Exceptions due to race condition during build deletion (I hypothesize that each build runs Project Build Discarder twice in parallel and those race against each other I don't remember how far along I was in the investigation when I posted the comment, but IIRC this is correct. Specifically, the "background" build discarders are also run after a build finishes, in addition to the regular build discarder. The idea being, why should it only run periodically, when it could as well run when a build is finished, and not accumulate a lot of builds for its periodic run? https://github.com/jenkinsci/jenkins/blob/da90af311587f6c3d37ec4e9c4b4637763924743/core/src/main/java/jenkins/model/GlobalBuildDiscarderListener.java#L46-L50 Now, if you configure the "job specific build discarder" as part of global/background build discarders (the default), there'll be two build discarder instantiations running after a build is finished, both configured to delete the exact same builds. So this configuration basically creates the circumstances under which the (AFAICT) long-standing concurrency bug occurs. (Strictly speaking, any global build discarder could trigger this, but if the configurations are different, it's likely that only one build discarder will ever find builds to delete.) A workaround for this specific case only could be to skip the "project specific build discarder" when we're triggered by a build finishing (i.e. only run it when triggered by the periodic run), but that seems like a hack, but doesn't consider fixed global build discarders that match the job configuration exactly. A more reasonable workaround, don't run multiple build discarders on a job in parallel. There's currently no synchronization there. Ideally however, we'd fix the concurrency issues in build deletion, then neither builds in quick succession, nor background build discarders will be a problem. Add Comment
[JIRA] (JENKINS-61687) Exceptions due to race condition during build deletion
Title: Message Title Kalle Niemitalo commented on JENKINS-61687 Re: Exceptions due to race condition during build deletion The problem occurs when many builds finish around the same time and is unrelated to the global build discarder. No, we got "An exception occurred when executing Project Build Discarder" from three consecutive builds even though there were several minutes of idle time between them. Build Start Get node Finish Build Discarder JobA #89 08:38:24 08:38:24 08:44:29 08:44:29 NoSuchFileException deleting JobA #39 JobB #579 08:49:18 08:49:19 08:53:32 08:53:53 NoSuchFileException deleting JobB #529 JobA #90 10:18:40 10:18:42 10:24:18 10:24:18 NoSuchFileException deleting JobA #40 JobB #580 10:19:50 10:24:17 10:28:53 No error
[JIRA] (JENKINS-61687) Exceptions due to race condition during build deletion
Title: Message Title Daniel Beck updated an issue Jenkins / JENKINS-61687 Exceptions due to race condition during build deletion Change By: Daniel Beck Summary: Exceptions due to race condition during build deeltion deletion Add Comment This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.205510.1585173096000.13650.1587238980623%40Atlassian.JIRA.
