[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Title: Message Title
Sébastien Boulet updated an issue
Jenkins / JENKINS-21034
Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Change By:
Sébastien Boulet
{code}Found one Java-level deadlock:="Handling GET /jenkins/view/Portal/ : RequestHandlerThread[#12]": waiting to lock monitor 0x1a449d08 (object 0x000419d757e8, a hudson.ExtensionList$Lock), which is held by "QueueSorter.installDefaultQueueSorter""QueueSorter.installDefaultQueueSorter": waiting to lock monitor 0x1ae56c48 (object 0x00041895e0f0, a hudson.model.Queue), which is held by "Queue.init""Queue.init": waiting to lock monitor 0x1a449d08 (object 0x000419d757e8, a hudson.ExtensionList$Lock), which is held by "QueueSorter.installDefaultQueueSorter"{code}
Add Comment
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Magnus Sandberg resolved JENKINS-21034 as Fixed Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Hopefully fixed in 2.3 The startup sequence is different from 2.2 so if this still occurs please provide new log-file. Change By: Magnus Sandberg (20/Dec/13 10:59 PM) Status: Open Resolved Resolution: Fixed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Magnus Sandberg
updated JENKINS-21034
Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Change By:
Magnus Sandberg
(20/Dec/13 2:49 PM)
Environment:
Jenkins 1.544
/ Jenkins Priority Sorter Plugin 2.2
Description:
{code}Found one Java-level deadlock:="Handling GET /jenkins/view/Portal/ : RequestHandlerThread[#12]": waiting to lock monitor 0x1a449d08 (object 0x000419d757e8, a hudson.ExtensionList$Lock), which is held by "QueueSorter.installDefaultQueueSorter""QueueSorter.installDefaultQueueSorter": waiting to lock monitor 0x1ae56c48 (object 0x00041895e0f0, a hudson.model.Queue), which is held by "Queue.init""Queue.init": waiting to lock monitor 0x1a449d08 (object 0x000419d757e8, a hudson.ExtensionList$Lock), which is held by "QueueSorter.installDefaultQueueSorter"{code}
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Magnus Sandberg commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init I've added some test cases in the 2.3 pull - testing the queue behaviour is not that easy though. Regarding the order I'm not sure - one option could be to use before=COMPLETED rather than after=JOB_LOADED but I'm not sure. I have restarted Jenkins with items in the queue like a gazillion times and I have never seen this so it is a bit tricky for me to to verify this. I reckon that it cannot be wore than it is so I'm considering releasing 2.3 with the current 2.3 pull (https://github.com/jenkinsci/priority-sorter-plugin/pull/10) This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Oleg Nenashev commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Most probably, it will help with the issue. BTW, any test cases will be appreciated Is it guaranteed that initializer's method call will be finished before starting of scheduling? If no, it makes sense to somehow synchronize methods. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Magnus Sandberg commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Yes this is actually already fixed for the 2.3 where the init is done with Initializer (after.JOB_LOADED)(I though that was already in 2.2 but is is not) https://github.com/emsa23/priority-sorter-plugin/commit/a2447a7e371869cb4ada8ed76a7114b3eb165ee8 @Oleg do you think that fix will do it? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Oleg Nenashev commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init In my opinion, the code needs some correction even if it works properly in previous Jenkins versions. Queue.init() initializes QueueSorter extensions, so it is quite strange to retrieve data from it at this stage... I see the following fix: Remove queue handling logic from AdvancedQueueSorter constructor Move it to a separate method an call on the first sortBuildableItems() call It also makes sense to call this method for new mode only (just don't do anything for legacy) This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Rob van Oostrum commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init @Magnus: I've had this happen after plugin upgrades as well. Yesterday I restarted after upgrading the JDK on Jenkins' server and it deadlocked. Once it deadlocks, restarting again just results in another deadlock. see previous answer I believe my queue had only 2 items in it when I restarted yesterday. It may have been a couple more than that, but not a huge number by any stretch. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Magnus Sandberg commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Does this happen on a "normal" restart or only directly after the upgrade? Does this only happen after a upgrade of Priority Sorter or in other situations as well? Did you have a huge queue at restart or does this happen with any queue size? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Magnus Sandberg assigned JENKINS-21034 to Magnus Sandberg Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Change By: Magnus Sandberg (19/Dec/13 12:11 PM) Assignee: bklarson Magnus Sandberg This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Rob van Oostrum commented on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Ran into this as well. Seems unrelated to the upgrade. It's simply that the queue that's persisted when the Jenkins service shuts down causes a deadlock when Jenkins re-initializes. The quick fix for me was to rename the file (this is on Ubuntu) from queue.xml into something like queue.keep (or you could just remove it altogether) This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Rob van Oostrum edited a comment on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Ran into this as well. Seems unrelated to the upgrade. It's simply that the queue that's persisted when the Jenkins service shuts down causes a deadlock when Jenkins re-initializes. The quick fix for me was to rename the file (this is on Ubuntu) from queue.xml into something like queue.xml.keep (or you could just remove it altogether). I've had this happen both at Jenkins upgrades, and simply restarting Jenkins after plugin updates, since 1.542 or so. Once in this deadlock, downgrading as far as 1.540 didn't do the trick. Ended up upgrading back to 1.544 and removing queue.xml before restarting. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Rob van Oostrum edited a comment on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Ran into this as well. Seems unrelated to the upgrade. It's simply that the queue that's persisted when the Jenkins service shuts down causes a deadlock when Jenkins re-initializes. The quick fix for me was to rename the file (this is on Ubuntu) from queue.xml into something like queue.keep (or you could just remove it altogether). I've had this happen both at Jenkins upgrades, and simply restarting Jenkins after plugin updates, since 1.542 or so. Once in this deadlock, downgrading as far as 1.540 didn't do the trick. Ended up upgrading back to 1.544 and removing queue.xml before restarting. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Rob van Oostrum edited a comment on JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Ran into this as well. Seems unrelated to the upgrade. It's simply that the queue that's persisted when the Jenkins service shuts down causes a deadlock when Jenkins re-initializes. The quick fix for me was to rename the file (this is on Ubuntu) from queue.xml into something like queue.keep (or you could just remove it altogether). I've had this happen both at Jenkins upgrades, and simply restarting Jenkins after plugin updates, since 1.542 or so. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
[JIRA] [core] (JENKINS-21034) Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init
Geoff Cummings created JENKINS-21034 Jenkins Startup Deadlock - QueueSorter.installDefaultQueueSorter and Queue.init Issue Type: Bug Assignee: bklarson Attachments: jenkinsQueueDeadlock.log Components: core, prioritysorter Created: 17/Dec/13 9:13 AM Description: Found one Java-level deadlock: = "Handling GET /jenkins/view/Portal/ : RequestHandlerThread[#12]": waiting to lock monitor 0x1a449d08 (object 0x000419d757e8, a hudson.ExtensionList$Lock), which is held by "QueueSorter.installDefaultQueueSorter" "QueueSorter.installDefaultQueueSorter": waiting to lock monitor 0x1ae56c48 (object 0x00041895e0f0, a hudson.model.Queue), which is held by "Queue.init" "Queue.init": waiting to lock monitor 0x1a449d08 (object 0x000419d757e8, a hudson.ExtensionList$Lock), which is held by "QueueSorter.installDefaultQueueSorter" Environment: Jenkins 1.544 Project: Jenkins Priority: Critical Reporter: Geoff Cummings This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
