[JIRA] [ec2] (JENKINS-22558) Deadlock condition when provisioning nodes and running the retention strategy to remove nodes

[tak...@java.net (JIRA)]

taksan created JENKINS-22558 Deadlock condition when provisioning nodes and running the retention strategy to remove nodes Issue Type: Bug Affects Versions: current Assignee: Francis Upton Components: ec2 Created: 10/Apr/14 2:08 AM Description: There's a racing condition between node provisioning and the retention strategy to remove idle nodes. Using the threadDump, I retrieved the thread conditions when the deadlock happened: Here's the Retention Strategy Timer thread: jenkins.util.Timer 6 "jenkins.util.Timer 6" Id=51 Group=main BLOCKED on hudson.model.Hudson@2fdebb41 owned by "Computer.threadPoolForRemoting 1450" Id=13155 at jenkins.model.Jenkins.removeNode(Jenkins.java:1706) blocked on hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@5e642b0a at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.ComputerRetentionWork.doRun(ComputerRetentionWork.java:66) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Number of locked synchronizers = 1 Here's the Provivisioning thread: Computer.threadPoolForRemoting 1450 "Computer.threadPoolForRemoting 1450" Id=13155 Group=main BLOCKED on hudson.plugins.ec2.EC2SpotRetentionStrategy@5e642b0a owned by "jenkins.util.Timer 6" Id=51 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:65) blocked on hudson.plugins.ec2.EC2SpotRetentionStrategy@5e642b0a at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@208a81be at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@74c59c5f at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@1e4b63aa at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@edff3d5 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@6f859bc2 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@4b22eb23 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@4f008c44 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@252a5c30 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@1c5a1b87 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@360fa9ea at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@241929b at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@27f3ec80 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@1d08a504 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@c7c8313 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@3e3587e8 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@7a3ee5db at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@202ba370 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@2a3129be at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@38738704 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@c4748bb at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@654666f2 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@7d635e40 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@35b25a1e at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@e9d7bb2 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@2712efe8 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@712322b at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@5fbd1834 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@57068640 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@bd12c5d at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.removeNode(Jenkins.java:1712) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2SpotSlave.terminate(EC2SpotSlave.java:82) at hudson.plugins.ec2.EC2AbstractSlave.idleTimeout(EC2AbstractSlave.java:249) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:73) locked hudson.plugins.ec2.EC2SpotRetentionStrategy@34b01559 at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:39) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:663) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:120) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:180) locked java.lang.Object@7f564ca9 at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1219) at jenkins.model.Jenkins.setNodes(Jenkins.java:1717) at jenkins.model.Jenkins.addNode(Jenkins.java:1699) locked hudson.model.Hudson@2fdebb41 at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:360) at hudson.plugins.ec2.EC2Cloud$1.call(EC2Cloud.java:355) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Number of locked synchronizers = 1 java.util.concurrent.ThreadPoolExecutor$Worker@126cd40b Project: Jenkins Priority: Critical Reporter: taksan

This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.