Issue Type: Bug Bug
Affects Versions: current
Assignee: abayer
Components: throttle-concurrents
Created: 17/Dec/13 2:34 PM
Description:

Jenkins had stopped responding to browser requests for Jenkins pages and I think it may be caused by the recent upgrade to Throttle Concurrent Builds 1.8.1

Requests were getting blocked waiting on 0x00000004181e4520

"Handling GET /jenkins/ : RequestHandlerThread[#171]" daemon prio=10 tid=0x00000000168ee800 nid=0x193b waiting for monitor entry [0x000000004335b000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at hudson.model.Queue.getItems(Queue.java:687)
	- waiting to lock <0x00000004181e4520> (a hudson.model.Queue)
	at hudson.model.Queue$CachedItemList.get(Queue.java:216)
	at hudson.model.Queue.getApproximateItemsQuickly(Queue.java:717)
	at hudson.model.View.getApproximateQueueItemsQuickly(View.java:483)
	at sun.reflect.GeneratedMethodAccessor355.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

This seems to be waiting on Throttle Concurrent Builds code.
Further dumps after 10 minutes, 20 minutes and 30 minutes showed this same stack trace.

"Thread-126" daemon prio=10 tid=0x00002aaae0529800 nid=0x1785 runnable [0x0000000046590000]
   java.lang.Thread.State: RUNNABLE
	at java.util.WeakHashMap$HashIterator.hasNext(WeakHashMap.java:875)
	at java.util.AbstractCollection.toArray(AbstractCollection.java:139)
	at java.util.ArrayList.<init>(ArrayList.java:164)
	at hudson.plugins.throttleconcurrents.ThrottleJobProperty.getCategoryProjects(ThrottleJobProperty.java:141)
	- locked <0x000000041a79b778> (a java.util.HashMap)
	at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:118)
	at hudson.plugins.throttleconcurrents.ThrottleQueueTaskDispatcher.canRun(ThrottleQueueTaskDispatcher.java:90)
	at hudson.model.Queue.isBuildBlocked(Queue.java:937)
	at hudson.model.Queue.maintain(Queue.java:1006)
	- locked <0x00000004181e4520> (a hudson.model.Queue)
	at hudson.model.Queue$1.call(Queue.java:303)
	at hudson.model.Queue$1.call(Queue.java:300)
	at jenkins.util.AtmostOneTaskExecutor$1.call(AtmostOneTaskExecutor.java:69)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:104)
	at java.lang.Thread.run(Thread.java:724)

   Locked ownable synchronizers:
	- None

CPU usage was at ~100% for this thread for the 30 minutes that I was watching it before I restarted Jenkins.
(6021 = 0x1785)

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                                   
 6021 rcbuild_  35  10 18.0g 6.2g  32m R 99.7 19.7  30:45.97 java

I have rolled back to Throttle Concurrent Builds 1.8 for now.

Still learning how to investigate thread dumps but please let me know if there is anything I can do to help.

Environment: Jenkins 1.544
Project: Jenkins
Priority: Critical Critical
Reporter: Geoff Cummings
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira

--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.

Reply via email to