Hi Tim,
On 2015-02-12, 7:24 PM, "Timothy Worman" wrote: All: I have an application that deadlocks every time I try to bounce instances. The previous instance of the app never will completely shutdown. Since this isn’t (yet) my forte, this is what my jstack output looks like: General jstack reading tips welcome as well - gotta do some learnin’. The first thing I do is pop these into a text editor and search and replace on the repeated traced (replace with nothing) so you can get it down to the interesting ones. At least then you can see the trees. Next, "No deadlocks found.” it is not deadlocked. The run loop: Thread 1295: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - com.webobjects.appserver._WORunLoop._acceptInputBeforeDate(com.webobjects.foundation.NSTimestamp) @bci=98, line=244 (Interpreted frame) - com.webobjects.appserver._WORunLoop.runBeforeDate(com.webobjects.foundation.NSTimestamp) @bci=39, line=95 (Interpreted frame) - com.webobjects.appserver.WOApplication._runOnce() @bci=17, line=1095 (Interpreted frame) - com.webobjects.appserver.WOApplication.run() @bci=106, line=1248 (Interpreted frame) - er.extensions.appserver.ERXApplication.run() @bci=292, line=1417 (Interpreted frame) - com.webobjects.appserver.WOApplication.main(java.lang.String[], java.lang.Class) @bci=55, line=548 (Interpreted frame) - er.extensions.appserver.ERXApplication.main(java.lang.String[], java.lang.Class) @bci=6, line=861 (Interpreted frame) - edu.ucla.gseis.ets.Application.main(java.lang.String[]) @bci=3, line=68 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method, java.lang.Object, java.lang.Object[]) @bci=0 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=100, line=62 (Interpreted frame) - sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=6, line=43 (Interpreted frame) - java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) @bci=56, line=483 (Interpreted frame) - com.webobjects._bootstrap.WOBootstrap.main(java.lang.String[]) @bci=118, line=87 (Interpreted frame) Can’t exit as some thread is still running. And this here is the problem: Thread 62723: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.Object.wait() @bci=2, line=502 (Compiled frame) - com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(java.lang.String, com.webobjects.appserver.WORequest) @bci=48, line=191 (Compiled frame) - com.webobjects.appserver.WOApplication.restoreSessionWithID(java.lang.String, com.webobjects.appserver.WOContext) @bci=9, line=1913 (Compiled frame) - er.extensions.appserver.ERXApplication.restoreSessionWithID(java.lang.String, com.webobjects.appserver.WOContext) @bci=134, line=2406 (Compiled frame) - er.extensions.appserver.ERXComponentRequestHandler._dispatchWithPreparedApplication(com.webobjects.appserver.WOApplication, com.webobjects.appserver.WOContext, com.webobjects.foundation.NSDictionary) @bci=55, line=260 (Compiled frame) - er.extensions.appserver.ERXComponentRequestHandler._handleRequest(com.webobjects.appserver.WORequest) @bci=113, line=302 (Compiled frame) - er.extensions.appserver.ERXComponentRequestHandler.handleRequest(com.webobjects.appserver.WORequest) @bci=44, line=378 (Compiled frame) - com.webobjects.appserver.WOApplication.dispatchRequest(com.webobjects.appserver.WORequest) @bci=32, line=1687 (Compiled frame) - er.extensions.appserver.ERXApplication.dispatchRequestImmediately(com.webobjects.appserver.WORequest) @bci=74, line=2112 (Compiled frame) - er.extensions.appserver.ERXApplication.dispatchRequest(com.webobjects.appserver.WORequest) @bci=13, line=2077 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.runOnce() @bci=473, line=144 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=129, line=226 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) It is waiting to check out a session that is already checked out. As there are zero other active threads, the only possible explanation for this is that the session never got checked back in. This is almost certainly because one of our Session method threw an exception. The most likely culprit is sleep(). You should have a try…finally block in this method and call super() in the finally. Exceptions thrown from awake() and terminate() might possibly also cause this, I don’t recall at the moment, but I would use a try…finally block around the call to super() just to be safe. If you are lucky the exception might be in your logs someplace. Chuck ================================== Attaching to process ID 51753, please wait... Debugger attached successfully. Server compiler detected. JVM version is 25.25-b02 Deadlock Detection: No deadlocks found. Thread 71171: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - er.javamail.ERMailSender.run() @bci=26, line=356 (Compiled frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 66307: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 65795: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 65283: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 64771: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 64259: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 63747: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 63235: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 62723: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.Object.wait() @bci=2, line=502 (Compiled frame) - com.webobjects.appserver.WOSessionStore.checkOutSessionWithID(java.lang.String, com.webobjects.appserver.WORequest) @bci=48, line=191 (Compiled frame) - com.webobjects.appserver.WOApplication.restoreSessionWithID(java.lang.String, com.webobjects.appserver.WOContext) @bci=9, line=1913 (Compiled frame) - er.extensions.appserver.ERXApplication.restoreSessionWithID(java.lang.String, com.webobjects.appserver.WOContext) @bci=134, line=2406 (Compiled frame) - er.extensions.appserver.ERXComponentRequestHandler._dispatchWithPreparedApplication(com.webobjects.appserver.WOApplication, com.webobjects.appserver.WOContext, com.webobjects.foundation.NSDictionary) @bci=55, line=260 (Compiled frame) - er.extensions.appserver.ERXComponentRequestHandler._handleRequest(com.webobjects.appserver.WORequest) @bci=113, line=302 (Compiled frame) - er.extensions.appserver.ERXComponentRequestHandler.handleRequest(com.webobjects.appserver.WORequest) @bci=44, line=378 (Compiled frame) - com.webobjects.appserver.WOApplication.dispatchRequest(com.webobjects.appserver.WORequest) @bci=32, line=1687 (Compiled frame) - er.extensions.appserver.ERXApplication.dispatchRequestImmediately(com.webobjects.appserver.WORequest) @bci=74, line=2112 (Compiled frame) - er.extensions.appserver.ERXApplication.dispatchRequest(com.webobjects.appserver.WORequest) @bci=13, line=2077 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.runOnce() @bci=473, line=144 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=129, line=226 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 62211: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 61699: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 61187: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 60675: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 60163: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 59651: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 59139: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 58627: (state = IN_NATIVE) - java.net.PlainSocketImpl.socketAccept(java.net.SocketImpl) @bci=0 (Compiled frame; information may be imprecise) - java.net.AbstractPlainSocketImpl.accept(java.net.SocketImpl) @bci=7, line=404 (Compiled frame) - java.net.ServerSocket.implAccept(java.net.Socket) @bci=60, line=545 (Compiled frame) - java.net.ServerSocket.accept() @bci=48, line=513 (Compiled frame) - com.webobjects.appserver._private.WOWorkerThread.run() @bci=26, line=210 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 58115: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.util.TimerThread.mainLoop() @bci=201, line=552 (Interpreted frame) - java.util.TimerThread.run() @bci=1, line=505 (Interpreted frame) Thread 57603: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.core.QuartzSchedulerThread.run() @bci=807, line=411 (Interpreted frame) Thread 52995: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 52483: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 51971: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 51459: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 50947: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 50435: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 49923: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 49411: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 48899: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 48387: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - org.quartz.simpl.SimpleThreadPool$WorkerThread.run() @bci=51, line=543 (Compiled frame) Thread 43779: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - com.webobjects.appserver.WOApplication$_LifebeatThread.run() @bci=233, line=1619 (Interpreted frame) Thread 43267: (state = BLOCKED) - java.lang.Thread.sleep(long) @bci=0 (Compiled frame; information may be imprecise) - com.webobjects.appserver.WOSessionStore$_SessionTimeoutManager.run() @bci=12, line=98 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=745 (Interpreted frame) Thread 31491: (state = BLOCKED) Thread 23811: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.ref.ReferenceQueue.remove(long) @bci=44, line=142 (Compiled frame) - java.lang.ref.ReferenceQueue.remove() @bci=2, line=158 (Compiled frame) - java.lang.ref.Finalizer$FinalizerThread.run() @bci=36, line=209 (Interpreted frame) Thread 23299: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - java.lang.Object.wait() @bci=2, line=502 (Compiled frame) - java.lang.ref.Reference$ReferenceHandler.run() @bci=36, line=157 (Compiled frame) Thread 1295: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Compiled frame; information may be imprecise) - com.webobjects.appserver._WORunLoop._acceptInputBeforeDate(com.webobjects.foundation.NSTimestamp) @bci=98, line=244 (Interpreted frame) - com.webobjects.appserver._WORunLoop.runBeforeDate(com.webobjects.foundation.NSTimestamp) @bci=39, line=95 (Interpreted frame) - com.webobjects.appserver.WOApplication._runOnce() @bci=17, line=1095 (Interpreted frame) - com.webobjects.appserver.WOApplication.run() @bci=106, line=1248 (Interpreted frame) - er.extensions.appserver.ERXApplication.run() @bci=292, line=1417 (Interpreted frame) - com.webobjects.appserver.WOApplication.main(java.lang.String[], java.lang.Class) @bci=55, line=548 (Interpreted frame) - er.extensions.appserver.ERXApplication.main(java.lang.String[], java.lang.Class) @bci=6, line=861 (Interpreted frame) - edu.ucla.gseis.ets.Application.main(java.lang.String[]) @bci=3, line=68 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke0(java.lang.reflect.Method, java.lang.Object, java.lang.Object[]) @bci=0 (Interpreted frame) - sun.reflect.NativeMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=100, line=62 (Interpreted frame) - sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, java.lang.Object[]) @bci=6, line=43 (Interpreted frame) - java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object[]) @bci=56, line=483 (Interpreted frame) - com.webobjects._bootstrap.WOBootstrap.main(java.lang.String[]) @bci=118, line=87 (Interpreted frame) Tim Worman UCLA GSE&IS _______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list ([email protected]<mailto:[email protected]>) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-dev/chill%40gevityinc.com This email sent to [email protected]<mailto:[email protected]>
_______________________________________________ Do not post admin requests to the list. They will be ignored. Webobjects-dev mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com This email sent to [email protected]
