That looks like the whole JVM locks up. I have _no_ idea what might
cause that!
Chuck
On Mar 26, 2008, at 1:56 PM, Gary Teter wrote:
n Mar 26, 2008, at 1:39 PM, Mike Schrag wrote:
Where are those threads blocked? Most will be blocked waiting for
incoming web requests, i assume. Are there are threads blocked
anywhere other than chilling in their WorkThread wait state?
Can you hit the direct connect port during this time?
ms
Yes, many worker threads are normally blocked waiting for incoming
requests (on a normal, undeaf instance), but normally you'll see at
least one or more threads that are actually doing something, e.g.:
Ordinary worker thread waiting:
Thread [EMAIL PROTECTED]: (state = BLOCKED)
- java.net.PlainSocketImpl.accept(java.net.SocketImpl) @bci=0,
line=382 (Interpreted frame)
- java.net.ServerSocket.implAccept(java.net.Socket) @bci=50,
line=450 (Interpreted frame)
- java.net.ServerSocket.accept() @bci=48, line=421 (Interpreted frame)
- com.webobjects.appserver._private.WOWorkerThread.run() @bci=26,
line=238 (Interpreted frame)
- java.lang.Thread.run() @bci=11, line=613 (Interpreted frame)
Ordinary thread doing something:
Thread [EMAIL PROTECTED]: (state = IN_NATIVE)
- java.net.SocketInputStream.socketRead0(java.io.FileDescriptor,
byte[], int, int, int) @bci=0 (Compiled frame; information may be
imprecise)
- java.net.SocketInputStream.read(byte[], int, int) @bci=84,
line=129 (Compiled frame)
- org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill()
@bci=31, line=50 (Interpreted frame)
Deaf instance blocking at random point:
Thread [EMAIL PROTECTED]: (state = BLOCKED)
- java.util.regex.Pattern.matcher(java.lang.CharSequence) @bci=25,
line=879 (Compiled frame)
-
com
.paizo
.util.StringUtilities.stringWithoutHTMLEntities(java.lang.String)
@bci=423, line=2206 (Compiled frame)
-
com
.paizo
.util.StringUtilities.htmlStringToStandardString(java.lang.String)
@bci=34, line=730 (Compiled frame)
Deaf instance blocking at random spot:
Thread [EMAIL PROTECTED]: (state = BLOCKED)
-
com
.webobjects
.eocontrol.EOGenericRecord._keyGetBindingForKey(java.lang.String)
@bci=10, line=171 (Compiled frame; information may be imprecise)
-
com
.webobjects.eocontrol.EOCustomObject.valueForKey(java.lang.String)
@bci=8, line=1558 (Compiled frame)
- sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object,
java.lang.Object[]) @bci=6, line=25 (Compiled frame)
-
com
.webobjects.eocontrol.EOCustomObject.valueForKey(java.lang.String)
@bci=14, line=1559 (Compiled frame)
- com.webobjects.foundation.NSKeyValueCoding
$Utility.valueForKey(java.lang.Object, java.lang.String) @bci=26,
line=498 (Compiled frame)
- com.webobjects.foundation.NSKeyValueCodingAdditions
$DefaultImplementation.valueForKeyPath(java.lang.Object,
java.lang.String) @bci=19, line=212 (Compiled frame)
-
com
.webobjects
.eocontrol.EOKeyValueQualifier.evaluateWithObject(java.lang.Object)
@bci=5, line=131 (Compiled frame)
-
com
.webobjects
.eocontrol
.EOQualifier
.filteredArrayWithQualifier(com.webobjects.foundation.NSArray,
com.webobjects.eocontrol.EOQualifier) @bci=55, line=635 (Compiled
frame)
O
On Mar 26, 2008, at 4:37 PM, Gary Teter wrote:
Now I'm completely baffled. I've switched memcached clients, and
now I'm still getting temporarily deaf instances, but when I point
jstack at one, it now says that ALL threads are "state = BLOCKED".
After a couple of minutes the instance comes back.
Anyone have any ideas what could possibly be going on?
On Mar 25, 2008, at 4:18 PM, Chuck Hill wrote:
On Mar 25, 2008, at 4:05 PM, Gary Teter wrote:
jstack for the win! Not sure how I forgot about that tool, but
it was able to get me good info.
Chalk one up for jstack!
I switched to ActiveMQ, which is a billion times better than
OpenJMS, but we're still having temporarily deaf instances.
jstack reported that Every. Single. Thread. was "state =
BLOCKED", except one:
Thread [EMAIL PROTECTED]: (state = IN_NATIVE)
- sun.nio.ch.KQueueArrayWrapper.kevent0(int, long, int, long)
@bci=0 (Interpreted frame)
- sun.nio.ch.KQueueArrayWrapper.poll(long) @bci=12, line=118
(Interpreted frame)
- sun.nio.ch.KQueueSelectorImpl.doSelect(long) @bci=46, line=69
(Interpreted frame)
- sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=69
(Interpreted frame)
- sun.nio.ch.SelectorImpl.select(long) @bci=30, line=80
(Compiled frame)
- net.spy.memcached.MemcachedClient.run() @bci=11, line=1054
(Compiled frame)
Error occurred during stack walking:
java.lang.NullPointerException
at
sun.jvm.hotspot.runtime.Frame.addressOfStackSlot(Frame.java:214)
at
sun.jvm.hotspot.runtime.x86.X86Frame.getSenderSP(X86Frame.java:
355)
at
sun.jvm.hotspot.runtime.x86.X86Frame.sender(X86Frame.java:218)
at sun.jvm.hotspot.runtime.Frame.sender(Frame.java:184)
at sun.jvm.hotspot.runtime.Frame.realSender(Frame.java:189)
at sun.jvm.hotspot.runtime.VFrame.sender(VFrame.java:102)
at sun.jvm.hotspot.runtime.VFrame.javaSender(VFrame.java:134)
at sun.jvm.hotspot.tools.StackTrace.run(StackTrace.java:50)
at sun.jvm.hotspot.tools.JStack.run(JStack.java:41)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:204)
at sun.jvm.hotspot.tools.JStack.main(JStack.java:62)
So it looks like I'm off to download a different memcached client.
--
WireHose: Smart metadata and personalization for WebObjects.
http://wirehose.com/
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list ([email protected])
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/chill%40global-village.net
This email sent to [EMAIL PROTECTED]
--
Practical WebObjects - for developers who want to increase their
overall knowledge of WebObjects or who are trying to solve specific
problems.
http://www.global-village.net/products/practical_webobjects
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Webobjects-dev mailing list ([email protected])
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/webobjects-dev/archive%40mail-archive.com
This email sent to [EMAIL PROTECTED]
