Unfortunately I can still reproduce a deadlock on 8.5. I saw the non-8.5
update a few minutes ago, so maybe more changes are coming. Just wanted
to let you know. jstack output attached. Reproduction based on
6285e750e2b640fa9c59c11c09a753a36a5396b8.
Am 03.01.2022 um 18:25 schrieb ma...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/8.5.x by this push:
new 6285e75 Improve fix to avoid deadlock reported on some systems
6285e75 is described below
commit 6285e750e2b640fa9c59c11c09a753a36a5396b8
Author: Mark Thomas <ma...@apache.org>
AuthorDate: Mon Jan 3 17:24:59 2022 +0000
Improve fix to avoid deadlock reported on some systems
https://markmail.org/message/ldpjfdwpkmqc7ved
---
java/org/apache/coyote/http2/Http2UpgradeHandler.java | 14 ++++++++++++--
java/org/apache/coyote/http2/Stream.java | 18 ++++++++++--------
2 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
index 1195997..a2c87ed 100644
--- a/java/org/apache/coyote/http2/Http2UpgradeHandler.java
+++ b/java/org/apache/coyote/http2/Http2UpgradeHandler.java
@@ -345,7 +345,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
// continue reading frames
Stream stream = getStream(se.getStreamId(),
false);
if (stream == null) {
- sendStreamReset(se);
+ sendStreamReset(null, se);
} else {
stream.close(se);
}
@@ -535,7 +535,7 @@ class Http2UpgradeHandler extends AbstractStream implements
InternalHttpUpgradeH
}
- void sendStreamReset(StreamException se) throws IOException {
+ void sendStreamReset(StreamStateMachine state, StreamException se) throws
IOException {
if (log.isDebugEnabled()) {
log.debug(sm.getString("upgradeHandler.rst.debug", connectionId,
@@ -554,7 +554,17 @@ class Http2UpgradeHandler extends AbstractStream
implements InternalHttpUpgradeH
// Payload
ByteUtil.setFourBytes(rstFrame, 9, se.getError().getCode());
+ // Need to update state atomically with the sending of the RST
+ // frame else other threads currently working with this stream
+ // may see the state change and send a RST frame before the RST
+ // frame triggered by this thread. If that happens the client
+ // may see out of order RST frames which may hard to follow if
+ // the client is unaware the RST frames may be received out of
+ // order.
synchronized (socketWrapper) {
+ if (state != null) {
+ state.sendReset();
+ }
socketWrapper.write(true, rstFrame, 0, rstFrame.length);
socketWrapper.flush(true);
}
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index fffd240..a4d6484 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -614,14 +614,16 @@ class Stream extends AbstractNonZeroStream implements
HeaderEmitter {
log.debug(sm.getString("stream.reset.send",
getConnectionId(), getIdAsString(),
se.getError()));
}
- // Sync ensures that if the call to sendReset() triggers resets
- // in other threads, that the RST frame associated with this
- // thread is sent before the RST frames associated with those
- // threads.
- synchronized (state) {
- state.sendReset();
- handler.sendStreamReset(se);
- }
+
+ // Need to update state atomically with the sending of the RST
+ // frame else other threads currently working with this stream
+ // may see the state change and send a RST frame before the RST
+ // frame triggered by this thread. If that happens the client
+ // may see out of order RST frames which may hard to follow if
+ // the client is unaware the RST frames may be received out of
+ // order.
+ handler.sendStreamReset(state, se);
+
cancelAllocationRequests();
if (inputBuffer != null) {
inputBuffer.swallowUnread();
2022-01-03 20:05:11
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.212-b10 mixed mode):
"Attach Listener" #29 daemon prio=9 os_prio=64 tid=0x0000000100d96800 nid=0x20
waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"http-nio-127.0.0.1-auto-1-34445-AsyncTimeout" #28 daemon prio=5 os_prio=64
tid=0x0000000100e46800 nid=0x1f waiting on condition [0xfffffffe9e0ff000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.coyote.AbstractProtocol$AsyncTimeout.run(AbstractProtocol.java:1213)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-Acceptor-0" #27 daemon prio=5 os_prio=64
tid=0x0000000100e45800 nid=0x1e runnable [0xfffffffe9e2fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
- locked <0xffffffff43402250> (a java.lang.Object)
at
org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:512)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-ClientPoller-1" #26 daemon prio=5 os_prio=64
tid=0x0000000100e65000 nid=0x1d runnable [0xfffffffe9e4fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0xffffffff41ab83b8> (a sun.nio.ch.Util$3)
- locked <0xffffffff41ab83a8> (a java.util.Collections$UnmodifiableSet)
- locked <0xffffffff41ab8290> (a sun.nio.ch.DevPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:875)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-ClientPoller-0" #25 daemon prio=5 os_prio=64
tid=0x0000000100d5d800 nid=0x1c runnable [0xfffffffe9e6fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0xffffffff41aa7908> (a sun.nio.ch.Util$3)
- locked <0xffffffff41aa78f8> (a java.util.Collections$UnmodifiableSet)
- locked <0xffffffff41aa77e0> (a sun.nio.ch.DevPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at
org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:875)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-10" #24 daemon prio=5 os_prio=64
tid=0x0000000100ee5000 nid=0x1b waiting on condition [0xfffffffe9e8fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-9" #23 daemon prio=5 os_prio=64
tid=0x0000000100ee3800 nid=0x1a waiting for monitor entry [0xfffffffe9eafe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.coyote.http2.Http2Parser.readDataFrame(Http2Parser.java:181)
- waiting to lock <0xffffffff41be8c68> (a java.nio.HeapByteBuffer)
at org.apache.coyote.http2.Http2Parser.readFrame(Http2Parser.java:95)
at org.apache.coyote.http2.Http2Parser.readFrame(Http2Parser.java:69)
at
org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch(Http2UpgradeHandler.java:340)
at
org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:60)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:59)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:849)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1677)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
- locked <0xffffffff41f5eda0> (a
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-8" #22 daemon prio=5 os_prio=64
tid=0x0000000100ee1800 nid=0x19 waiting on condition [0xfffffffe9ecfe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-7" #21 daemon prio=5 os_prio=64
tid=0x0000000100bf4800 nid=0x18 waiting on condition [0xfffffffe9eefe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-6" #20 daemon prio=5 os_prio=64
tid=0x0000000100e12000 nid=0x17 waiting on condition [0xfffffffe9f0fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-5" #19 daemon prio=5 os_prio=64
tid=0x0000000100bf2000 nid=0x16 waiting for monitor entry [0xfffffffe9f2fe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.apache.coyote.http2.Http2UpgradeHandler.writeWindowUpdate(Http2UpgradeHandler.java:832)
- waiting to lock <0xffffffff41f5eda0> (a
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at
org.apache.coyote.http2.Http2UpgradeHandler.onSwallowedDataFramePayload(Http2UpgradeHandler.java:1697)
at
org.apache.coyote.http2.Stream$StreamInputBuffer.swallowUnread(Stream.java:1319)
- locked <0xffffffff41be8c68> (a java.nio.HeapByteBuffer)
at
org.apache.coyote.http2.Stream$StreamInputBuffer.access$600(Stream.java:1011)
at org.apache.coyote.http2.Stream.close(Stream.java:629)
at
org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:85)
- locked <0xffffffff41bd30f8> (a
org.apache.coyote.http2.StreamProcessor)
at org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-4" #18 daemon prio=5 os_prio=64
tid=0x0000000100bf0000 nid=0x15 waiting on condition [0xfffffffe9f4fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-3" #17 daemon prio=5 os_prio=64
tid=0x0000000100d1b800 nid=0x14 waiting on condition [0xfffffffe9f6fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-2" #16 daemon prio=5 os_prio=64
tid=0x0000000100d70000 nid=0x13 waiting on condition [0xfffffffe9f8fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-1" #15 daemon prio=5 os_prio=64
tid=0x0000000100b81800 nid=0x12 waiting on condition [0xfffffffe9fafe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff41507c28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:146)
at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1114)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1176)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"ContainerBackgroundProcessor[StandardEngine[Tomcat]]" #14 daemon prio=5
os_prio=64 tid=0x0000000100d2f000 nid=0x11 waiting on condition
[0xfffffffe9fcff000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1363)
at java.lang.Thread.run(Thread.java:748)
"NioBlockingSelector.BlockPoller-0" #11 daemon prio=5 os_prio=64
tid=0x0000000100d47000 nid=0xe runnable [0xfffffffea05fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
- locked <0xffffffff43200788> (a sun.nio.ch.Util$3)
- locked <0xffffffff43200798> (a java.util.Collections$UnmodifiableSet)
- locked <0xffffffff43200740> (a sun.nio.ch.DevPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at
org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:343)
"AsyncFileHandlerWriter-127618319" #10 daemon prio=5 os_prio=64
tid=0x00000001008d8800 nid=0xd waiting on condition [0xfffffffea10fe000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xffffffff43208a00> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:492)
at
java.util.concurrent.LinkedBlockingDeque.take(LinkedBlockingDeque.java:680)
at
org.apache.juli.AsyncFileHandler$LoggerThread.run(AsyncFileHandler.java:204)
"Service Thread" #7 daemon prio=9 os_prio=64 tid=0x0000000100310000 nid=0xb
runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread1" #6 daemon prio=9 os_prio=64 tid=0x000000010030b800 nid=0xa
waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C2 CompilerThread0" #5 daemon prio=9 os_prio=64 tid=0x0000000100308800 nid=0x9
waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=64 tid=0x0000000100307800 nid=0x8
runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=64 tid=0x00000001002d1800 nid=0x7 in
Object.wait() [0xfffffffea40fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xffffffff43200aa0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
- locked <0xffffffff43200aa0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
"Reference Handler" #2 daemon prio=10 os_prio=64 tid=0x00000001002c9800 nid=0x6
in Object.wait() [0xfffffffea42fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xffffffff43200c58> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0xffffffff43200c58> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"main" #1 prio=5 os_prio=64 tid=0x0000000100111000 nid=0x2 runnable
[0xffffffff7bcfb000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
at java.net.SocketOutputStream.write(SocketOutputStream.java:143)
at
org.apache.coyote.http2.Http2TestBase.writeFrame(Http2TestBase.java:448)
at
org.apache.coyote.http2.Http2TestBase.writeFrame(Http2TestBase.java:442)
at
org.apache.coyote.http2.Http2TestBase.writeFrame(Http2TestBase.java:436)
at
org.apache.coyote.http2.TestCancelledUpload.testCancelledRequest(TestCancelledUpload.java:63)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:50)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:536)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1178)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1032)
"VM Thread" os_prio=64 tid=0x00000001002bf000 nid=0x5 runnable
"GC task thread#0 (ParallelGC)" os_prio=64 tid=0x000000010011e000 nid=0x3
runnable
"GC task thread#1 (ParallelGC)" os_prio=64 tid=0x0000000100126800 nid=0x4
runnable
"VM Periodic Task Thread" os_prio=64 tid=0x0000000100314800 nid=0xc waiting on
condition
JNI global references: 343
Found one Java-level deadlock:
=============================
"http-nio-127.0.0.1-auto-1-exec-9":
waiting to lock monitor 0x00000001002d0728 (object 0xffffffff41be8c68, a
java.nio.HeapByteBuffer),
which is held by "http-nio-127.0.0.1-auto-1-exec-5"
"http-nio-127.0.0.1-auto-1-exec-5":
waiting to lock monitor 0x00000001002d0938 (object 0xffffffff41f5eda0, a
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper),
which is held by "http-nio-127.0.0.1-auto-1-exec-9"
Java stack information for the threads listed above:
===================================================
"http-nio-127.0.0.1-auto-1-exec-9":
at
org.apache.coyote.http2.Http2Parser.readDataFrame(Http2Parser.java:181)
- waiting to lock <0xffffffff41be8c68> (a java.nio.HeapByteBuffer)
at org.apache.coyote.http2.Http2Parser.readFrame(Http2Parser.java:95)
at org.apache.coyote.http2.Http2Parser.readFrame(Http2Parser.java:69)
at
org.apache.coyote.http2.Http2UpgradeHandler.upgradeDispatch(Http2UpgradeHandler.java:340)
at
org.apache.coyote.http11.upgrade.UpgradeProcessorInternal.dispatch(UpgradeProcessorInternal.java:60)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:59)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:849)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1677)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
- locked <0xffffffff41f5eda0> (a
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
"http-nio-127.0.0.1-auto-1-exec-5":
at
org.apache.coyote.http2.Http2UpgradeHandler.writeWindowUpdate(Http2UpgradeHandler.java:832)
- waiting to lock <0xffffffff41f5eda0> (a
org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
at
org.apache.coyote.http2.Http2UpgradeHandler.onSwallowedDataFramePayload(Http2UpgradeHandler.java:1697)
at
org.apache.coyote.http2.Stream$StreamInputBuffer.swallowUnread(Stream.java:1319)
- locked <0xffffffff41be8c68> (a java.nio.HeapByteBuffer)
at
org.apache.coyote.http2.Stream$StreamInputBuffer.access$600(Stream.java:1011)
at org.apache.coyote.http2.Stream.close(Stream.java:629)
at
org.apache.coyote.http2.StreamProcessor.process(StreamProcessor.java:85)
- locked <0xffffffff41bd30f8> (a
org.apache.coyote.http2.StreamProcessor)
at org.apache.coyote.http2.StreamRunnable.run(StreamRunnable.java:35)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Found 1 deadlock.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
