Hello Mates, I am stuck with deadlock in DefaultClientExchangeHandlerImpl. Code works fine with 4.0-beta3 version. Looked at Async code and I am assuming deadlock is because of incorrect callbacks returned while leasing connection. I am running with load of 180 rps. Connections per service is configured to 512.
Anyone else hit this issue before ? Can this be a bug with new version or I am not using it correctly ? Appreciate any help or pointers. httpclient jars in classpath: httpcore-nio-4.3.jar httpcore-4.3.jar httpclient-4.3.jar httpasyncclient-4.0-beta4.jar Please find stack for 2 threads which are in deadlock. Found one Java-level deadlock: ============================= "ActiveMQ Session Task-15": waiting to lock monitor 0x00007fbde00016a0 (object 0x0000000225e00770, a org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl), which is held by "ActiveMQ Session Task-9" "ActiveMQ Session Task-9": waiting to lock monitor 0x00007fbf2400d478 (object 0x0000000225e00810, a org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl), which is held by "ActiveMQ Session Task-15" Java stack information for the threads listed above: =================================================== "ActiveMQ Session Task-15": at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.connectionAllocated(DefaultClientExchangeHandlerImpl.java:266) - waiting to lock <0x0000000225e00770> (a org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.access$000(DefaultClientExchangeHandlerImpl.java:57) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl$1.completed(DefaultClientExchangeHandlerImpl.java:333) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl$1.completed(DefaultClientExchangeHandlerImpl.java:340) at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalPoolEntryCallback.completed(PoolingNHttpClientConnectionManager.java:439) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalPoolEntryCallback.completed(PoolingNHttpClientConnectionManager.java:423) at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115) at org.apache.http.nio.pool.AbstractNIOConnPool.fireCallbacks(AbstractNIOConnPool.java:443) at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:276) at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:248) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:239) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.requestConnection(DefaultClientExchangeHandlerImpl.java:326) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:120) - locked <0x0000000225e00810> (a org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl) at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:188) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:63) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:93) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:78) at com.Sender.sendUpdate(Sender.java:68) at com.Consumer.onMessage(Consumer.java:97) at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1321) - locked <0x000000022307e698> (a java.lang.Object) at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131) at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202) at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129) at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) "ActiveMQ Session Task-9": at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.connectionAllocated(DefaultClientExchangeHandlerImpl.java:266) - waiting to lock <0x0000000225e00810> (a org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.access$000(DefaultClientExchangeHandlerImpl.java:57) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl$1.completed(DefaultClientExchangeHandlerImpl.java:333) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl$1.completed(DefaultClientExchangeHandlerImpl.java:340) at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalPoolEntryCallback.completed(PoolingNHttpClientConnectionManager.java:439) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$InternalPoolEntryCallback.completed(PoolingNHttpClientConnectionManager.java:423) at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:115) at org.apache.http.nio.pool.AbstractNIOConnPool.fireCallbacks(AbstractNIOConnPool.java:443) at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:276) at org.apache.http.nio.pool.AbstractNIOConnPool.lease(AbstractNIOConnPool.java:248) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.requestConnection(PoolingNHttpClientConnectionManager.java:239) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.requestConnection(DefaultClientExchangeHandlerImpl.java:326) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:120) - locked <0x0000000225e00770> (a org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl) at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:188) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:63) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:93) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:78) at com.Sender.sendUpdate(Sender.java:68) at com.Consumer.onMessage(Consumer.java:97) at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1321) - locked <0x0000000222f57100> (a java.lang.Object) at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131) at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202) at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129) at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) --jaikit
