Rajan Arul created AMQ-8034:
-------------------------------
Summary: Client connection get blocked at ensureConnectionInfoSent
Key: AMQ-8034
URL: https://issues.apache.org/jira/browse/AMQ-8034
Project: ActiveMQ
Issue Type: Bug
Components: JMS client
Affects Versions: 5.15.11
Reporter: Rajan Arul
The activemq client seems to get blocked forever on the application side. We
analyzed the thread dump and the most of the threads are stuck here
{code:java}
org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1479)
org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:329)
org.apache.activemq.jms.pool.ConnectionPool.makeSession(ConnectionPool.java:112)
org.apache.activemq.jms.pool.ConnectionPool$1.makeObject(ConnectionPool.java:82)
org.apache.activemq.jms.pool.ConnectionPool$1.makeObject(ConnectionPool.java:78)
org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:342)
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:265)
org.apache.activemq.jms.pool.ConnectionPool.createSession(ConnectionPool.java:144)
org.apache.activemq.jms.pool.PooledConnection.createSession(PooledConnection.java:167)
org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:208)
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:495)
org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:526)
org.apache.camel.component.jms.JmsProducer.doSend(JmsProducer.java:440)
org.apache.camel.component.jms.JmsProducer.processInOnly(JmsProducer.java:394)
org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:157)
org.apache.camel.processor.SendProcessor.process(SendProcessor.java:148)
org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
{code}
The above threads were blocked by other thread which has progressed further and
was stuck here
{code:java}
java.lang.Thread.State: WAITING (parking) at
jdk.internal.misc.Unsafe.park([email protected]/Native Method) -
parking to wait for (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at
java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
at
java.util.concurrent.ArrayBlockingQueue.take([email protected]/ArrayBlockingQueue.java:417)
at
org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:48)
at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:87)
at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1392)
at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1436)
at
org.apache.activemq.AdvisoryConsumer.(AdvisoryConsumer.java:51)
at
org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQConnection.java:1495)
- locked (a java.lang.Object) at
org.apache.activemq.ActiveMQConnection.createSession(ActiveMQConnection.java:329)
at
org.apache.activemq.jms.pool.ConnectionPool.makeSession(ConnectionPool.java:112)
at
org.apache.activemq.jms.pool.ConnectionPool$1.makeObject(ConnectionPool.java:82)
at
org.apache.activemq.jms.pool.ConnectionPool$1.makeObject(ConnectionPool.java:78)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.create(GenericKeyedObjectPool.java:1041)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:342)
at
org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:265)
at
org.apache.activemq.jms.pool.ConnectionPool.createSession(ConnectionPool.java:144)
at
org.apache.activemq.jms.pool.PooledConnection.createSession(PooledConnection.java:167)
at
org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:208)
at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:495)
at
org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:526)
at
org.apache.camel.component.jms.JmsProducer.doSend(JmsProducer.java:440)
at
org.apache.camel.component.jms.JmsProducer.processInOnly(JmsProducer.java:394)
at
org.apache.camel.component.jms.JmsProducer.process(JmsProducer.java:157)
{code}
This issue seems to happen only on clustered (master/slave) environments . We
have not seen this on a single node environment.
Based on code analysis so far, the client seems to be sending an advisory
message back to the server and it seem to get stuck forever and never recovers.
I am not super familiar with this code base but I think adding a timeout to the
syncSendPack in this code might help see comments in ==> in the code section
below.
{code:java}
public AdvisoryConsumer(ActiveMQConnection connection, ConsumerId consumerId)
throws JMSException {
this.connection = connection;
info = new ConsumerInfo(consumerId);
info.setDestination(AdvisorySupport.TEMP_DESTINATION_COMPOSITE_ADVISORY_TOPIC);
info.setPrefetchSize(1000);
info.setNoLocal(true);
info.setDispatchAsync(true);
this.connection.addDispatcher(info.getConsumerId(), this);
this.connection.syncSendPacket(this.info); ======> Use the syncSendPacket
method with timeout option
}
{code}
Please review and let me know if this change will help.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
