FWIW - since I don't use exclusive consumers in my application, I
commented out the first three lines in IndirectMessageReference.lock()
below, and that bottleneck has been removed.
if (!regionDestination.lock(this, subscription)) {
return false;
}
Anyway - hopefully this is of interest to somebody. Perhaps we can
optimise this so that if we don't use exclusive consumers to make the
above call a no-op?
Cheers,
David
David Sitsky wrote:
In my application, I have a large number of threads in the broker, which
are calling dispatchPending() on their subscriptions, but are unable to
make fast progress, as can be seen in the following stack trace:
"ActiveMQ Transport: tcp:///192.168.222.75:56495" daemon prio=4
tid=0x4be82000 n
id=0x1028 waiting for monitor entry [0x54fdf000..0x54fdfb00]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.activemq.broker.region.Queue.lock(Queue.java:166)
- locked <0x0a60bf58> (a java.lang.Object)
at
org.apache.activemq.broker.region.IndirectMessageReference.lock(IndirectMessageReference.java:143)
at
org.apache.activemq.broker.region.QueueSubscription.canDispatch(QueueSubscription.java:119)
at
org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:466)
- locked <0x0b55bd78> (a java.lang.Object)
at
org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:347)
at
org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:342)
at
org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:461)
at
org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:194)
at
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
at
org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:73)
at
org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:87)
at
org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:440)
at
org.apache.activemq.command.MessageAck.visit(MessageAck.java:196)
....
I am trying to understand why IndirectMessageReference.lock() would
require synchronisation on the entire Queue object it is associated with?
This is why the performance in my application is grinding... each time a
prefetch/queue subscription is testing to see if they can dispatch a
message, we are effectively serialising on the Queue object.
Any ideas on how we can fix this? I have some ideas, but thought the
experts are better qualified. :)
This became evident when I had 20 consumers hammering away. With 2,
you'd never see this.
Cheers,
David
