Thanks! Sounds like there might be another way we can configure the flow control that blocks only one producer? Do you have any recommendation on what changes I should make to the config?
On Wed, Jan 2, 2013 at 12:33 PM, Christian Posta <christian.po...@gmail.com>wrote: > Enqueue count = number of messages sent to the queue > Dequeue count = number of messages ack'd by the consumer > Inflight count = number of messages sent to consumer's session (and are not > ack'd by consumer) > Dispatch count = messages sent to the consumer (dequeue + inflight) > > I asked about the async sends because the producer flow control you hit was > the kind that will block the entire connection (instead of just blocking > that one producer). > > > > > > On Wed, Jan 2, 2013 at 12:41 PM, Mohit Anchlia <mohitanch...@gmail.com > >wrote: > > > Thanks again! Is the dispatch queue size the one that relates to what's > > held in memory? > > > > We are using mule and the endpoint has no transactions configured. What > did > > you mean by forcing async? > > > > On Wed, Jan 2, 2013 at 11:06 AM, Christian Posta > > <christian.po...@gmail.com>wrote: > > > > > Correct. The memoryUsage is increased when messages are held in memory. > > > This happens when messages are paged in from the store. They are then > > > dispatched to consumers and held until the consumer acks. > > > > > > In your case looks like a particular destination (eventsEndpoint) is > > > filling up somehow. Can you look in JMX and see how many enqueued, > > > dispatched, dequeued, inflight etc? Also check to see how many > > > subscriptions to that destination, and see what those dispatch-counter > > and > > > dispatch queue size looks like. > > > > > > BTW any chance you're sending (producers) using a transaction, or > forcing > > > sends to be async? > > > > > > > > > > > > On Wed, Jan 2, 2013 at 11:51 AM, Mohit Anchlia <mohitanch...@gmail.com > > > >wrote: > > > > > > > So if I understand correctly I shouldn't be seeing this behaviour > > unless > > > > something is being held in memory. We ack immediately so I am kind of > > > > wondering what might be going on here. Is there a way to find out > what > > > > might be happening here? > > > > > > > > On Wed, Jan 2, 2013 at 10:03 AM, Christian Posta > > > > <christian.po...@gmail.com>wrote: > > > > > > > > > What info can you provide about how clients are consuming from the > > > > queues? > > > > > Any chance there are a bunch of messages "inflight" which haven't > > been > > > > > ack'd by the consumer? > > > > > > > > > > > > > > > On Wed, Jan 2, 2013 at 10:20 AM, Mohit Anchlia < > > mohitanch...@gmail.com > > > > > >wrote: > > > > > > > > > > > I see these messages being logged. Queues are setup to use > > persistent > > > > > > store. I am wondering what's causing this messages to get > blocked. > > I > > > > see > > > > > > there is a memoryLimit but if I am using persistent=true then > > should > > > > this > > > > > > be the desired behaviour? > > > > > > > > > > > > > > > > > > 2013-01-02 05:06:13,388 | INFO | > > > > > > Usage(default:memory:queue://eventsEndpoint:memory) > > percentUsage=0%, > > > > > > usage=0, limit=20971520, > > > > > > percentUsageMinDelta=1%;Parent:Usage(default:memory) > > > percentUsage=106%, > > > > > > usage=22366029, limit=20971520, percentUsageMinDelta=1%: Usage > > > Manager > > > > > > Memory Limit reached. Producer > > > > > > (ID:pfdamq301.ie.net-54582-1351809386355-2:1:1:1) stopped to > > prevent > > > > > > flooding queue://eventsEndpoint. See > > > > > > > > > > > > > > > > > > <broker xmlns="http://activemq.apache.org/schema/core"; > > > > > > brokerName="static-broker1" persistent="true" > > > > > > dataDirectory="${activemq.data}"> > > > > > > > > > > > > <!-- Destination specific policies using destination > names > > or > > > > > > wildcards --> > > > > > > <destinationPolicy> > > > > > > <policyMap> > > > > > > <policyEntries> > > > > > > <policyEntry queue=">" > > producerFlowControl="true" > > > > > > memoryLimit="20mb"> > > > > > > <deadLetterStrategy> > > > > > > <individualDeadLetterStrategy > > > > > queuePrefix="DLQ." > > > > > > useQueueForQueueMessages="true" /> > > > > > > </deadLetterStrategy> > > > > > > </policyEntry> > > > > > > <policyEntry topic=">" > > producerFlowControl="true" > > > > > > memoryLimit="20mb"> > > > > > > </policyEntry> > > > > > > </policyEntries> > > > > > > </policyMap> > > > > > > </destinationPolicy> > > > > > > > > > > > > <!-- Use the following to configure how ActiveMQ is > exposed > > > in > > > > > JMX > > > > > > --> > > > > > > <managementContext> > > > > > > <managementContext createConnector="true"/> > > > > > > </managementContext> > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > *Christian Posta* > > > > > http://www.christianposta.com/blog > > > > > twitter: @christianposta > > > > > > > > > > > > > > > > > > > > > -- > > > *Christian Posta* > > > http://www.christianposta.com/blog > > > twitter: @christianposta > > > > > > > > > -- > *Christian Posta* > http://www.christianposta.com/blog > twitter: @christianposta >
