So your overall broker memory limit and the individual destination limits are set to the same number. Each destination will think it has 200MB, but it will not (unless there is only one destination). How many destinations do you have? When you run into this issue again, can you check which destination is using all this memory?
On Fri, Jan 4, 2013 at 11:00 AM, Mohit Anchlia <mohitanch...@gmail.com>wrote: > I restarted activemq so everything was reset and currently all of them > showing same values: > > <broker xmlns="http://activemq.apache.org/schema/core"; brokerName=" > pprfdaamq301.ie.id.net" persistent="true" > dataDirectory="${activemq.data}"> > <!-- Destination specific policies using destination names or > wildcards --> > <destinationPolicy> > <policyMap> > <policyEntries> > <policyEntry queue=">" producerFlowControl="true" > memoryLimit="200mb"> > <deadLetterStrategy> > <individualDeadLetterStrategy queuePrefix="DLQ." > useQueueForQueueMessages="true" /> > </deadLetterStrategy> > </policyEntry> > <policyEntry topic=">" producerFlowControl="true" > memoryLimit="20mb"> > </policyEntry> > </policyEntries> > </policyMap> > </destinationPolicy> > <!-- Use the following to configure how ActiveMQ is exposed in JMX > --> > <managementContext> > <managementContext createConnector="true"/> > </managementContext> > <!-- > The store and forward broker networks ActiveMQ will listen to. > We'll leave it empty as duplex network will be configured by > another broker > Take a look at activemq-static_network-broker2.xml for example > --> > <networkConnectors> > <networkConnector uri="static:(tcp:// > pprfdaamq300.ie.id.net:61616)"/> > </networkConnectors> > <persistenceAdapter> > <kahaDB directory="${activemq.data}/ > pprfdaamq301.ie.id.net/kahadb" enableJournalDiskSyncs="false" > indexWriteBatchSize="10000" indexCacheSize="1000"/> > </persistenceAdapter> > <!-- The maximum amount of space the broker will use before > slowing down producers --> > <systemUsage> > <systemUsage> > <memoryUsage> > <memoryUsage limit="200 mb"/> > </memoryUsage> > <storeUsage> > <storeUsage limit="100 gb" name="foo"/> > </storeUsage> > <tempUsage> > <tempUsage limit="1000 mb"/> > </tempUsage> > </systemUsage> > </systemUsage> > <!-- The transport connectors ActiveMQ will listen to --> > <transportConnectors> > <transportConnector name="openwire" uri="tcp://0.0.0.0:61616 > "/> > </transportConnectors> > </broker> > > > On Fri, Jan 4, 2013 at 9:53 AM, Christian Posta > <christian.po...@gmail.com>wrote: > > > DispatchAsync controls how the broker delivers messages to the bridge's > > network (proxy) consumer. > > > > please post the broker configs you're using as well as the > > destination/subscriptions stats (inflight, dispatchcount, enqueue, > dequeue, > > etc)? > > > > > > On Fri, Jan 4, 2013 at 10:29 AM, Mohit Anchlia <mohitanch...@gmail.com > > >wrote: > > > > > Is it better to set dispathAsync=false on the network of brokers? > > Currently > > > it looks like this? > > > > > > Name< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > Network TTL< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > Dynamic Only< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > Conduit Subscriptions< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > Bridge Temps< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > Decrease Priorities< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > Dispatch Async< > > > > > > http://pprfdaamq300.ie.intuit.net:8161/admin/connections.jsp;jsessionid=ndx7f1ktjogh158jxtqfmr79t# > > > > > > > NC 1 false true true false true > > > > > > On Fri, Jan 4, 2013 at 9:26 AM, James Green <james.mk.gr...@gmail.com > > > >wrote: > > > > > > > We have observed this when the link between A & B is slow. Probably a > > > > different scenario to the one Mohit is describing though. > > > > > > > > In our case we had a hub behind a shared ADSL link and several spokes > > in > > > > "production land". We would see messages piling up at random on the > > > spokes > > > > until restarts where issued of the brokers. In the end, out of ideas, > > we > > > > shifted the hub to be in the same network at the spokes and messages > > > began > > > > flowing far more reliably. > > > > > > > > James > > > > > > > > > > > > On 4 January 2013 15:10, Christian Posta <christian.po...@gmail.com> > > > > wrote: > > > > > > > > > PFC can still affect the network bridge. We should figure out why > > > > messages > > > > > are piling up (unexpectedly??). Do you have a test case that can > > > > reproduce > > > > > this? > > > > > > > > > > > > > > > On Thu, Jan 3, 2013 at 3:47 PM, Mohit Anchlia < > > mohitanch...@gmail.com > > > > > >wrote: > > > > > > > > > > > I think it might be because of this flow control message that I > see > > > > > between > > > > > > brokers. Here pprfdaa300 is another broker. But why would this > > happen > > > > > > between brokers though? > > > > > > > > > > > > 2013-01-03 14:44:21,587 | INFO | > > > > > > Usage(default:memory:queue://eventsEndpoint:memory) > > percentUsage=0%, > > > > > > usage=0, limit=20971520, > > > > > > percentUsageMinDelta=1%;Parent:Usage(default:memory) > > > percentUsage=106%, > > > > > > usage=22325599, limit=20971520, percentUsageMinDelta=1%: Usage > > > Manager > > > > > > Memory Limit reached. Producer > > > > > > (ID:pprfdaa300.ind.net-51311-1351809358414-145227:1:1:1) stopped > to > > > > > prevent > > > > > > flooding queue://eventsEndpoint. See > > > > > > http://activemq.apache.org/producer-flow-control.html for more > > info > > > > > > (blocking for: 6822s) | org.apache.activemq.broker.region.Queue | > > > > > ActiveMQ > > > > > > Transport: tcp://pprfdaa300.ind.net/10.145.67.135:61616@41478 > > > > > > > > > > > > On Thu, Jan 3, 2013 at 2:41 PM, Christian Posta > > > > > > <christian.po...@gmail.com>wrote: > > > > > > > > > > > > > Can you post the config for the two brokers? > > > > > > > > > > > > > > > > > > > > > On Thu, Jan 3, 2013 at 3:35 PM, Mohit Anchlia < > > > > mohitanch...@gmail.com > > > > > > > >wrote: > > > > > > > > > > > > > > > It's A <----> B > > > > > > > > > > > > > > > > Broker A has 2 messages but no consumers > > > > > > > > Broker B has consumers but don't get messages forwarded > > > > > > > > > > > > > > > > On Thu, Jan 3, 2013 at 1:33 PM, Christian Posta > > > > > > > > <christian.po...@gmail.com>wrote: > > > > > > > > > > > > > > > > > Can you explain how the brokers are set up? A ---> B ? and > > > which > > > > > has > > > > > > > the > > > > > > > > > messages and which has the consumer? > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jan 3, 2013 at 2:19 PM, Mohit Anchlia < > > > > > > mohitanch...@gmail.com > > > > > > > > > >wrote: > > > > > > > > > > > > > > > > > > > As I understand that if I have network of brokers than > the > > > > > messages > > > > > > > > will > > > > > > > > > be > > > > > > > > > > sent to the broker that has active consumers on it. I > have > > 2 > > > > > > network > > > > > > > of > > > > > > > > > > brokers and one of them has 2 messages and no client > > > consumers > > > > > and > > > > > > > > other > > > > > > > > > > one has consumers. I was expecting those 2 messages to be > > > > pulled > > > > > by > > > > > > > > other > > > > > > > > > > broker for processing. Could you please help me > understand > > > why > > > > > > those > > > > > > > 2 > > > > > > > > > > messages wouldn't get processed? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > *Christian Posta* > > > > > > > > > http://www.christianposta.com/blog > > > > > > > > > twitter: @christianposta > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > *Christian Posta* > > > > > > > http://www.christianposta.com/blog > > > > > > > twitter: @christianposta > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > *Christian Posta* > > > > > http://www.christianposta.com/blog > > > > > twitter: @christianposta > > > > > > > > > > > > > > > > > > > > -- > > *Christian Posta* > > http://www.christianposta.com/blog > > twitter: @christianposta > > > -- *Christian Posta* http://www.christianposta.com/blog twitter: @christianposta
