[ https://issues.apache.org/activemq/browse/AMQ-688?page=comments#action_36516 ]
james strachan commented on AMQ-688: ------------------------------------ A few comments on this issue... #1 we already support the dropping of messages for non-durable topic subscribers... http://incubator.apache.org/activemq/slow-consumer-handling.html so that parts done right? #2 we support eviction and reloading of messages for durable topic subscribers/durable messages and for durable queues #3 the prefetch buffer act as the buffer of in RAM messages; we back fill these one at a time. We could have some pluggable strategy to provide more optimised back-filling - though its a tricky algorithm as thanks to selectors and different subscription rates - its hard to know what messages are required to be fetched. #4 currently for performance, we tend to dispatch to non-durable topic subscriptions in a single thread to many producers to minimise context switching when using blocking IO. If we used a thread pool for this (async sending) or AIO or NIO then it would be easier to avoid the thread blocking on the send - right now blocking on the send there's no real way we can interupt it. So its clearly a performance hit tradeoff; fast versus fair. #5 basically means to use spooling to disk right? > Avoid blocking producers > ------------------------ > > Key: AMQ-688 > URL: https://issues.apache.org/activemq/browse/AMQ-688 > Project: ActiveMQ > Type: New Feature > Components: Broker > Versions: 4.0 RC2 > Reporter: Christopher A. Larrieu > Assignee: Christopher A. Larrieu > Fix For: 4.1 > > Original Estimate: 8 weeks > Remaining: 8 weeks > > Our main goal > is to avoid stalled producers by addressing the main culprit: too many > undispatched messages > in the broker's memory. Our motivation is to handle significant --though > temporary-- imbalances > between production and consumption rates. > Reaching this goal entails specific broker modifications: > 1. When memory gets tight, start dropping undispatched non-persistent > messages. This is the > first-cut attempt to maintain throughput of persistent messages. > Unlike the approach documented at > http://docs.codehaus.org/display/ACTIVEMQ/Slow+Consumer+Handling, > the message dropping will only occur after the UsageManager reaches capacity. > Non-persistent > messages in dispatch lists will be dropped according to per-destination > policy. Subscriptions > can purge their own messages triggered via callback from the UsageManager. > 2. Evict messages if memory remains tight, to be fetched from backing store > prior to dispatch. > ActiveMQ already supports this for persistent messages on Topics with durable > subscriptions. > If a consumer's prefetch buffer is full, the splash-over messages remain as > IndirectMessageReference > objects in the dispatch list, with the actual message body loaded from store > on demand. I > believe we can extend this approach for Queues as well. > 3. Improve the efficiency with which evicted messages are loaded back into > memory. > Currently, they are loaded one at a time as needed. It would make sense to > batch-load message > sets periodically. This will require a significant shift in responsibilities > between objects, > since an IndirectMessageReference doesn't know about other instances that can > be loaded in > mass. > > The goal will be to keep each subscription dispatch list stocked with a > decent number of messages > in-memory to reasonably trade-off between it's consumer's performance and > resource usage in > the broker. As with everything else, we can implement this as a strategy > class with the first > cut implementing a simple resource allocation strategy: divvy up available > memory amongst > all subscriptions and keep that memory filled with messages for dispatch. I > envision a worker > task assuming responsibility for keeping these lists filled. > 4. Even with the above modifications, we still can't entirely avoid blocked > producers, so > we'd like to add client-configurable time-outs to provide a bound for the > time a producer > can remain stalled. > Maybe this should be a new attribute of ActiveMQConnection: > maxProducerFlowControlWait. Calls > to UsageManager.waitForSpace can take this quantity as an argument. Failure > to reach sufficient > space for the new message will throw an exception back up the stack and > across the wire, letting > the producer know that the message was not delivered. > 5. Finally, we need to extend disk support for Topics that have only > non-durable subscribers, > otherwise their dispatch lists can fill up with persistent messages. In > order to maintain > compliance with JMS, it would be nice to provide some alternative to dropping > persistent messages. > One possible first cut is to layer this on top of a DurableTopicSubscription > by creating an > anonymous subscriber for every Topic that has only non-durable subscriptions. > When all such > subscriptions terminate, the broker can remove the anonymous subscriber. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/activemq/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
