On Sun, Apr 24, 2011 at 6:58 AM, ripienaar <r...@devco.net> wrote: > I am working on a system that has large amounts of topics often connected in > a network of brokers. Clients are mostly using the Stomp protocol. I am > now looking at also adding potentially a lot of queues and was seeking some > design advice on scalability. > > For me the most important characteristic is that message latency should be > low even for 1000s of subscribes, this works well today with just topics and > subscribers subscribing to a topic of messages they want. > > I am concerned that adding a lot - 1000s - of queues would impact > performance a lot due to the typical 1 thread per queue, topic and > subscriber (I know these are tunable and I can use NIO). > > I can now redesign things so that instead of many many topics and queues I > use headers and JMS selectors when subscribing and publishing, this way I > can have only a hand full of queues and topics with subscribers maintaining > many subscriptions on these queues/topics. > > It's not clear which of these two strategies will work best in the long run > and so was wondering what would be the most resource intensive on the broker > and the network of brokers wrt to threads, cpu, memory etc. and what would > be the best practice for this kind of application keeping in mind message > latency is very important.
I'm assuming that you have read the following page on the ActiveMQ website: http://activemq.apache.org/how-do-i-configure-10s-of-1000s-of-queues-in-a-single-broker-.html Instead of thousands of destinations or thousands of JMS selectors, my experience has taught me that a mix of the two is the best approach (especially in a situation where you have thousands of JMS clients connecting to the network of brokers). Although you can tune ActiveMQ to handle it, thousands of destinations tend to tax the broker unnecessarily, especially if you can send messages on the same destination and partition those messages using JMS selectors and/or message groups. But remember that selectors must be applied to every message that flows through the broker, so there is a tipping point for them as well. I have achieved the best results by dividing messages across a smaller number of destinations using a smaller number of selectors. Finding the appropriate balance requires experimentation in your environment with lifelike load testing. Bruce -- perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*" );' ActiveMQ in Action: http://bit.ly/2je6cQ Blog: http://bruceblog.org/ Twitter: http://twitter.com/brucesnyder
