Instead of using such a potentially large network of 'embedded' brokers, could you use a smaller cluster of 'standalone' brokers? A single broker is capable of handling many clients and throwing in a master/slave configuration along with 'failover' connectors for the clients should provide the necessary high availability.
Joe http://www.ttmsolutions.com Martin_Cornelius wrote: > > Hi all, > <p> > i am currently evaluating the scalability of Networks of ActiveMQ Brokers > with a large number of brokers. The actual drive behind this evaluation: I > have to design a distributed system that is completely serverless. It > shall be possible to switch of any of the machines in the system without > alloying the distributed functionality. The number of active machines in > the system might vary from only 2 to about 100.<p> > My current idea: Have an (presumably embedded) ActiveMQ Broker running on > any of the machines, and let all this Brokers form a 'Network of Brokers' > (abbreviated NOB in the follwing) > <p> > So far, i found that in an NOB a message is forwarded only to those > brokers where at least one client has subscribed to the topic resp. queue > the message is sent to. Really fine. If a client subscribes to a queue, > the broker where the subscription is made sends a single short > control-message to all other brokers of the NOB. That would still allow > for thousands of brokers, fine again. > <p> > However, i just realized one fact that may compromise scalabilty: If a > client subscribes to a topic, the broker where the subscription is done > sends a not-so-short control-message to all other brokers. So far not too > bad, but now comes the problem: Every broker seems to react on this > control-message by sending another not-so-short control-message to all > other brokers. This means, if N brokers are present, every subscription to > a topic results in about (N-1)*N*2 control-messages (The additional factor > two comes from the fact, that the final recipient-broker seems to send > another response control-message). BTW, The length of the control-messages > being sent seems to be closely related to the string length of the topic > name. > <p> > I have verified this behaviour with only up to 6 brokers, so i'm not > totally shure if it will extrapolate to a larger number, but i suspect it > does. This would mean, that with e.g. 100 Brokers every subscription to a > topic would result in 100*99*2 control messages being sent. If a single > control message has about 400 bytes (what i observed), this means a single > subscription to a topic would result in about 8 Megabyte network traffic. > This might even be sustainable, but the quadratic order kills scalabilty > -- with 1000 Brokers we would have to transfer about 1 Gigabyte, what is > no longer feasible. > <p> > Finally, my questions: Is there any configuration option by which this > quadratic order can be avoided, is my idea simply foolish, or are my > conclusions wrong ? Note that i gained my conclusions more or less > 'empirically' using wireshark. > <p> > thanks for any advice, Martin > > -- View this message in context: http://www.nabble.com/Scalability-of-%27Networks-of-Brokers%27-tp19335699p19337053.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
