Hi all, i am currently evaluating the scalability of Networks of ActiveMQ Brokers with a large number of brokers. The actual drive behind this evaluation: I have to design a distributed system that is completely serverless. It shall be possible to switch of any of the machines in the system without alloying the distributed functionality. The number of active machines in the system might vary from only 2 to about 100.
My current idea: Have an (presumably embedded) ActiveMQ Broker running on any of the machines, and let all this Brokers form a 'Network of Brokers' (abbreviated NOB in the follwing) So far, i found that in an NOB a message is forwarded only to those brokers where at least one client has subscribed to the topic resp. queue the message is sent to. Really fine. If a client subscribes to a queue, the broker where the subscription is made sends a single short control-message to all other brokers of the NOB. That would still allow for thousands of brokers, fine again. However, i just realized one fact that may compromise scalabilty: If a client subscribes to a topic, the broker where the subscription is done sends a not-so-short control-message to all other brokers. So far not too bad, but now comes the problem: Every broker seems to react on this control-message by sending another not-so-short control-message to all other brokers. This means, if N brokers are present, every subscription to a topic results in about (N-1)*N*2 control-messages (The additional factor two comes from the fact, that the final recipient-broker seems to send another response control-message). BTW, The length of the control-messages being sent seems to be closely related to the string length of the topic name. I have verified this behaviour with only up to 6 brokers, so i'm not totally shure if it will extrapolate to a larger number, but i suspect it does. This would mean, that with e.g. 100 Brokers every subscription to a topic would result in 100*99*2 control messages being sent. If a single control message has about 400 bytes (what i observed), this means a single subscription to a topic would result in about 8 Megabyte network traffic. This might even be sustainable, but the quadratic order kills scalabilty -- with 1000 Brokers we would have to transfer about 1 Gigabyte, what is no longer feasible. Finally, my questions: Is there any configuration option by which this quadratic order can be avoided, is my idea simply foolish, or are my conclusions wrong ? Note that i gained my conclusions more or less 'empirically' using wireshark. thanks for any advice, Martin -- View this message in context: http://www.nabble.com/Scalability-of-%27Networks-of-Brokers%27-tp19335699p19335699.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.
