Dear All, We were fairly happy users of RabbitMQ until we got a use case which demands for a large number of queues. Large being in the hundreds of thousands queues range (not counting DLQ).
It looks like we can have a cluster of Qpid brokers, each with a sane number of queues, and use qpid-dispatch to route the messages to their brokers. However, I am concerned about the size of the qpid-dispatch config we would end up writing. Is there anything that may help us automate this config management? My main concern is a new queue creation. Ideally we would want to find the least busy broker and create a new queue there. Would such a large routing table affect qpid-dispatch performance? I will work on setting up a Proof-of-Concept Qpid cluster with 100K queues shortly. If you have any tips or have ran Qpid in a similar mode, I would appreciate the information. P.S. The need for a large number of queues comes out of a fair consumption requirement which we are doing this way: https://nithril.github.io/amqp/2015/07/05/fair-consuming-with-rabbitmq/ If there is a Qpid way of achieving this without 100K queues, I would love to hear about it. -- Aleksey --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
