Interesting and valuable feedback, thanks Phil, Rajith and Gordon. On Thu, Sep 6, 2012 at 8:42 PM, Rajith Attapattu <[email protected]> wrote:
> I would also lean towards option one. > Agree with Gordon that there is a lot of overhead of reconnecting > periodically. > > I don't have enough info about your situation, but are you able to > partition your queues across several brokers? > If so you could reduce the number of connections per broker. > If you plan to deploy your broker on just a single box I guess it > doesn't really make much of a diff. > As of now we are planning to have a single broker, and havent really thought of partitioning, but if need arises we should be able to partition. > Btw what clients and broker are you planning to use ? > > We are thinking of using Qpid Java broker with RabbitMQ C or Python client. > I would also suggest building a POC to see how it performs in your > targeted env. > I've personally seen production apps that use 5k+ queues with close to > 200 odd connections with decent latency and performance. > > Yeah, will do a POC > Rajith. > > On Thu, Sep 6, 2012 at 6:20 AM, Gordon Sim <[email protected]> wrote: > > On 09/05/2012 09:41 PM, Sajith Kariyawasam wrote: > >> > >> Hi all, > >> > >> This time its more like a design/architectural query I have to make, > that > >> it, we are going to implement a notifier functionality in our app > >> deployment. > >> > >> when an application is deployed to the central server it needs to > notify a > >> collection of nodes that an application is uploaded/updated. > >> > >> There can be thousands of nodes of different types based on the type of > >> application they are interested about. > >> > >> Here, node is the receiver and what is the best way to implement > >> receiver functionality? > >> 1. Should we make each and every node continuously listening to a > >> queue > >> of his type of application ? > >> 2. Should the node make a call to a queue periodically and check > >> whether any messages are there? > >> > >> In 1 advantage is that, the moment application gets uploaded relevent > >> nodes > >> gets the notification, but concerns about the performance. > >> > >> If solution 1 is followed, there will be thousands of active connections > >> to > >> the broker, is that manageable ? How many number of active connections > >> Qpid > >> broker can handle at large ? > > > > > > > > I would strongly lean towards option 1. Periodically resubscribing to the > > queue and checking whether there are any messages may result in more > broker > > load (in AMQP 0-10, the connection handshake is quite involved for > example. > > You also would need to make sure that the reconnect/check cycles of the > > nodes were evenly spaced (since if they all checked at the same time that > > would defeat the purpose). > > > > Several thousand connections doesn't seem problematic to me. The number > can > > affect latency and even throughput, but it doesn't sound like that would > be > > an issue if the alternative is periodic polling anyway. > > > > The c++ broker uses a fixed pool of threads. It does allocate some buffer > > space per connection, but that shouldn't rule out thousands of > connections. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Best Regards Sajith
