Hi Phil, On Thu, Sep 6, 2012 at 3:03 AM, Phil Harvey <[email protected] > wrote:
> Hi Sajith, > > Apart from the fact that you have ~1000 nodes, what other non-functional > requirements do you have? For example: > > - What is the acceptable time lag between the application being deployed > and a node receiving notification? > About 30 seconds. (Upon receiving the notification the node should get the updated copy from an SVN repo) > - What frequency of application deployments do you expect? > Not very frequently > - Which of the following notification semantics do you require: > exactly-once / at-least-once / at-most-once ? - Do you need to handle message "conflation" addressed by last-value queues > (https://cwiki.apache.org/qpid/use-last-value-queues-lvq.html)? > I guess, its "exactly-once" and yes, now I feel its better to incorporate LVQ, because the notification message is always be a simple text message with the value "updated", and nodes get synced with an SVN repo upon receiving that "update" message. When an application is uploaded into the central server, the producer adds the "updated" message to the queue. Think of a situation if the same app is uploaded 3 times (with updates) and at that time nodes may not have started. This will create 3 messages in the queue, and upon the node is started it will get 3 notifications and will invoke "svn update" 3 times, which is not necessary. Therefore it seems we should have LVQ - How are your nodes distributed, e.g. is each nodes a separate process? > Each node is a separate machine > > On the technical side, have you already made any decisions? For example: > > - Are you planning to use the Java or C++ Qpid broker? > We are planning to use Java Broker > - What platform/language do your nodes use? > Nodes will be installed with Ubuntu 12.04, and preferred language of client code is C/C++. Even python is ok. > > Most of my experience with the Java broker, which dedicates a thread to > each connection. I don't expect a few thousand connections would cause > problems but I believe others on this list have more experience of testing > the broker's upper limits. > > > > Phil > > > On 5 September 2012 21:41, Sajith Kariyawasam <[email protected]> wrote: > > > Hi all, > > > > This time its more like a design/architectural query I have to make, that > > it, we are going to implement a notifier functionality in our app > > deployment. > > > > when an application is deployed to the central server it needs to notify > a > > collection of nodes that an application is uploaded/updated. > > > > There can be thousands of nodes of different types based on the type of > > application they are interested about. > > > > Here, node is the receiver and what is the best way to implement > > receiver functionality? > > 1. Should we make each and every node continuously listening to a > queue > > of his type of application ? > > 2. Should the node make a call to a queue periodically and check > > whether any messages are there? > > > > In 1 advantage is that, the moment application gets uploaded relevent > nodes > > gets the notification, but concerns about the performance. > > > > If solution 1 is followed, there will be thousands of active connections > to > > the broker, is that manageable ? How many number of active connections > Qpid > > broker can handle at large ? > > > > -- > > Best Regards > > Sajith > > > -- Best Regards Sajith
