Hi Eran, Yes, I agree there can be many internal communications. The communication I was referring to was the communication involved with running jobs in a distributed manner. For example there can be many GFac instances capable of running an experiment. I believe at the moment Airavata uses or plan to use a Thrift call to ask the GFac to execute the experiment. This communication can be done through ZooKeeper using a state change.
Going back to Storm, Nimbus uses ZooKeeper to communicate the things like worker node assignments to typologies through the ZooKeeper [1]. In Storm there is nothing between Nimbus and Supervisors other than ZooKeeper. The state is saved in the ZooKeeper and state change is being used as the communication. If you look at how Storm is using ZooKeeper it is very close to what Airavata needs. [1] https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/cluster.clj Regards, Supun.. On Sat, Jun 14, 2014 at 7:42 PM, Eran Chinthaka Withana < [email protected]> wrote: > Hi > > On Fri, Jun 13, 2014 at 9:27 PM, Supun Kamburugamuva <[email protected]> > wrote: > > > Best example of using ZooKeeper as a communication + state holding place > is > > Apache Storm. > > > > http://storm.incubator.apache.org/ > > > Supun, IIRC, storm uses ZK to keep status and coordination between workers > (bolts and spouts). The fact that multiple workers in storm uses that > information to take the decisions is, IMO, not considered as internal > communication. Its distributed coordination. > > I felt like the meaning of "internal communication" is overloaded here. > > *Suresh *we desperately need an example here for internal communication you > are talking about :) > > -- Eran > -- Supun Kamburugamuva Member, Apache Software Foundation; http://www.apache.org E-mail: [email protected]; Mobile: +1 812 369 6762 Blog: http://supunk.blogspot.com
