What you've explained is distributed coordination. When Suresh mentioned about internal communication I thought its about how internal services communicate to get the work done. Lets see what the usecases are. May be I misunderstood the requirements.
Thanks, Eran Chinthaka Withana On Sat, Jun 14, 2014 at 5:39 PM, Supun Kamburugamuva <[email protected]> wrote: > Hi Eran, > > Yes, I agree there can be many internal communications. The communication I > was referring to was the communication involved with running jobs in a > distributed manner. For example there can be many GFac instances capable of > running an experiment. I believe at the moment Airavata uses or plan to use > a Thrift call to ask the GFac to execute the experiment. This communication > can be done through ZooKeeper using a state change. > > Going back to Storm, Nimbus uses ZooKeeper to communicate the things like > worker node assignments to typologies through the ZooKeeper [1]. In Storm > there is nothing between Nimbus and Supervisors other than ZooKeeper. The > state is saved in the ZooKeeper and state change is being used as the > communication. If you look at how Storm is using ZooKeeper it is very close > to what Airavata needs. > > [1] > > https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/cluster.clj > > Regards, > Supun.. > > > On Sat, Jun 14, 2014 at 7:42 PM, Eran Chinthaka Withana < > [email protected]> wrote: > > > Hi > > > > On Fri, Jun 13, 2014 at 9:27 PM, Supun Kamburugamuva <[email protected]> > > wrote: > > > > > Best example of using ZooKeeper as a communication + state holding > place > > is > > > Apache Storm. > > > > > > http://storm.incubator.apache.org/ > > > > > > Supun, IIRC, storm uses ZK to keep status and coordination between > workers > > (bolts and spouts). The fact that multiple workers in storm uses that > > information to take the decisions is, IMO, not considered as internal > > communication. Its distributed coordination. > > > > I felt like the meaning of "internal communication" is overloaded here. > > > > *Suresh *we desperately need an example here for internal communication > you > > are talking about :) > > > > -- Eran > > > > > > -- > Supun Kamburugamuva > Member, Apache Software Foundation; http://www.apache.org > E-mail: [email protected]; Mobile: +1 812 369 6762 > Blog: http://supunk.blogspot.com >
