> What about the barrier sync of zookeeper? Does he can deal with these > multiple tasks? Would each task be a znode?
Hmm, good question. I think, ZK have to manage all tasks. Otherwise, it'll increase the complexity of the program. On Fri, Jul 8, 2011 at 11:46 AM, Edward J. Yoon <[email protected]> wrote: > Just FYI, > > To better understand, refer the diagram, described in 0.2 user guide: > > http://incubator.apache.org/hama/docs/r0.2.0/ApacheHama-0.2_UserGuide.pdf > > On Fri, Jul 8, 2011 at 6:33 AM, Thomas Jungblut > <[email protected]> wrote: >> Yes, that is already implemented in the latest patch. >> That is quite okay, I would be +1 to let each task be a BSPPeer. Or >> actually, has a BSPPeer, maybe we are going to add some kind of JVM reuse, >> then we just have to set a new BSPPeer instead of swapping the whole task. >> >> Overall I thought of this cascading design: >> >>>BSPMaster >> ->Groom1 >> -->Task1 >> -->Task2 >> ->Groom2 >> -->Task3 >> >> So each task can directly communicate with other tasks using RPC. (Altough >> I'm not a great friend of this RPC stuff [1]) >> Grooms are only there to communicate with each task, for pinging tasks to be >> alive. And the BSPMaster is responsible to keep track of the availability of >> the grooms. >> >> We should take care of syncs and use them as sparse as possible, since they >> tend to be a large bottleneck. >> What about the barrier sync of zookeeper? Does he can deal with these >> multiple tasks? Would each task be a znode? >> >> [1]: >> https://issues.apache.org/jira/browse/HAMA-358?focusedCommentId=13059229&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13059229 >> >> 2011/7/7 Edward J. Yoon <[email protected]>: >>> Invoked (child) process will become a BSPPeer. >>> >>> On Thu, Jul 7, 2011 at 9:14 PM, Thomas Jungblut >>> <[email protected]> wrote: >>>> Just for clarification: >>>> What is your plan now? >>>> To setup a BSPPeer for several tasks on a server (groom) or is the >>>> groom now the one and only BSPPeer? >>>> >>>> 2011/7/7 Edward J. Yoon <[email protected]>: >>>>> Hi, >>>>> >>>>> To support multi-tasks, I'm thinking about merging BSPPeer and Task. >>>>> Then, communication will be occurred among Tasks directly. I think, >>>>> there's no need to manage BSPPeers inside GroomServer. >>>>> >>>>> Can we think about the latent side-effects from this decision, together? >>>>> >>>>> Thanks. >>>>> >>>>> -- >>>>> Best Regards, Edward J. Yoon >>>>> @eddieyoon >>>>> >>>> >>>> >>>> >>>> -- >>>> Thomas Jungblut >>>> Berlin >>>> >>>> mobile: 0170-3081070 >>>> >>>> business: [email protected] >>>> private: [email protected] >>>> >>> >>> >>> >>> -- >>> Best Regards, Edward J. Yoon >>> @eddieyoon >>> >> >> >> >> -- >> Thomas Jungblut >> Berlin >> >> mobile: 0170-3081070 >> >> business: [email protected] >> private: [email protected] >> > > > > -- > Best Regards, Edward J. Yoon > @eddieyoon > -- Best Regards, Edward J. Yoon @eddieyoon
