I think that this kind of strategy could easily scale to clusters of thousands of nodes and possibly much more (if you have a good communication substrate).
I looked at the comet documentation and it really looks like they went a long ways to make sure that they are a pure p2p coordination layer. That is definitely an interesting research project, but it makes it much, much less interesting in terms of realistic systems, if only because the resulting system is so much more complex. This sounds like an academic project, though, so comparing a p2p coordination layer with state of the commercial art alternatives is probably a good thing. I would recommend that if you want a very interesting result, you should implement your coordination with BOTH zookeeper and comet and figure out what the differences are. On Fri, Apr 24, 2009 at 11:47 PM, Arun C Murthy <a...@yahoo-inc.com> wrote: > I am not sure if this strategy is good for large scale Map_Reduce >> applications. But it might work well for small scale Map-reduce jobs. >> >> More Information on the co-ordination space that I want to use can be >> found >> here : >> http://www.caip.rutgers.edu/~zhljenny/comet.htm<http://www.caip.rutgers.edu/%7Ezhljenny/comet.htm>. >> > > -- Ted Dunning, CTO DeepDyve 111 West Evelyn Ave. Ste. 202 Sunnyvale, CA 94086 www.deepdyve.com 858-414-0013 (m) 408-773-0220 (fax)
