You know, the more that I work with map-reduce, the less I find that is really hard to do with it.
Graph traversal is a great case. Looks hard because you potentially have lots of passes through MR to get your result. But most problems of practical import scale the number of map-reduce passes with the diameter of the graph. For small world graphs, this diameter is much less than log N (that is what 6 degrees of separation is all about!) and in some cases looks more like log log N. To put this in perspective, if the planet has 10^10 people (I call this 10 billion, but this is confusing) and each person has 100 computers, then log N scaling for finding all shortest paths will require somewhere around 30 passes. Log-log scaling will require 5. This is not a big deal. On Thu, Aug 28, 2008 at 7:31 AM, Robin Anil <[EMAIL PROTECTED]> wrote: > I havent gone through the features of it. .But from the motive of it. > Parallelizing it by threads will help is cases where the problem is not > easy > to reduce to Map-Reduce.(like Graph Traversal) There will be counter > arguments also. Grid-Computing, p2p algorithms are tailor made for such a > system > > Robin > -- ted
