Awesome! I can't wait to try this out. Is there any other resources / updated documentation where I can get insight into how Giraph works internally?
I am specifically interested in understanding how Mapper-only makes this entire thing possible, what are the key-value pairs (since it's built on top of Hadoop), and where Zookeeper fits in. Best, Deepak On Sun, Jan 22, 2012 at 5:30 PM, Avery Ching <ach...@apache.org> wrote: > Hi Deepak, > > Answers inline. > > Happy sunday! > > Avery > > > On 1/22/12 11:30 AM, Deepak Nettem wrote: > > Hi Folks, > > I have a Graph processing problem where after each iteration, some > vertices get vanished. That is, they get merged into their neighbouring > nodes based on certain conditions and the graph keeps getting simplified > after each iteration. > > I was wondering if Giraph is worth trying for this. > > Giraph can support this graph mutation at any iteration in the graph. > See > https://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/MutableVertex.java, > the method r emoveVertexRequest() will remove vertices for you between > iterations. > > > I was going through the documentation and it says that the input data has > to be sorted.Why is this necessary? > > > Ouch, this used to be a requirement, but no longer. You can load vertices > however you like. The workers will forward them to the appropriate > partition. > > > Also, how does the so called 'master' divide vertices in ranges? Does it > use some kind of a range partitioner? If there is range partitioning, > that's a problem for me, because of vanishing vertices - there will be load > imbalance. > > Again, out of date documentation. Please see > https://issues.apache.org/jira/browse/GIRAPH-11 for the relevant change. > Let us know if you have any other questions. > > Best, > Deepak > > >