Hi Vishal, you can control the checkpoint frequency with the setting "giraph.checkpointFrequency" in your JobConfiguration. The default is set to 0 right now, meaning no checkpoints are made. You should def. check out the GiraphJob [0] code, where all these tuning knobs are documented.
--André [0] https://github.com/apache/giraph/blob/trunk/src/main/java/org/apache/giraph/graph/GiraphJob.java#L308 2012/8/11 Vishal Patel <[email protected]>: > Hi, > > How do I specify the interval for saving checkpoints? When working with > Amazon's Elastic Mapreduce on a large number of workers (> 80 workers, 40 x > m1.xlarge machines), sometimes there is RPC communication errors and > Zookeeper waits on that worker for a while before timing out and killing the > job all together. > > As my graph and number of workers is becoming larger I would like to learn > how to save it since that extra cost might be well worth it-- say every 50 > supersteps. Here is the command I use currently, how should I modify it. > > hadoop jar giraph-0.2-SNAPSHOT-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.ConnectedComponentsVertex \ > --inputFormat org.apache.giraph.examples.IntIntNullIntTextInputFormat \ > --inputPath giraph_in/adj_list.txt \ > --outputFormat > org.apache.giraph.examples.VertexWithComponentTextOutputFormat \ > --outputPath giraph_out > --combiner org.apache.giraph.examples.MinimumIntCombiner > --workers 95 > > Also, how do I restart from a specific checkpoint. The help for the > GiraphRunner class did not have instructions on this. > > Thank you! > > Vishal > >
