> On Jan. 30, 2013, 5:51 p.m., Eli Reisman wrote: > > Maybe put in a named constant where you're using a -1 flag a few places, > > but this is a big +1 from me, its just a nit, up to you. This will be very > > helpful!
Thanks for the review Eli, I pushed after r2 changes. - Avery ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9138/#review15871 ----------------------------------------------------------- On Jan. 30, 2013, 7:01 p.m., Avery Ching wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/9138/ > ----------------------------------------------------------- > > (Updated Jan. 30, 2013, 7:01 p.m.) > > > Review request for giraph. > > > Description > ------- > > Adds status updates to saving of the vertices and improve the overall logic > of when to print status of loading/storing the graph (every 250k vertices or > 15 secs). This will help us to see which are the slow workers when saving > output. This updates the Hadoop status messages as well as prints to the task > log. I also made this consistent for the loading. > > Task log messages look like the following: > > INFO 2013-01-29 12:51:46,044 [main] > org.apache.giraph.worker.BspServiceWorker - saveVertices: Saved 98751 out of > 1000000 vertices, on partition 2 out of 24 > INFO 2013-01-29 12:52:25,539 [main] > org.apache.giraph.worker.BspServiceWorker - saveVertices: Saved 348752 out > of 1000000 vertices, on partition 8 out of 24 > INFO 2013-01-29 12:53:28,062 [main] > org.apache.giraph.worker.BspServiceWorker - saveVertices: Saved 598753 out > of 1000000 vertices, on partition 14 out of 24 > > I added an option for dumping output to PageRankBenchmark to test this as > well. > > > This addresses bug GIRAPH-492. > https://issues.apache.org/jira/browse/GIRAPH-492 > > > Diffs > ----- > > > giraph-core/src/main/java/org/apache/giraph/benchmark/PageRankBenchmark.java > 3ef471a711183dd147990c5f6bb07485a58f5a71 > giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedService.java > 83fba57af3f11f6c412ec59e2d121e57ea280d98 > > giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceMaster.java > 399dc72896673fbf75d2b7b933c4ea02a08f25ea > > giraph-core/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java > 294c2c71017b5273240c9dce1f3de2be65aed289 > > giraph-core/src/main/java/org/apache/giraph/graph/FinishedSuperstepStats.java > d888d1038026c83b5f82956b6872eb64a44dd700 > giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java > 401e07bb346e8ac43600992718f1f231db85aa7c > giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java > 7ad290244946b89c6a563a86536d8e898e1c0aec > giraph-core/src/main/java/org/apache/giraph/utils/LoggerUtils.java > 81dfd1d8d9b27442543db1259235fb1825f72f7e > giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java > d5ad62b39517f06884ef1c003c5e91348cbe2459 > > giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallable.java > 7522027b7306aff17668e17f4895a39e23e8a590 > > Diff: https://reviews.apache.org/r/9138/diff/ > > > Testing > ------- > > Passsed unittests and tested on a real cluster with PageRankBenchmark. > > > Thanks, > > Avery Ching > >
