Re: Review Request: Move part of the graph out-of-core when memory is low

2012-08-22 Thread Maja Kabiljo
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5987/#review10614 --- Alessandro, can you please try the same test with a lot of vertices

[jira] [Updated] (GIRAPH-293) Should aggregators be checkpointed?

2012-08-22 Thread Maja Kabiljo (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maja Kabiljo updated GIRAPH-293: Attachment: GIRAPH-293.patch Forgot to mark as 'Patch Available' last time. I'm just rebasing it

[jira] [Commented] (GIRAPH-293) Should aggregators be checkpointed?

2012-08-22 Thread Alessandro Presta (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439414#comment-13439414 ] Alessandro Presta commented on GIRAPH-293: -- If there's no merge conflict, you

[jira] [Updated] (GIRAPH-249) Move part of the graph out-of-core when memory is low

2012-08-22 Thread Alessandro Presta (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Presta updated GIRAPH-249: - Attachment: GIRAPH-249.patch Updated to match review

[jira] [Commented] (GIRAPH-293) Should aggregators be checkpointed?

2012-08-22 Thread Maja Kabiljo (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439475#comment-13439475 ] Maja Kabiljo commented on GIRAPH-293: -

[jira] [Created] (GIRAPH-310) Simplify vertex initialization

2012-08-22 Thread Alessandro Presta (JIRA)
Alessandro Presta created GIRAPH-310: Summary: Simplify vertex initialization Key: GIRAPH-310 URL: https://issues.apache.org/jira/browse/GIRAPH-310 Project: Giraph Issue Type:

[jira] [Created] (GIRAPH-311) Master halting in superstep 0 is ignored by workers

2012-08-22 Thread Maja Kabiljo (JIRA)
Maja Kabiljo created GIRAPH-311: --- Summary: Master halting in superstep 0 is ignored by workers Key: GIRAPH-311 URL: https://issues.apache.org/jira/browse/GIRAPH-311 Project: Giraph Issue Type:

Re: Giraph Job Task attempt_* failed to report status Problem

2012-08-22 Thread Vishal Patel
After several supersteps, sometimes a worker thread dies (say it ran out of memory). Zookeeper waits for ~5 mins (600 seconds) and then decides that the worker is not responsive and fails the entire job. At this point if you have a checkpoint saved it will resume from there otherwise you have to

Re: Review Request: Move part of the graph out-of-core when memory is low

2012-08-22 Thread Alessandro Presta
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5987/ --- (Updated Aug. 22, 2012, 10:34 a.m.) Review request for giraph and Avery Ching.

Re: Review Request: Move part of the graph out-of-core when memory is low

2012-08-22 Thread Maja Kabiljo
On Aug. 22, 2012, 8:55 a.m., Maja Kabiljo wrote: Alessandro, can you please try the same test with a lot of vertices (you only had 10 vertices here)? If vertices are visited in random order in the prepareSuperstep, I think you should have much more disk operations. Also,

Re: Review Request: Move part of the graph out-of-core when memory is low

2012-08-22 Thread Alessandro Presta
On Aug. 22, 2012, 8:55 a.m., Maja Kabiljo wrote: Alessandro, can you please try the same test with a lot of vertices (you only had 10 vertices here)? If vertices are visited in random order in the prepareSuperstep, I think you should have much more disk operations. Also,

Re: Review Request: Move part of the graph out-of-core when memory is low

2012-08-22 Thread Alessandro Presta
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5987/ --- (Updated Aug. 22, 2012, 12:36 p.m.) Review request for giraph and Avery Ching.

Re: Review Request: Move part of the graph out-of-core when memory is low

2012-08-22 Thread Alessandro Presta
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5987/ --- (Updated Aug. 22, 2012, 1 p.m.) Review request for giraph and Avery Ching.

Review Request: Fixing checkpointing for aggregators

2012-08-22 Thread Maja Kabiljo
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6731/ --- Review request for giraph. Description --- Making aggregators work

[jira] [Updated] (GIRAPH-301) InputSplit Reservations are clumping, leaving many workers asleep while other process too many splits and get overloaded.

2012-08-22 Thread Eli Reisman (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Reisman updated GIRAPH-301: --- Attachment: GIRAPH-301-7.patch just a quick rebase, no changes. InputSplit

[jira] [Commented] (GIRAPH-211) Add secure authentication to Netty IPC

2012-08-22 Thread Eugene Koontz (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439703#comment-13439703 ] Eugene Koontz commented on GIRAPH-211: -- Hi Alessandro, I am working actively on it

[jira] [Commented] (GIRAPH-211) Add secure authentication to Netty IPC

2012-08-22 Thread Alessandro Presta (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439739#comment-13439739 ] Alessandro Presta commented on GIRAPH-211: -- Awesome, looking forward to it!

[jira] [Updated] (GIRAPH-312) Giraph needs an admin script

2012-08-22 Thread Eli Reisman (JIRA)
[ https://issues.apache.org/jira/browse/GIRAPH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Reisman updated GIRAPH-312: --- Attachment: GIRAPH-312-1.patch Giraph needs an admin script