Re: Review Request: GIRAPH-57 Add new RPC call (putVertexIdMessagesList) to batch putMsgList RPCs together

[Claudio Martella]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3201/#review3928
---

Ship it!


Looks cool and straight from here. All those CODE_CONVENTIONS fixes made it 
looks scary from far.

- Claudio


On 2011-12-14 19:50:29, Avery Ching wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3201/
> ---
> 
> (Updated 2011-12-14 19:50:29)
> 
> 
> Review request for giraph.
> 
> 
> Summary
> ---
> 
> Added new putVertexIdMessagesList RPC and supporting classes 
> (VertexIdMessages and VertexIdMessagesList) to reduce the total number of 
> RPCs during a flush.  This improves the number of RPC / sec and overall I/O 
> bandwidth.  The amount of batching is done by the total number of messages 
> and is configurable at runtime (default of 5000, weighted toward helping 
> smaller messages).  I have noted some performance results in 
> https://issues.apache.org/jira/browse/GIRAPH-57 (between 25 - 1075 percent 
> improvements).
> 
> Also, while tinkering with BasicRPCCommunications, notices inconsistent 
> spaces between 'synchronized' and '('.  Removed spaces and standardized in 
> the CODE_CONVENTIONS.
> 
> 
> This addresses bug GIRAPH-57.
> https://issues.apache.org/jira/browse/GIRAPH-57
> 
> 
> Diffs
> -
> 
>   http://svn.apache.org/repos/asf/incubator/giraph/trunk/CODE_CONVENTIONS 
> 1213849 
>   
> http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/ArrayListWritable.java
>  1213849 
>   
> http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
>  1214406 
>   
> http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java
>  1213849 
>   
> http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/VertexIdMessages.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java
>  PRE-CREATION 
>   
> http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GiraphJob.java
>  1213849 
> 
> Diff: https://reviews.apache.org/r/3201/diff
> 
> 
> Testing
> ---
> 
> Passed local and Hadoop unittests.  Used the RandomMessageBenchmark on a 
> small cluster.
> 
> 
> Thanks,
> 
> Avery
> 
>

Re: Review Request: GIRAPH-57 Add new RPC call (putVertexIdMessagesList) to batch putMsgList RPCs together

[Avery Ching]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3201/
---

(Updated 2011-12-14 19:50:29.358317)


Review request for giraph.


Changes
---

Forgot a few important files (VertexIdMessages.java and 
VertexIdMessagesList.java)


Summary
---

Added new putVertexIdMessagesList RPC and supporting classes (VertexIdMessages 
and VertexIdMessagesList) to reduce the total number of RPCs during a flush.  
This improves the number of RPC / sec and overall I/O bandwidth.  The amount of 
batching is done by the total number of messages and is configurable at runtime 
(default of 5000, weighted toward helping smaller messages).  I have noted some 
performance results in https://issues.apache.org/jira/browse/GIRAPH-57 (between 
25 - 1075 percent improvements).

Also, while tinkering with BasicRPCCommunications, notices inconsistent spaces 
between 'synchronized' and '('.  Removed spaces and standardized in the 
CODE_CONVENTIONS.


This addresses bug GIRAPH-57.
https://issues.apache.org/jira/browse/GIRAPH-57


Diffs (updated)
-

  http://svn.apache.org/repos/asf/incubator/giraph/trunk/CODE_CONVENTIONS 
1213849 
  
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/ArrayListWritable.java
 1213849 
  
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
 1214406 
  
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java
 1213849 
  
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/VertexIdMessages.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java
 PRE-CREATION 
  
http://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GiraphJob.java
 1213849 

Diff: https://reviews.apache.org/r/3201/diff


Testing
---

Passed local and Hadoop unittests.  Used the RandomMessageBenchmark on a small 
cluster.


Thanks,

Avery