[
https://issues.apache.org/jira/browse/GIRAPH-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169611#comment-13169611
]
Hudson commented on GIRAPH-104:
-------------------------------
Integrated in Giraph-trunk-Commit #47 (See
[https://builds.apache.org/job/Giraph-trunk-Commit/47/])
GIRAPH-104: Save half of maximum memory used from messaging. (aching)
aching : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1214406
Files :
* /incubator/giraph/trunk/CHANGELOG
*
/incubator/giraph/trunk/src/main/java/org/apache/giraph/benchmark/RandomMessageBenchmark.java
*
/incubator/giraph/trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java
*
/incubator/giraph/trunk/src/main/java/org/apache/giraph/examples/LongSumAggregator.java
*
/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java
*
/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/WorkerContext.java
* /incubator/giraph/trunk/src/main/java/org/apache/giraph/utils/MemoryUtils.java
> Save half of maximum memory used from messaging
> -----------------------------------------------
>
> Key: GIRAPH-104
> URL: https://issues.apache.org/jira/browse/GIRAPH-104
> Project: Giraph
> Issue Type: Improvement
> Reporter: Avery Ching
> Assignee: Avery Ching
> Priority: Critical
> Attachments: GIRAPH-104.diff
>
>
> Currently, the amount of memory that Giraph uses for messaging is huge. This
> JIRA will reduce the messaging memory by half and provide periodic updates of
> memory for debugging. Details are below:
> Refactored RandomMessageBenchmark to an internal vertex class. Added
> aggregators to RandomMessagesBenchmark to track bytes, messages, and time for
> the messaging. Adjusted the postSuperstep() to be called after the flush()
> for more accurate timings.
> Added periodic minute updates for message flushing (which can take a while,
> especially on the memory benchmark). This helps to see how progress is going
> and gives an ETA.
> Memory optimizations include:
> - Clear the message list after computation
> - Free vertex messages on the source as the flush is going on
> - TreeMap -> HashMap for VertexMutations
> - Sizing the ArrayList properly in transientInMessages
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
