[
https://issues.apache.org/jira/browse/HAMA-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736696#comment-13736696
]
Anastasis Andronidis commented on HAMA-794:
-------------------------------------------
This issue was also bugging me. Just some thoughts:
1) if we can make something like peer.sync_without_deleting_queue(); So we
don't delete the messages that we didn't read and append new messages later on.
2) if we can implement multiple queues into peers. So we read and write to
different destinations in each superstep
3) Copy the messages from the queue efficiently (e.g. if the queue is in a
file, copy the file, or if it is in memory don't release the pointer) and
device a mechanism inside the graph module that will replay the messages to the
vertices.
> When aggregator is enabled, resending message in each message receiving round
> impacts the iteration performance.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: HAMA-794
> URL: https://issues.apache.org/jira/browse/HAMA-794
> Project: Hama
> Issue Type: Improvement
> Components: graph
> Reporter: MaoYuan Xian
>
> When aggregator is enabled, in each message receiving round, all received
> message have been resent to put them behind the aggregate message.
> doAggregationUpdates method of GraphJobRunner Class:
> {code}
> if (aggregationRunner.isEnabled() && iteration > 1) {
> // in case we need to sync, we need to replay the messages that already
> // are added to the queue. This prevents loosing messages when using
> // aggregators.
> if (firstVertexMessage != null) {
> peer.send(peer.getPeerName(), firstVertexMessage);
> }
> GraphJobMessage msg = null;
> while ((msg = peer.getCurrentMessage()) != null) {
> peer.send(peer.getPeerName(), msg);
> }
> // now sync
> peer.sync();
> ...
> {code}
> Should we do some improvement here? Record the original receiveQueue before
> the sync, and after sync operation, read the aggregator updated value
> firstly, then restore the original receiveQueue such that avoiding the
> message resending operations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
