[
https://issues.apache.org/jira/browse/HAMA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Edward J. Yoon resolved HAMA-554.
---------------------------------
Resolution: Won't Fix
I don't think this is good idea. I'd rather use local fs and scp than hdfs.
> Implement a new message manager which writes messages to HDFS rather than
> sending them over RPC.
> ------------------------------------------------------------------------------------------------
>
> Key: HAMA-554
> URL: https://issues.apache.org/jira/browse/HAMA-554
> Project: Hama
> Issue Type: New Feature
> Components: bsp core, messaging
> Reporter: Apurv Verma
>
> The current message managers send all messages over RPC. In this issue we
> would like to create a DFSMessageManager which writes all messages to HDFS
> and store its PATH. Later in the sync() phase these PATHs may be send to
> other peers which can then read the messages from the HDFS.
> Pros:
> This can handle the case when the number of messages sent in a single
> superstep are too many to be all contained in memory. HAMA-521 already
> provides the implementation of Disk based queue. We can abstract this
> DiskQueue in the util section and use it here.
> Cons:
> Reading data from HDFS might be slow.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
