[
https://issues.apache.org/jira/browse/HAMA-241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ChiaHung Lin updated HAMA-241:
------------------------------
Status: Patch Available (was: Open)
The file hama-241.patch is for jira issue hama-241. The patch adds features to
materialize task data to hdfs and clean those data once tasks are finished.
Also several things relates to this issue, therefore, I would like to get this
patch reviewed first and expect more refinement before the patch is accepted.
- BSPPeer
1. Is the purpose of BSPPeer to perform a task? From what I discover so far, it
seems to me the BSPPeer is used to perform assigned tasks; and in the future
there may have multiple BSPPeers (from wiki Hama architecture) concurrently
computing a task. But I do not find any way to obtain task information, which
is required to materialize data, from BSPPeer, so at the moment a Task object
is passed to BSPPeer for reference.
2. Location of materialize data. Currently data is materialized to hdfs. This
considers the issue of rescheduling a task in the future, preventing from
problem e.g. media failure which causes the materialized data lost.
- Zookeeper
BSPPeer contains an instance of ZooKeeper, but GroomServer does not.
Temporarily solution is to create another zk instance, but in the future the
cross cutting issue may need to be addressed?
I appreciate any suggestion.
> send method in BSPPeer.java should store messages to local storage
> temporarily.
> --------------------------------------------------------------------------------
>
> Key: HAMA-241
> URL: https://issues.apache.org/jira/browse/HAMA-241
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.2.1
> Reporter: Hyunsik Choi
> Assignee: Edward J. Yoon
> Fix For: 0.2.1
>
>
> Because of data size and fault tolerance, each BSP peer should store messages
> into its local storage at each super step.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
