[
https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103601#comment-13103601
]
Vinod Kumar Vavilapalli commented on HAMA-431:
----------------------------------------------
ChiaHung,
bq. With polling, it seems that chances are the polling would not reach the
agreement (there could always have 1 process missing) in an unfortunate timing
case. Also, as the processes increase probably it would increase the loading
for master to deal with polling tasks.
Regarding the missing processes, which we call stragglers in mapreduce, isn't
the API such that there should be no progress till all the processes perform
the barrier sync?
Regarding the load, even MR AM which uses a Hadoop RPC server has similar
requirements, in the order of ten's of thousands of tasks. That amount of
scalability should be enough for Hama's case also. And like Thomas mentioned,
each BSPMaster is needed to serve the same job's BSPPeers, so that should help
too.
bq. In addition, my understanding is the integration with MRV2 would be just an
additional support so that MR job/ application can be submitted without
rewriting to use hama for computation.
It is not clear to me. But if you are talking of the ability to run the current
BSP jobs without rewriting them, then yes, we will support API level
compatibility.
> MapReduce NG integration
> ------------------------
>
> Key: HAMA-431
> URL: https://issues.apache.org/jira/browse/HAMA-431
> Project: Hama
> Issue Type: New Feature
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's
> nextGen application platform.
> Can be currently found in the 0.23 branch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
