[
https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104430#comment-13104430
]
ChiaHung Lin commented on HAMA-431:
-----------------------------------
Vinod,
{quote}
Regarding the missing processes, which we call stragglers in mapreduce, isn't
the API such that there should be no progress till all the processes perform
the barrier sync?
{quote}
Yes, in that case there would have no progress. However, it differs from the
barrier sync with zookeeper in that there could always have different
stragglers do not poll each round due to networking loading, etc. For instance,
with time interval e.g. 1 secs each GroomServer polls to check if he can
proceed; unfortunately due to network congestion, master server always receives
parts of response (not response from all GroomServers). So the rate of barrier
sync with no progress probably could be higher than expected. Or we will have
master to help coordinate between stragglers, but this seems the tasks that
should be handled by zookeeper service. In addition, if it is going to have
multiple masters, to replicate the poll information should also be taken into
account.
I was just thinking some issues that maybe we need to consider beforehand if it
is decided to work toward this direction. Thanks Vinod, that inspires me a lot.
> MapReduce NG integration
> ------------------------
>
> Key: HAMA-431
> URL: https://issues.apache.org/jira/browse/HAMA-431
> Project: Hama
> Issue Type: New Feature
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's
> nextGen application platform.
> Can be currently found in the 0.23 branch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
