[
https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000576#comment-13000576
]
Mahadev konar commented on MAPREDUCE-279:
-----------------------------------------
@Scott,
With respect your comments on ResourceManager/ZooKeeper/RPC:
We intend to take it slow with ZooKeeper, initially the intention is to put
just the allocations (what each job/application is allocated in ZooKeeper, this
is mainly for ResourceManager and Application Master restart). I am not really
in favor of using ZK notification for getting rid of RPC's. For the scale we
are talking abt, the "first to get the work will take it" approach will cause
herd affect and will definitely be a cause for concern. I think ZK can be used
much more than what we have proposed but itll be gradual process to see what
all we can offload to ZK.
I am pretty hesitant to put RPC load onto ZK and use it as a workload queue for
something like this.
> Map-Reduce 2.0
> --------------
>
> Key: MAPREDUCE-279
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-279
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker, tasktracker
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 0.23.0
>
>
> Re-factor MapReduce into a generic resource scheduler and a per-job,
> user-defined component that manages the application execution.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
