[
https://issues.apache.org/jira/browse/MAPREDUCE-279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997599#comment-12997599
]
Jeff Hammerbacher commented on MAPREDUCE-279:
---------------------------------------------
Hey Arun,
As long as that evolution is happening in a branch, that seems totally
reasonable to me. When it comes time to migrate the code into trunk, I hope for
the same end state as Matei: I think the resource management system should be a
separate project from MapReduce so that each system can evolve and release
separately. When we have more clients than just MapReduce for the resource
manager, we'll want those new clients to evolve as separate projects rather
than all living under the Apache Hadoop umbrella. Now seems like an excellent
time to facilitate that end state.
More specifically, in an ideal world, we'd have four separate projects here:
Common (probably folded into Guava or Apache Commons), HDFS, Yahoo! Cluster
Manager (Resource Manager + Node Manager), and MapReduce (the ApplicationMaster
for MapReduce, I guess). Then, if someone wanted to write Pregel to run against
the Cluster Manager, they could implement their own ApplicationMaster in a
separate project. Similarly, if someone wanted to run MapReduce against a
different cluster manager, that would be simple. More practically, we have the
opportunity to get the Cluster Manager project started up as a separate ASF
project once it has gestated in a branch here for a bit. Are there any
technical barriers to making that happen?
I'm a huge fan of this work, and having watched a number of ASF projects evolve
over the past several years, I suspect that a small, focused project dedicated
to cluster resource management will have the best chance of moving quickly.
Thanks,
Jeff
> Map-Reduce 2.0
> --------------
>
> Key: MAPREDUCE-279
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-279
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker, tasktracker
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 0.23.0
>
>
> Re-factor MapReduce into a generic resource scheduler and a per-job,
> user-defined component that manages the application execution.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
