[
https://issues.apache.org/jira/browse/MAPREDUCE-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12780992#action_12780992
]
dhruba borthakur commented on MAPREDUCE-1044:
---------------------------------------------
The end goal is *really* to get isolation of jobs having different SLAs. One
way to do this is to use VMs and solaris has relatively good ones from what I
understand (not so in Linux).
Torque can be used, but it is the policies of what-to-move, when-to-move, etc
that will be important, rather than which daemon will move it. The daemon could
be something that we build (inside or outside the Hadoop framework) or it could
be a extension to Torque. This requires that the JT exposes certain APIs that
this external piece of software can use to build it policies. Maybe we can use
this JIRA to discuss what these policies could be and when they get triggered?
> Ability to automatically move machines from one MR compute cluster to another
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-1044
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1044
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: dhruba borthakur
> Assignee: Dmytro Molkov
> Attachments: DCD.pdf
>
>
> We have multiple map-reduce clusters that provide different service and
> support levels for their users. We have seen that utilization of hardware
> resources are not optimized if we have a static partition of existing
> hardware resources into these separate MR clusters. It would be nice to have
> a automatic way to move nodes from one MR cluster to another based on load
> characteristics and configured policies. This JIRA will discuss some of the
> ideas and possible implementations of those ideas.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
