[ 
https://issues.apache.org/jira/browse/YARN-914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15062997#comment-15062997
 ] 

Daniel Zhi commented on YARN-914:
---------------------------------

Maybe you are already aware of this: EMR team has implemented graceful 
decommission in recent AMIs (for example, AMI 3.10.0 or 4.2.0). In these new 
AMIs, when you re-size down the cluster, the control logic will select best 
candidates and graceful decommission them instead of terminate them right away 
as before. You can move on to use AMI 3.10.0 (which is hadoop 2.6.0).

> (Umbrella) Support graceful decommission of nodemanager
> -------------------------------------------------------
>
>                 Key: YARN-914
>                 URL: https://issues.apache.org/jira/browse/YARN-914
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: graceful
>    Affects Versions: 2.0.4-alpha
>            Reporter: Luke Lu
>            Assignee: Junping Du
>         Attachments: Gracefully Decommission of NodeManager (v1).pdf, 
> Gracefully Decommission of NodeManager (v2).pdf, 
> GracefullyDecommissionofNodeManagerv3.pdf
>
>
> When NMs are decommissioned for non-fault reasons (capacity change etc.), 
> it's desirable to minimize the impact to running applications.
> Currently if a NM is decommissioned, all running containers on the NM need to 
> be rescheduled on other NMs. Further more, for finished map tasks, if their 
> map output are not fetched by the reducers of the job, these map tasks will 
> need to be rerun as well.
> We propose to introduce a mechanism to optionally gracefully decommission a 
> node manager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to