[
https://issues.apache.org/jira/browse/TEZ-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe reassigned TEZ-2972:
-------------------------------
Assignee: Jason Lowe
This can also be important on clusters where the UNHEALTHY state is used as
part of the node decommission process. Currently there is no graceful
decommission of nodes in YARN, so sometimes nodes are put in an UNHEALTHY state
intentionally. Those nodes won't receive any more containers, but they will
remain up and continue to respond to shuffle requests. Rerunning completed
tasks from those nodes would be a waste.
> Ability for Tez AM to ignore node updates from YARN
> ---------------------------------------------------
>
> Key: TEZ-2972
> URL: https://issues.apache.org/jira/browse/TEZ-2972
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Jason Lowe
> Assignee: Jason Lowe
>
> This is similar to MAPREDUCE-6119. Sometimes reacting to a node update event
> can cause more harm than good. For example, an UNHEALTHY node may be able to
> shuffle just fine. Therefore obsoleting the output of tasks that ran on that
> node and re-running them simply adds more overhead to the job with no
> benefit. It would be nice to be able to configure Tez to ignore node update
> events if desired.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
