[
https://issues.apache.org/jira/browse/TEZ-2972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046088#comment-15046088
]
Bikas Saha commented on TEZ-2972:
---------------------------------
I was thinking that the config would go the to AMNode/AMNodeMap code that
handles the node update events. IMO, events should keep flowing as they do
today because other decisions may be taken based on these updates (e.g. new
machine came so plan with higher capacity etc. or machine went away - so remove
from existing attempt affinity). So we should probably use the new config in
the place that uses the unhealthy status to send kill events to the attempts
and just disable that part. Thoughts?
Also, I would be +1 to actually changing the default for this to disable the
feature. Normal, read error handling should cover most cases and such
preemptive rescheduling has probably not been tested out meaningfully.
> Ability for Tez AM to ignore node updates from YARN
> ---------------------------------------------------
>
> Key: TEZ-2972
> URL: https://issues.apache.org/jira/browse/TEZ-2972
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: TEZ-2972.001.patch
>
>
> This is similar to MAPREDUCE-6119. Sometimes reacting to a node update event
> can cause more harm than good. For example, an UNHEALTHY node may be able to
> shuffle just fine. Therefore obsoleting the output of tasks that ran on that
> node and re-running them simply adds more overhead to the job with no
> benefit. It would be nice to be able to configure Tez to ignore node update
> events if desired.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
