[
https://issues.apache.org/jira/browse/MAPREDUCE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158591#comment-14158591
]
Jason Lowe commented on MAPREDUCE-6119:
---------------------------------------
After MAPREDUCE-3921 the AM will re-run map tasks that ran on unusable nodes,
but we've found this to be problematic in practice for our clusters. There are
often cases where a node becomes unusable but is still able to shuffle (e.g.:
the UNHEALTHY state), and re-running map tasks on those nodes is actually
harmful to the job. It can also lead to odd job states such as cases where all
reducers have passed the shuffle phase or even have completed but map tasks
continue to run, see MAPREDUCE-5817.
These are all things that may eventually be fixed, but until then we'd like the
ability to turn off the AM reacting to unusable nodes to restore the behavior
from 0.23.
> Ability to disable node update processing in MR AM
> --------------------------------------------------
>
> Key: MAPREDUCE-6119
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6119
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am
> Affects Versions: 2.5.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
