[
https://issues.apache.org/jira/browse/HADOOP-4305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644217#action_12644217
]
Devaraj Das commented on HADOOP-4305:
-------------------------------------
Runping, I think taking the average blacklist count on a per tracker basis, and
penalizing only those TTs way above the average should help even in this
scenario. So for example, if a TT is really faulty, it's blacklist-count should
be way above the average number of blacklist-count per tracker, and this would
be penalized. The other case is where only certain tasks fail due to resource
limitations and the TT gets blacklisted for none of its fault, but IMO in a
practical setup, this problem would affect many other TTs as well, and hence
the average blacklist-count would be a bit higher... Makes sense?
> repeatedly blacklisted tasktrackers should get declared dead
> ------------------------------------------------------------
>
> Key: HADOOP-4305
> URL: https://issues.apache.org/jira/browse/HADOOP-4305
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Christian Kunz
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.20.0
>
>
> When running a batch of jobs it often happens that the same tasktrackers are
> blacklisted again and again. This can slow job execution considerably, in
> particular, when tasks fail because of timeout.
> It would make sense to no longer assign any tasks to such tasktrackers and to
> declare them dead.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
