[
https://issues.apache.org/jira/browse/HADOOP-5289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675111#action_12675111
]
Runping Qi commented on HADOOP-5289:
------------------------------------
Another aspect of this issue.
Task failed on those 3 trackers because they have too little memory.
Hadoop scheduler correctly detected that fact, and correctly determined that
each of those trackers has the capacity of one slot.
Thus, the cluster has the total capacity of 53 closts (10 * 5 + 3 * 1).
However, the scheduler did not use that information when computing the maximum
number of tasks each tracker can take for the job, and thought each task
tracker should take 4 tasks, regardless of their actual capacity.
> Tasks are not scheduled even though task trackers have extra slots
> ------------------------------------------------------------------
>
> Key: HADOOP-5289
> URL: https://issues.apache.org/jira/browse/HADOOP-5289
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Runping Qi
>
> I ran a job with 51 reduce tasks on a cluster with 13 task trackers running
> Hadoop 0.19. Each task tracker has 5 reduce slots.
> Initially, each task tracker accepted 4 reduce tasks as expected. However, 3
> task trackers were put into blacklist because many tasks failed on them.
> However, those failed tasks stayed in pending state, not being scheduled to
> other task trackers, even though each of the other healthy tracker has one
> free slot.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
