[
https://issues.apache.org/jira/browse/MAPREDUCE-771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12732998#action_12732998
]
Hemanth Yamijala commented on MAPREDUCE-771:
--------------------------------------------
In JobTracker.getSetupAndCleanupTasks(), before a task is assigned to a map
slot or a reduce slot, we count the number of running maps
(TaskTrackerStatus.countMapTasks) / reduces on the tracker from its status
instead of map / reduce slots. One simple solution could be to use
TaskTrackerStatus.countOccupiedMapSlots. Ideally we should also check if this
bug appears elsewhere in the code.
> Setup and cleanup tasks remain in UNASSIGNED state for a long time on
> tasktrackers with long running high RAM tasks
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-771
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-771
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.21.0
> Reporter: Hemanth Yamijala
> Fix For: 0.21.0
>
>
> When a high RAM job's task is scheduled on a tasktracker, the number of slots
> it occupies will be more than one. If enough such tasks are scheduled, there
> can come a situation when there are no more slots free on the node but the
> number of tasks running is less than the number of slots. The jobtracker
> currently schedules a setup or cleanup task based on how many tasks are
> running on the system, rather than slots. As a result of this, it can
> schedule a setup or cleanup task on a tasktracker without any free slots. If
> the high RAM job's tasks are long running, this will significantly delay the
> running of the setup or cleanup task, and thus the entire job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
