Setup and cleanup tasks remain in UNASSIGNED state for a long time on
tasktrackers with long running high RAM tasks
-------------------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-771
URL: https://issues.apache.org/jira/browse/MAPREDUCE-771
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.1
Reporter: Hemanth Yamijala
When a high RAM job's task is scheduled on a tasktracker, the number of slots
it occupies will be more than one. If enough such tasks are scheduled, there
can come a situation when there are no more slots free on the node but the
number of tasks running is less than the number of slots. The jobtracker
currently schedules a setup or cleanup task based on how many tasks are running
on the system, rather than slots. As a result of this, it can schedule a setup
or cleanup task on a tasktracker without any free slots. If the high RAM job's
tasks are long running, this will significantly delay the running of the setup
or cleanup task, and thus the entire job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
