[
https://issues.apache.org/jira/browse/HADOOP-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12635652#action_12635652
]
Hemanth Yamijala commented on HADOOP-4035:
------------------------------------------
The consensus on the implementation is as follows:
- Scheduler will assign tasks to a TT only if the amount of free memory is
greater than the task's requirements.
- If the memory requirements don't match, we do NOT move to the next job.
- We are not mapping tasks to slots at this point.
We understand point 3 means that the system is less fair than it should
(because users of high RAM jobs can cause more slots to go free than other
jobs), but in the interest of keeping things simple, we will follow this
approach for Hadoop 0.19, and take it up in the future.
Note that if HADOOP-4306 is addressed, we will be handling scheduling w.r.to
disk and memory uniformly.
> Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory
> requirements and task trackers free memory
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-4035
> URL: https://issues.apache.org/jira/browse/HADOOP-4035
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/capacity-sched
> Affects Versions: 0.19.0
> Reporter: Hemanth Yamijala
> Assignee: Vinod K V
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: 4035.1.patch, HADOOP-4035-20080918.1.txt
>
>
> HADOOP-3759 introduced configuration variables that can be used to specify
> memory requirements for jobs, and also modified the tasktrackers to report
> their free memory. The capacity scheduler in HADOOP-3445 should schedule
> tasks based on these parameters. A task that is scheduled on a TT that uses
> more than the default amount of memory per slot can be viewed as effectively
> using more than one slot, as it would decrease the amount of free memory on
> the TT by more than the default amount while it runs. The scheduler should
> make the used capacity account for this additional usage while enforcing
> limits, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
