[
https://issues.apache.org/jira/browse/YARN-4333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132002#comment-15132002
]
Tao Jie commented on YARN-4333:
-------------------------------
In updated patch，some improvements for preemption within queue：
1, When calculating resource deficit to preempt, we have {{resToPreempt}} which
is total resource to preempt among queues, and {{Map<String, Resource>
withinQueueResToPreempt}} which contains amount of resource to preempt within
queues.
2, In preemptResource, we should consider both {{resToPreempt}} and
{{withinQueueResToPreempt}}. And we will preempt resource due to
{{withinQueueResToPreempt}}(if any) before {{resToPreempt}}.
3, Once we preempt container due to preemption within queue, we will add
resouce to {{Map<String, Resource> resourceDeficitWithinQueues}}, which
maintains resource preempted due to within-queue preemption for each queue.
When scheduling resource, we will prefer queues those have resource deficit in
resourceDeficitWithinQueues.
> Fair scheduler should support preemption within queue
> -----------------------------------------------------
>
> Key: YARN-4333
> URL: https://issues.apache.org/jira/browse/YARN-4333
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Tao Jie
> Assignee: Tao Jie
> Attachments: YARN-4333.001.patch, YARN-4333.002.patch
>
>
> Now each app in fair scheduler is allocated its fairshare, however fairshare
> resource is not ensured even if fairSharePreemption is enabled.
> Consider:
> 1, When the cluster is idle, we submit app1 to queueA,which takes maxResource
> of queueA.
> 2, Then the cluster becomes busy, but app1 does not release any resource,
> queueA resource usage is over its fairshare
> 3, Then we submit app2(maybe with higher priority) to queueA. Now app2 has
> its own fairshare, but could not obtain any resource, since queueA is still
> over its fairshare and resource will not assign to queueA anymore. Also,
> preemption is not triggered in this case.
> So we should allow preemption within queue, when app is starved for fairshare.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
