[
https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478026#comment-16478026
]
Wangda Tan commented on YARN-8292:
----------------------------------
Thanks [~eepayne],
Let me revise the example a bit, I was trying to simplify it to avoid confusing
to people, I think the example is wrong. Here's one I was using in test:
{code}
// guaranteed, max, used, pending
"root(=[30:18:6 30:18:6 12:12:6 1:1:1]);" + //root
"-a(=[10:6:2 10:6:2 6:6:3 0:0:0]);" + // a
"-b(=[10:6:2 10:6:2 6:6:3 0:0:0]);" + // b
"-c(=[10:6:2 10:6:2 0:0:0 1:1:1])"; // c
{code}
There're 3 resource types. Total resource of the cluster is 30:18:6
For both of a/b, there're 3 containers running, each of container is 2:2:1
Prior to the attached patch. The preemption cannot happen, you can check
org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicyInterQueueWithDRF#test3ResourceTypesInterQueuePreemption
test for details.
> Fix the dominant resource preemption cannot happen when some of the resource
> vector becomes negative
> ----------------------------------------------------------------------------------------------------
>
> Key: YARN-8292
> URL: https://issues.apache.org/jira/browse/YARN-8292
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Sumana Sathish
> Assignee: Wangda Tan
> Priority: Critical
> Fix For: 3.2.0, 3.1.1
>
> Attachments: YARN-8292.001.patch
>
>
>
> This is an example of the problem: (Same if we have more than 2 resources)
>
> Let's say we have 3 queues A/B/C. All containers with equal size <2,3>
>
> ||Queue||Guaranteed||Used ||Pending||
> |A|<20, 10>|<20,30>| |
> |B|<20, 10>|0|0|
> |C|<20, 10>|0|<20, 30>|
> | | | | |
>
> Under current logic, A's calculated to-preempt (how much resource other queue
> can preempt) will be <0, 20>. The preemption will not happen. However, under
> the context of DRC, queue A is using more resource than guaranteed, so queue
> C will be starved
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
