[
https://issues.apache.org/jira/browse/YARN-8292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16476304#comment-16476304
]
Wangda Tan commented on YARN-8292:
----------------------------------
I propose to change the logic to following:
If the preemption can make a positive contribution to to-obtain resource, we
will do that.
Copy part of the code comments added to the patch:
{code}
// When we want to do more aggressive preemption, we will do preemption:
// (when following conditions are all satisified
// - After preempt the container, the to-obtain should be either > 0
// OR any major resource equals to 0.
// - The preempt of the container makes positive contribution to the
// to-obtain resource. Positive contribution means any positive
// resource type decreases.
//
// This is example of positive contribution:
// * before: <30, 10, 5>, after <20, 10, -10>
// But this not positive contribution:
// * before: <30, 10, 0>, after <30, 10, -15>
{code}
And the attached patch doesn't change behavior of existing intra-queue
preemption (I called it conservative-DRF).
Please review the patch and let me know your thoughts.
[~sunilg], [~eepayne]
> Fix the dominant resource preemption cannot happen when some of the resource
> vector becomes negative
> ----------------------------------------------------------------------------------------------------
>
> Key: YARN-8292
> URL: https://issues.apache.org/jira/browse/YARN-8292
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Sumana Sathish
> Assignee: Wangda Tan
> Priority: Critical
>
>
> This is an example of the problem: (Same if we have more than 2 resources)
>
> Let's say we have 3 queues A/B/C. All containers with equal size <2,3>
>
> ||Queue||Guaranteed||Used ||Pending||
> |A|<20, 10>|<20,30>| |
> |B|<20, 10>|0|0|
> |C|<20, 10>|0|<20, 30>|
> | | | | |
>
> Under current logic, A's calculated to-preempt (how much resource other queue
> can preempt) will be <0, 20>. The preemption will not happen. However, under
> the context of DRC, queue A is using more resource than guaranteed, so queue
> C will be starved
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
