[
https://issues.apache.org/jira/browse/YARN-10033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16999203#comment-16999203
]
Eric Payne commented on YARN-10033:
-----------------------------------
Thanks [~ebadger]!
> TestProportionalCapacityPreemptionPolicy not initializing vcores for
> effective max resources
> --------------------------------------------------------------------------------------------
>
> Key: YARN-10033
> URL: https://issues.apache.org/jira/browse/YARN-10033
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler, test
> Affects Versions: 3.3.0, 3.2.1, 3.1.3
> Reporter: Eric Payne
> Assignee: Eric Payne
> Priority: Major
> Fix For: 3.3.0, 3.2.2, 3.1.4
>
> Attachments: YARN-10033.001.patch, YARN-10033.002.patch,
> YARN-10033.003.patch
>
>
> TestProportionalCapacityPreemptionPolicy#testPreemptionWithVCoreResource is
> preempting more containers than would happen on a real cluster.
> This is because the process for mocking CS queues in
> {{TestProportionalCapacityPreemptionPolicy}} fails to take into consideration
> vcores when mocking effective max resources.
> This causes miscalculations for how many vcores to preempt when the DRF is
> being used in the test:
> {code:title=TempQueuePerPartition#offer}
> Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax(
> Resources.subtract(getMax(), idealAssigned),
> Resource.newInstance(0, 0));
> {code}
> In the above code, the preemption policy is offering resources to an
> underserved queue. {{getMax()}} will use the effective max resource if it
> exists. Since this test is mocking effective max resources, it will return
> that value. However, since the mock doesn't include vcores, the test treats
> memory as the dominant resource and awards too many preempted containers to
> the underserved queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
