Eric Payne created YARN-10033:
---------------------------------
Summary: TestProportionalCapacityPreemptionPolicy not initializing
vcores for effective max resources
Key: YARN-10033
URL: https://issues.apache.org/jira/browse/YARN-10033
Project: Hadoop YARN
Issue Type: Improvement
Components: capacity scheduler, test
Affects Versions: 3.3.0
Reporter: Eric Payne
TestProportionalCapacityPreemptionPolicy#testPreemptionWithVCoreResource is
preempting more containers than would happen on a real cluster.
This is because the process for mocking CS queues in
{{TestProportionalCapacityPreemptionPolicy}} fails to take into consideration
vcores when mocking effective max resources.
This causes miscalculations for how many vcores to preempt when the DRF is
being used in the test:
{code:title=TempQueuePerPartition#offer}
Resource absMaxCapIdealAssignedDelta = Resources.componentwiseMax(
Resources.subtract(getMax(), idealAssigned),
Resource.newInstance(0, 0));
{code}
In the above code, the preemption policy is offering resources to an
underserved queue. {{getMax()}} will use the effective max resource if it
exists. Since this test is mocking effective max resources, it will return that
value. However, since the mock doesn't include vcores, the test treats memory
as the dominant resource and awards too many preempted containers to the
underserved queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
