[
https://issues.apache.org/jira/browse/YARN-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Susheel Gupta reassigned YARN-11682:
------------------------------------
Assignee: Susheel Gupta
> Legacy auto created queue in absolute mode has zero capacity after creation
> during app recovery
> -----------------------------------------------------------------------------------------------
>
> Key: YARN-11682
> URL: https://issues.apache.org/jira/browse/YARN-11682
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Brian Goerlitz
> Assignee: Susheel Gupta
> Priority: Major
>
> During recovery of a running app in a legacy auto created queue configured in
> absolute mode the configured min resources will be set to zero as
> NodeManagers have not registered yet (clusterResource is zero).
> {code:java}
> GuaranteedOrZeroCapacityOverTimePolicy.getInitialLeafQueueConfiguration(AbstractAutoCreatedLeafQueue
> leafQueue)
> ...
> if (availableCapacity >= leafQueueTemplateCapacities
> .getAbsoluteCapacity(nodeLabel)) {
> updateCapacityFromTemplate(capacities, nodeLabel);
> activate(leafQueue, nodeLabel);
> } else{
> updateToZeroCapacity(capacities, nodeLabel, leafQueue);
> }
> GuaranteedOrZeroCapacityOverTimePolicy.updateToZeroCapacity(QueueCapacities
> capacities, String nodeLabel, AbstractLeafQueue leafQueue)
> private void updateToZeroCapacity(QueueCapacities capacities,
> String nodeLabel, AbstractLeafQueue leafQueue) {
> capacities.setCapacity(nodeLabel, 0.0f);
> capacities.setMaximumCapacity(nodeLabel,
> leafQueueTemplateCapacities.getMaximumCapacity(nodeLabel));
> leafQueue.getQueueResourceQuotas().
> setConfiguredMinResource(nodeLabel, Resource.newInstance(0, 0));
> }
> {code}
> When a NodeManager is registered,
> {{AbstractCSQueue.updateEffectiveResources(Resource clusterResource)}} is
> called, but specifically absolute mode queues are updated using the
> configured min resource, which will now be zero.
> {code:java}
> AbstractCSQueue.updateEffectiveResources(Resource clusterResource)
> ...
> if (getCapacityConfigType().equals(
> CapacityConfigType.ABSOLUTE_RESOURCE)) {
> newEffectiveMinResource = createNormalizedMinResource(
>
> usageTracker.getQueueResourceQuotas().getConfiguredMinResource(label),
> ((AbstractParentQueue) parent).getEffectiveMinRatio(label));
> ...
> usageTracker.getQueueResourceQuotas().setEffectiveMinResource(label,
> newEffectiveMinResource);
> {code}
> Reinitializing the queue via a config change will correctly recalculate the
> capacity based on current clusterResource.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
