[
https://issues.apache.org/jira/browse/YARN-10813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17363185#comment-17363185
]
Szilard Nemeth commented on YARN-10813:
---------------------------------------
Thanks [~gandras] for reporting this,
This is quite a trivial fix, but it's good that you have spotted it.
Some questions / observations:
1. How come our tests didn't catch this? Is it easy to add a unit test to cover
the fixed scenario if it isn't any?
2. I would have been baffled to realize if we don't have a common constant for
the queue name "root" anywhere. The thing is, we have many constants, just
search for "root" from package:
org/apache/hadoop/yarn/server/resourcemanager/scheduler
I know it's not strongly related to this, but could you please report a
follow-up to clean up those? I just don't want to increase the number of
occurrences of "root" in production code anymore.
Thanks.
> Root queue capacity is not set when using node labels
> -----------------------------------------------------
>
> Key: YARN-10813
> URL: https://issues.apache.org/jira/browse/YARN-10813
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Andras Gyori
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10813.001.patch
>
>
> CapacitySchedulerConfiguration#getNonLabeledQueueCapacity handles root in the
> following way:
> {code:java}
> if (absoluteResourceConfigured || configuredWeightAsCapacity(
> configuredCapacity)) {
> // Return capacity in percentage as 0 for non-root queues and 100 for
> // root.From AbstractCSQueue, absolute resource will be parsed and
> // updated. Once nodes are added/removed in cluster, capacity in
> // percentage will also be re-calculated.
> return queue.equals("root") ? 100.0f : 0f;
> }
> {code}
> CapacitySchedulerConfiguration#internalGetLabeledQueueCapacity on the other
> hand does not take root queue into consideration:
> {code:java}
> if (absoluteResourceConfigured || configuredWeightAsCapacity(
> configuredCapacity)) {
> // Return capacity in percentage as 0 for non-root queues and 100 for
> // root.From AbstractCSQueue, absolute resource, and weight will be
> parsed
> // and updated separately. Once nodes are added/removed in cluster,
> // capacity is percentage will also be re-calculated.
> return defaultValue;
> }
> float capacity = getFloat(capacityPropertyName, defaultValue);
> {code}
> Due to this, labeled root capacity is 0, which is not set in in
> AbstractCSQueue#derivedCapacityFromAbsoluteConfigurations, because root is
> never in Absolute mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
