[
https://issues.apache.org/jira/browse/YARN-11000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17740093#comment-17740093
]
ASF GitHub Bot commented on YARN-11000:
---------------------------------------
tomicooler commented on code in PR #5783:
URL: https://github.com/apache/hadoop/pull/5783#discussion_r1252757919
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java:
##########
@@ -422,10 +458,20 @@ protected void setDynamicQueueProperties() {
.getConfiguredNodeLabelsForAllQueues()
.getLabelsByQueue(parentTemplate);
- if (parentNodeLabels != null && parentNodeLabels.size() > 1) {
- queueContext.getQueueManager()
- .getConfiguredNodeLabelsForAllQueues()
- .setLabelsByQueue(getQueuePath(), new HashSet<>(parentNodeLabels));
+ if (parentNodeLabels != null) {
+ if (parentNodeLabels.size() > 1) {
+ queueContext.getQueueManager().getConfiguredNodeLabelsForAllQueues()
+ .setLabelsByQueue(getQueuePath(), new
HashSet<>(parentNodeLabels));
+ }
+ // Default to weight 1
+ for (String label : parentNodeLabels) {
+ float weightByLabel = queueContext.getConfiguration()
+ .getLabeledQueueWeight(queuePath, label);
+ if (weightByLabel == -1) {
+ queueContext.getConfiguration()
+ .setLabeledQueueWeight(queuePath.getFullPath(), label, 1);
+ }
+ }
Review Comment:
BTW this is done in legacy mode, in APQ:
```
// Normalize all dynamic queue queue's weight to 1 for all accessible
node
// labels, this is important because existing node labels could keep
// changing when new node added, or node label mapping changed. We need
// this to ensure auto created queue can access all labels.
for (String nodeLabel : queueCapacities.getExistingNodeLabels()) {
for (CSQueue queue : childQueues) {
// For dynamic queue, we will set weight to 1 every time, because
it
// is possible new labels added to the parent.
if (((AbstractCSQueue) queue).isDynamicQueue()) {
if (queue.getQueueCapacities().getWeight(nodeLabel) == -1f) {
queue.getQueueCapacities().setWeight(nodeLabel, 1f);
}
}
}
}
```
> Replace queue resource calculation logic in updateClusterResource
> -----------------------------------------------------------------
>
> Key: YARN-11000
> URL: https://issues.apache.org/jira/browse/YARN-11000
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Andras Gyori
> Assignee: Andras Gyori
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> YARN-10965 introduces a brand new queue calculation system. In order to
> simplify the review process, this issue replaces the current logic with the
> newly introduced one.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
