[
https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262660#comment-17262660
]
zhuqi edited comment on YARN-10506 at 1/11/21, 2:11 PM:
--------------------------------------------------------
[~wangda] [~gandras]
Add an updated YARN-10506-007-10504-010.patch for fix all test in
TestCapacitySchedulerNewQueueAutoCreation.
The logic is :
In ParentQueue's addDynamicChildQueue
{code:java}
// New method to add child queue
public CSQueue addDynamicChildQueue(String childQueuePath, boolean isLeaf)
throws SchedulerDynamicEditException {
writeLock.lock();
try {
// Changed here
boolean weightsAreUsed = false;
try {
if (isLeaf && childQueues.isEmpty()) {
weightsAreUsed = true;
} else {
weightsAreUsed = getCapacityConfigurationTypeForQueues(childQueues)
== QueueCapacityType.WEIGHT;
}
} catch (IOException e) {
LOG.warn("Caught Exception during auto queue creation", e);
}
...
}{code}
Reason:
When add a dynamic leaf queue, but we still not update the weight it, when the
childQueues is empty, the original getCapacityConfigurationTypeForQueues will
return percentage, we should handle this case to weightsAreUsed = true. When
the childQueues is not empty, we just can use
getCapacityConfigurationTypeForQueues for check. And then, the new created leaf
will update weight.
Any other test fail, may be related queueplacement.
Thanks.
was (Author: zhuqi):
[~wangda] [~gandras]
Add an updated YARN-10506-007-10504-010.patch for fix all test in
TestCapacitySchedulerNewQueueAutoCreation.
The logic is :
In ParentQueue's addDynamicChildQueue
{code:java}
// New method to add child queue
public CSQueue addDynamicChildQueue(String childQueuePath, boolean isLeaf)
throws SchedulerDynamicEditException {
writeLock.lock();
try {
// Changed here
boolean weightsAreUsed = false;
try {
if (isLeaf && childQueues.isEmpty()) {
weightsAreUsed = true;
} else {
weightsAreUsed = getCapacityConfigurationTypeForQueues(childQueues)
== QueueCapacityType.WEIGHT;
}
} catch (IOException e) {
LOG.warn("Caught Exception during auto queue creation", e);
}
...
}{code}
Reason:
When add a dynamic leaf queue, but we still not update the weight it, when the
childQueues is empty, the original getCapacityConfigurationTypeForQueues will
return percentage, we should handle this case to weightsAreUsed = true. When
the childQueues is not empty, we just can use
getCapacityConfigurationTypeForQueues for check.
Any other test fail, may be related queueplacement.
Thanks.
> Update queue creation logic to use weight mode and allow the flexible
> static/dynamic creation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-10506
> URL: https://issues.apache.org/jira/browse/YARN-10506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Benjamin Teke
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10506-006-10504-010.patch,
> YARN-10506-007-10504-010.patch, YARN-10506.001.patch, YARN-10506.002.patch,
> YARN-10506.003.patch, YARN-10506.004.patch, YARN-10506.005.patch,
> YARN-10506.006-combined.patch, YARN-10506.006.patch
>
>
> The queue creation logic should be updated to use weight mode and support the
> flexible creation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
