[
https://issues.apache.org/jira/browse/YARN-10506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262660#comment-17262660
]
zhuqi edited comment on YARN-10506 at 1/11/21, 2:10 PM:
--------------------------------------------------------
[~wangda] [~gandras]
Add an updated YARN-10506-007-10504-010.patch for fix all test in
TestCapacitySchedulerNewQueueAutoCreation.
The logic is :
In ParentQueue's addDynamicChildQueue
{code:java}
// New method to add child queue
public CSQueue addDynamicChildQueue(String childQueuePath, boolean isLeaf)
throws SchedulerDynamicEditException {
writeLock.lock();
try {
// Changed here
boolean weightsAreUsed = false;
try {
if (isLeaf && childQueues.isEmpty()) {
weightsAreUsed = true;
} else {
weightsAreUsed = getCapacityConfigurationTypeForQueues(childQueues)
== QueueCapacityType.WEIGHT;
}
} catch (IOException e) {
LOG.warn("Caught Exception during auto queue creation", e);
}
...
}{code}
Reason:
When add a dynamic leaf queue, but we still not update the weight it, when the
childQueues is empty, the original getCapacityConfigurationTypeForQueues will
return percentage, we should handle this case to weightsAreUsed = true. When
the childQueues is not empty, we just can use
getCapacityConfigurationTypeForQueues for check.
Any other test fail, may be related queueplacement.
Thanks.
was (Author: zhuqi):
[~wangda] [~gandras]
Add an updated patch for fix all test in
TestCapacitySchedulerNewQueueAutoCreation.
In ParentQueue's addDynamicChildQueue
{code:java}
// New method to add child queue
public CSQueue addDynamicChildQueue(String childQueuePath, boolean isLeaf)
throws SchedulerDynamicEditException {
writeLock.lock();
try {
// Changed here
boolean weightsAreUsed = false;
try {
if (isLeaf && childQueues.isEmpty()) {
weightsAreUsed = true;
} else {
weightsAreUsed = getCapacityConfigurationTypeForQueues(childQueues)
== QueueCapacityType.WEIGHT;
}
} catch (IOException e) {
LOG.warn("Caught Exception during auto queue creation", e);
}
...
}{code}
Reason:
When add a dynamic leaf queue, but we still not update the weight it, also the
childQueues is empty, when original getCapacityConfigurationTypeForQueues will
return percentage, we should handle this case to weightsAreUsed = true, avoid
the error.
Any other test fail, may be related queueplacement.
Thanks.
> Update queue creation logic to use weight mode and allow the flexible
> static/dynamic creation
> ---------------------------------------------------------------------------------------------
>
> Key: YARN-10506
> URL: https://issues.apache.org/jira/browse/YARN-10506
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Benjamin Teke
> Assignee: Andras Gyori
> Priority: Major
> Attachments: YARN-10506-006-10504-010.patch,
> YARN-10506-007-10504-010.patch, YARN-10506.001.patch, YARN-10506.002.patch,
> YARN-10506.003.patch, YARN-10506.004.patch, YARN-10506.005.patch,
> YARN-10506.006-combined.patch, YARN-10506.006.patch
>
>
> The queue creation logic should be updated to use weight mode and support the
> flexible creation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
