[
https://issues.apache.org/jira/browse/YARN-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10257:
--------------------------------
Description:
Two issues have been discovered during fs2cs testing:
1. The conversion of allocation increment properties are not needed:
{{yarn.scheduler.increment-allocation-mb}}
{{yarn.scheduler.increment-allocation-vcores}}
{{yarn.resource-types.memory-mb.increment-allocation}}
{{yarn.resource-types.vcores.increment-allocation}}
2. The following piece of code is incorrect - the default scheduling policy can
be different from DRF, which is a problem if DRF is used everywhere else:
{code}
private boolean isDrfUsed(FairScheduler fs) {
FSQueue rootQueue = fs.getQueueManager().getRootQueue();
AllocationConfiguration allocConf = fs.getAllocationConfiguration();
String defaultPolicy = allocConf.getDefaultSchedulingPolicy().getName();
if (DominantResourceFairnessPolicy.NAME.equals(defaultPolicy)) {
return true;
} else {
return isDrfUsedOnQueueLevel(rootQueue);
}
}
{code}
was:
Two issues have been discovered during fs2cs testing:
1. The value of two properties are not checked:
{{yarn.scheduler.increment-allocation-mb}}
{{yarn.scheduler.increment-allocation-vcores}}
Although these two are marked as deprecated, they're still in use and must be
handled.
2. The following piece of code is incorrect - the default scheduling policy can
be different from DRF, which is a problem if DRF is used everywhere else:
{code}
private boolean isDrfUsed(FairScheduler fs) {
FSQueue rootQueue = fs.getQueueManager().getRootQueue();
AllocationConfiguration allocConf = fs.getAllocationConfiguration();
String defaultPolicy = allocConf.getDefaultSchedulingPolicy().getName();
if (DominantResourceFairnessPolicy.NAME.equals(defaultPolicy)) {
return true;
} else {
return isDrfUsedOnQueueLevel(rootQueue);
}
}
{code}
> FS-CS converter: skip increment properties for mem/vcores and fix DRF check
> ---------------------------------------------------------------------------
>
> Key: YARN-10257
> URL: https://issues.apache.org/jira/browse/YARN-10257
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
>
> Two issues have been discovered during fs2cs testing:
> 1. The conversion of allocation increment properties are not needed:
> {{yarn.scheduler.increment-allocation-mb}}
> {{yarn.scheduler.increment-allocation-vcores}}
> {{yarn.resource-types.memory-mb.increment-allocation}}
> {{yarn.resource-types.vcores.increment-allocation}}
> 2. The following piece of code is incorrect - the default scheduling policy
> can be different from DRF, which is a problem if DRF is used everywhere else:
> {code}
> private boolean isDrfUsed(FairScheduler fs) {
> FSQueue rootQueue = fs.getQueueManager().getRootQueue();
> AllocationConfiguration allocConf = fs.getAllocationConfiguration();
> String defaultPolicy = allocConf.getDefaultSchedulingPolicy().getName();
> if (DominantResourceFairnessPolicy.NAME.equals(defaultPolicy)) {
> return true;
> } else {
> return isDrfUsedOnQueueLevel(rootQueue);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
