Prabhu Joseph created YARN-10139:
------------------------------------
Summary: ValidateAndGetSchedulerConfiguration API fails when
cluster max allocation > default 8GB
Key: YARN-10139
URL: https://issues.apache.org/jira/browse/YARN-10139
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 3.3.0
Reporter: Prabhu Joseph
Assignee: Prabhu Joseph
ValidateAndGetSchedulerConfiguration fails when the cluster max allocation
(yarn.scheduler.maximum-allocation-mb) is set to resource (eg: 16GB) > default
8GB in yarn-site.xml.
As part of validation API, there are two configuration used -
CapacitySchedulerConfiguration and Configuration (yarn-site.xml). When
CapacityScheduler is initialized with CapacitySchedulerConfiguration, as part
of queues initialization, it checks the queue maximum allocation which is not
present and so checks cluster max allocation which is not present (it is
present only in YarnConfiguration) and defaults to 8GB. This will fail as queue
max allocation 8GB is decreased from previous 16GB.
{code}
2020-02-14 07:38:46,087 WARN
org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices:
CapacityScheduler configuration validation failed:java.io.IOException: Failed
to re-init queues : Trying to reinitialize root.default.c1.c3 the maximum
allocation size can not be decreased! Current setting: <memory:164860,
vCores:88>, trying to set it to: <memory:8192, vCores:4>
{code}
CapacityScheduler initialize code reads a yarn config from
CapacitySchedulerConfiguration causing the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
