This is an automated email from the ASF dual-hosted git repository.
snemeth pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/hadoop.git
The following commit(s) were added to refs/heads/trunk by this push:
new 5446308 YARN-9326. Fair Scheduler configuration defaults are not
documented in case of min and maxResources. Contributed by Adam Antal
5446308 is described below
commit 5446308360f57cb98c54c416231788ba9ae332f8
Author: Szilard Nemeth <[email protected]>
AuthorDate: Mon Jul 15 13:28:01 2019 +0200
YARN-9326. Fair Scheduler configuration defaults are not documented in case
of min and maxResources. Contributed by Adam Antal
---
.../hadoop-yarn-site/src/site/markdown/FairScheduler.md | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git
a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
index 5f9e779..991796a 100644
---
a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
+++
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
@@ -86,13 +86,19 @@ The allocation file must be in XML format. The format
contains five types of ele
* **Queue elements**: which represent queues. Queue elements can take an
optional attribute 'type', which when set to 'parent' makes it a parent queue.
This is useful when we want to create a parent queue without configuring any
leaf queues. Each queue element may contain the following properties:
- * **minResources**: minimum resources the queue is entitled to, in the
form of "X mb, Y vcores" or "vcores=X, memory-mb=Y". The latter form is
required when specifying resources other than memory and CPU. For the
single-resource fairness policy, the vcores value is ignored. If a queue's
minimum share is not satisfied, it will be offered available resources before
any other queue under the same parent. Under the single-resource fairness
policy, a queue is considered unsatisfied if its [...]
+ * **minResources**: minimum resources the queue is entitled to. For the
single-resource fairness policy, only the memory is used, other resources are
ignored. If a queue's minimum share is not satisfied, it will be offered
available resources before any other queue under the same parent. Under the
single-resource fairness policy, a queue is considered unsatisfied if its
memory usage is below its minimum memory share. Under dominant resource
fairness, a queue is considered unsatisfied [...]
- * **maxResources**: maximum resources a queue will allocated, expressed in
the form of "X%", "X% cpu, Y% memory", "X mb, Y vcores", or "vcores=X,
memory-mb=Y". The last form is required when specifying resources other than
memory and CPU. In the last form, X and Y can either be a percentage or an
integer resource value without units. In the latter case the units will be
inferred from the default units configured for that resource. A queue will not
be assigned a container that would p [...]
+ * **maxResources**: maximum resources a queue can be allocated. A queue
will not be assigned a container that would put its aggregate usage over this
limit. This limit is enforced recursively, the queue will not be assigned a
container if that assignment would put the queue or its parent(s) over the
maximum resources.
- * **maxContainerAllocation**: maximum resources a queue can allocate for a
single container, expressed in the form of "X mb, Y vcores" or "vcores=X,
memory-mb=Y". The latter form is required when specifying resources other than
memory and CPU. If the property is not set it's value is inherited from a
parent queue. It's default value is **yarn.scheduler.maximum-allocation-mb**.
Cannot be higher than **maxResources**. This property is invalid for root queue.
+ * **maxContainerAllocation**: maximum resources a queue can allocate for a
single container. If the property is not set it's value is inherited from a
parent queue. The default values are **yarn.scheduler.maximum-allocation-mb**
and **yarn.scheduler.maximum-allocation-vcores**. Cannot be higher than
**maxResources**. This property is invalid for root queue.
- * **maxChildResources**: maximum resources an ad hoc child queue will
allocated, expressed in the form of "X%", "X% cpu, Y% memory", "X mb, Y
vcores", or "vcores=X, memory-mb=Y". The last form is required when specifying
resources other than memory and CPU. In the last form, X and Y can either be a
percentage or an integer resource value without units. In the latter case the
units will be inferred from the default units configured for that resource. An
ad hoc child queue will not be [...]
+ * **maxChildResources**: maximum resources an ad hoc child queue can be
allocated. A child queue limit is enforced recursively and thus will not be
assigned a container if that assignment would put the child queue or its
parent(s) over the maximum resources.
+
+ * For minResources, maxResources, maxContainerAllocation and
maxChildResources properties one can give the parameter in one of the following
formats:
+
+ * *Old format*: "X mb, Y vcores", "X% cpu, Y% memory", "X%". When
not a single percentage is provided, configuring both memory and cpu is
mandatory, while other resource types are ignored, and set to zero.
+
+ * *New format (recommended)*: "vcores=X, memory-mb=Y" or
"vcores=X%, memory-mb=Y%". As seen, in this format one can give either a
percentage or an integer resource value without units. In the latter case the
units will be inferred from the default units configured for that resource.
This format is required when resources other than memory and CPU are specified.
Any unspecified resource will be set to 0 in case of minResources, or to the
maximum for that resource in case of ma [...]
* **maxRunningApps**: limit the number of apps from the queue to run at
once
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
