Repository: hadoop
Updated Branches:
refs/heads/branch-2.8 1b5b85e56 -> 67ee30272
YARN-4826. Document configuration of ReservationSystem for CapacityScheduler.
(Subru Krishnan via wangda)
(cherry picked from commit fcb3fcd4c674d58e5f40b17eff7048e8ae45d151)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
(cherry picked from commit dac2021f870af674ab1a64e937c4f2a388dcc692)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/67ee3027
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/67ee3027
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/67ee3027
Branch: refs/heads/branch-2.8
Commit: 67ee3027238a9f306c0bc478222c22024dcd9b68
Parents: 1b5b85e
Author: Wangda Tan <[email protected]>
Authored: Fri Apr 8 16:21:04 2016 -0700
Committer: Wangda Tan <[email protected]>
Committed: Fri Apr 8 16:33:55 2016 -0700
----------------------------------------------------------------------
.../src/site/markdown/CapacityScheduler.md | 28 ++++++++++++++++++++
1 file changed, 28 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/67ee3027/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
----------------------------------------------------------------------
diff --git
a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
index 8c0b8c8..007842a 100644
---
a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
+++
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
@@ -24,6 +24,7 @@ Hadoop: Capacity Scheduler
* [Queue Properties](#Queue_Properties)
* [Setup for application priority](#Setup_for_application_priority.)
* [Capacity Scheduler container
preemption](#Capacity_Scheduler_container_preemption)
+ * [Configuring `ReservationSystem` with
`CapacityScheduler`](#Configuring_ReservationSystem_with_CapacityScheduler)
* [Other Properties](#Other_Properties)
* [Reviewing the configuration of the
CapacityScheduler](#Reviewing_the_configuration_of_the_CapacityScheduler)
* [Changing Queue Configuration](#Changing_Queue_Configuration)
@@ -241,6 +242,33 @@ The following configuration parameters can be configured
in yarn-site.xml to con
| `yarn.scheduler.capacity.root.<queue>.acl_list_reservations` | The ACL which
controls who can *list* reservations to the given queue. If the given
user/group has necessary ACLs on the given queue they can list all
applications. ACLs for this property *are not* inherited from the parent queue
if not specified. |
| `yarn.scheduler.capacity.root.<queue>.acl_submit_reservations` | The ACL
which controls who can *submit* reservations to the given queue. If the given
user/group has necessary ACLs on the given queue they can submit reservations.
ACLs for this property *are not* inherited from the parent queue if not
specified. |
+### Configuring `ReservationSystem` with `CapacityScheduler`
+
+ The `CapacityScheduler` supports the **ReservationSystem** which allows users
to reserve resources ahead of time. The application can request the reserved
resources at runtime by specifying the `reservationId` during submission. The
following configuration parameters can be configured in yarn-site.xml for
`ReservationSystem`.
+
+| Property | Description |
+|:---- |:---- |
+| `yarn.resourcemanager.reservation-system.enable` | *Mandatory* parameter: to
enable the `ReservationSystem` in the **ResourceManager**. Boolean value
expected. The default value is *false*, i.e. `ReservationSystem` is not enabled
by default. |
+| `yarn.resourcemanager.reservation-system.class` | *Optional* parameter: the
class name of the `ReservationSystem`. The default value is picked based on the
configured Scheduler, i.e. if `CapacityScheduler` is configured, then it is
`CapacityReservationSystem`. |
+| `yarn.resourcemanager.reservation-system.plan.follower` | *Optional*
parameter: the class name of the `PlanFollower` that runs on a timer, and
synchronizes the `CapacityScheduler` with the `Plan` and viceversa. The default
value is picked based on the configured Scheduler, i.e. if `CapacityScheduler`
is configured, then it is `CapacitySchedulerPlanFollower`. |
+| `yarn.resourcemanager.reservation-system.planfollower.time-step` |
*Optional* parameter: the frequency in milliseconds of the `PlanFollower`
timer. Long value expected. The default value is *1000*. |
+
+
+The `ReservationSystem` is integrated with the `CapacityScheduler` queue
hierachy and can be configured for any **LeafQueue** currently. The
`CapacityScheduler` supports the following parameters to tune the
`ReservationSystem`:
+
+| Property | Description |
+|:---- |:---- |
+| `yarn.scheduler.capacity.<queue-path>.reservable` | *Mandatory* parameter:
indicates to the `ReservationSystem` that the queue's resources is available
for users to reserve. Boolean value expected. The default value is *false*,
i.e. reservations are not enabled in *LeafQueues* by default. |
+| `yarn.scheduler.capacity.<queue-path>.reservation-agent` | *Optional*
parameter: the class name that will be used to determine the implementation of
the `ReservationAgent` which will attempt to place the user's reservation
request in the `Plan`. The default value is
*org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.AlignedPlannerWithGreedy*.
|
+| `yarn.scheduler.capacity.<queue-path>.reservation-move-on-expiry` |
*Optional* parameter to specify to the `ReservationSystem` whether the
applications should be moved or killed to the parent reservable queue
(configured above) when the associated reservation expires. Boolean value
expected. The default value is *true* indicating that the application will be
moved to the reservable queue. |
+| `yarn.scheduler.capacity.<queue-path>.show-reservations-as-queues` |
*Optional* parameter to show or hide the reservation queues in the Scheduler
UI. Boolean value expected. The default value is *false*, i.e. reservation
queues will be hidden. |
+| `yarn.scheduler.capacity.<queue-path>.reservation-policy` | *Optional*
parameter: the class name that will be used to determine the implementation of
the `SharingPolicy` which will validate if the new reservation doesn't violate
any invariants.. The default value is
*org.apache.hadoop.yarn.server.resourcemanager.reservation.CapacityOverTimePolicy*.
|
+| `yarn.scheduler.capacity.<queue-path>.reservation-window` | *Optional*
parameter representing the time in milliseconds for which the `SharingPolicy`
will validate if the constraints in the Plan are satisfied. Long value
expected. The default value is one day. |
+| `yarn.scheduler.capacity.<queue-path>.instantaneous-max-capacity` |
*Optional* parameter: maximum capacity at any time in percentage (%) as a float
that the `SharingPolicy` allows a single user to reserve. The default value is
1, i.e. 100%. |
+| `yarn.scheduler.capacity.<queue-path>.average-capacity` | *Optional*
parameter: the average allowed capacity which will aggregated over the
*ReservationWindow* in percentage (%) as a float that the `SharingPolicy`
allows a single user to reserve. The default value is 1, i.e. 100%. |
+| `yarn.scheduler.capacity.<queue-path>.reservation-planner` | *Optional*
parameter: the class name that will be used to determine the implementation of
the *Planner* which will be invoked if the `Plan` capacity fall below (due to
scheduled maintenance or node failuers) the user reserved resources. The
default value is
*org.apache.hadoop.yarn.server.resourcemanager.reservation.planning.SimpleCapacityReplanner*
which scans the `Plan` and greedily removes reservations in reversed order of
acceptance (LIFO) till the reserved resources are within the `Plan` capacity |
+| `yarn.scheduler.capacity.<queue-path>.reservation-enforcement-window` |
*Optional* parameter representing the time in milliseconds for which the
`Planner` will validate if the constraints in the Plan are satisfied. Long
value expected. The default value is one hour. |
+
###Other Properties
* Resource Calculator
