hadoop git commit: YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty (cherry picked from commit a380643d2044a4974e379965f65066df2055d00
Repository: hadoop Updated Branches: refs/heads/branch-2 482504bef - 2690c7252 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty (cherry picked from commit a380643d2044a4974e379965f65066df2055d003) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/2690c725 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/2690c725 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/2690c725 Branch: refs/heads/branch-2 Commit: 2690c7252216f236bcc6e0a170a57ee52a2ffb4a Parents: 482504b Author: Jian He jia...@apache.org Authored: Tue Mar 10 10:54:08 2015 -0700 Committer: Jian He jia...@apache.org Committed: Tue Mar 10 11:16:51 2015 -0700 -- hadoop-yarn-project/CHANGES.txt | 3 +++ .../src/site/markdown/CapacityScheduler.md | 26 2 files changed, 29 insertions(+) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/2690c725/hadoop-yarn-project/CHANGES.txt -- diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index 52667f1..4167703 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -339,6 +339,9 @@ Release 2.7.0 - UNRELEASED YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. (Hitesh Shah via junping_du) +YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user +or group. (Gururaj Shetty via jianhe) + OPTIMIZATIONS YARN-2990. FairScheduler's delay-scheduling always waits for node-local and http://git-wip-us.apache.org/repos/asf/hadoop/blob/2690c725/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md index 3c32cdd..1cb963e 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md @@ -69,6 +69,8 @@ The `CapacityScheduler` supports the following features: * **Resource-based Scheduling** - Support for resource-intensive applications, where-in a application can optionally specify higher resource-requirements than the default, there-by accomodating applications with differing resource requirements. Currently, *memory* is the the resource requirement supported. +* **Queue Mapping based on User or Group** - This feature allows users to map a job to a specific queue based on the user or group. + Configuration - @@ -151,6 +153,30 @@ Configuration **Note:** An *ACL* is of the form *user1*, *user2spacegroup1*, *group2*. The special value of * implies *anyone*. The special value of *space* implies *no one*. The default is * for the root queue if not specified. + * Queue Mapping based on User or Group + + The `CapacityScheduler` supports the following parameters to configure the queue mapping based on user or group: + +| Property | Description | +|: |: | +| `yarn.scheduler.capacity.queue-mappings` | This configuration specifies the mapping of user or group to aspecific queue. You can map a single user or a list of users to queues. Syntax: `[u or g]:[name]:[queue_name][,next_mapping]*`. Here, *u or g* indicates whether the mapping is for a user or group. The value is *u* for user and *g* for group. *name* indicates the user name or group name. To specify the user who has submitted the application, %user can be used. *queue_name* indicates the queue name for which the application has to be mapped. To specify queue name same as user name, *%user* can be used. To specify queue name same as the name of the primary group for which the user belongs to, *%primary_group* can be used.| +| `yarn.scheduler.capacity.queue-mappings-override.enable` | This function is used to specify whether the user specified queues can be overridden. This is a Boolean value and the default value is *false*. | + +Example: + +``` + property + nameyarn.scheduler.capacity.queue-mappings/name + valueu:user1:queue1,g:group1:queue2,u:%user:%user,u:user2:%primary_group/value + description + Here, user1 is mapped to queue1, group1 is mapped to queue2, + maps users to queues with the same name as user, user2 is mapped + to queue name same as primary group respectively. The mappings will be + evaluated from left to right, and the first valid mapping will be used. + /description + /property +``` + ###Other
hadoop git commit: YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty (cherry picked from commit a380643d2044a4974e379965f65066df2055d00
Repository: hadoop Updated Branches: refs/heads/branch-2.7 368ab2cd3 - 8c5642296 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty (cherry picked from commit a380643d2044a4974e379965f65066df2055d003) Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/8c564229 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/8c564229 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/8c564229 Branch: refs/heads/branch-2.7 Commit: 8c5642296d207c7883f7a7a1d63f6771b1d15b0c Parents: 368ab2c Author: Jian He jia...@apache.org Authored: Tue Mar 10 10:54:08 2015 -0700 Committer: Jian He jia...@apache.org Committed: Tue Mar 10 11:17:16 2015 -0700 -- hadoop-yarn-project/CHANGES.txt | 3 +++ .../src/site/markdown/CapacityScheduler.md | 26 2 files changed, 29 insertions(+) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/8c564229/hadoop-yarn-project/CHANGES.txt -- diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index 71f8efd..74a7421 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -327,6 +327,9 @@ Release 2.7.0 - UNRELEASED YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. (Hitesh Shah via junping_du) +YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user +or group. (Gururaj Shetty via jianhe) + OPTIMIZATIONS YARN-2990. FairScheduler's delay-scheduling always waits for node-local and http://git-wip-us.apache.org/repos/asf/hadoop/blob/8c564229/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md index 3c32cdd..1cb963e 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md @@ -69,6 +69,8 @@ The `CapacityScheduler` supports the following features: * **Resource-based Scheduling** - Support for resource-intensive applications, where-in a application can optionally specify higher resource-requirements than the default, there-by accomodating applications with differing resource requirements. Currently, *memory* is the the resource requirement supported. +* **Queue Mapping based on User or Group** - This feature allows users to map a job to a specific queue based on the user or group. + Configuration - @@ -151,6 +153,30 @@ Configuration **Note:** An *ACL* is of the form *user1*, *user2spacegroup1*, *group2*. The special value of * implies *anyone*. The special value of *space* implies *no one*. The default is * for the root queue if not specified. + * Queue Mapping based on User or Group + + The `CapacityScheduler` supports the following parameters to configure the queue mapping based on user or group: + +| Property | Description | +|: |: | +| `yarn.scheduler.capacity.queue-mappings` | This configuration specifies the mapping of user or group to aspecific queue. You can map a single user or a list of users to queues. Syntax: `[u or g]:[name]:[queue_name][,next_mapping]*`. Here, *u or g* indicates whether the mapping is for a user or group. The value is *u* for user and *g* for group. *name* indicates the user name or group name. To specify the user who has submitted the application, %user can be used. *queue_name* indicates the queue name for which the application has to be mapped. To specify queue name same as user name, *%user* can be used. To specify queue name same as the name of the primary group for which the user belongs to, *%primary_group* can be used.| +| `yarn.scheduler.capacity.queue-mappings-override.enable` | This function is used to specify whether the user specified queues can be overridden. This is a Boolean value and the default value is *false*. | + +Example: + +``` + property + nameyarn.scheduler.capacity.queue-mappings/name + valueu:user1:queue1,g:group1:queue2,u:%user:%user,u:user2:%primary_group/value + description + Here, user1 is mapped to queue1, group1 is mapped to queue2, + maps users to queues with the same name as user, user2 is mapped + to queue name same as primary group respectively. The mappings will be + evaluated from left to right, and the first valid mapping will be used. + /description + /property +``` + ###Other
[33/49] hadoop git commit: YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty
YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/a380643d Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/a380643d Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/a380643d Branch: refs/heads/YARN-2928 Commit: a380643d2044a4974e379965f65066df2055d003 Parents: 8d5b01e Author: Jian He jia...@apache.org Authored: Tue Mar 10 10:54:08 2015 -0700 Committer: Jian He jia...@apache.org Committed: Tue Mar 10 11:15:57 2015 -0700 -- hadoop-yarn-project/CHANGES.txt | 3 +++ .../src/site/markdown/CapacityScheduler.md | 26 2 files changed, 29 insertions(+) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/a380643d/hadoop-yarn-project/CHANGES.txt -- diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index a6dcb29..82134db 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -381,6 +381,9 @@ Release 2.7.0 - UNRELEASED YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. (Hitesh Shah via junping_du) +YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user +or group. (Gururaj Shetty via jianhe) + OPTIMIZATIONS YARN-2990. FairScheduler's delay-scheduling always waits for node-local and http://git-wip-us.apache.org/repos/asf/hadoop/blob/a380643d/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md index 3c32cdd..1cb963e 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md @@ -69,6 +69,8 @@ The `CapacityScheduler` supports the following features: * **Resource-based Scheduling** - Support for resource-intensive applications, where-in a application can optionally specify higher resource-requirements than the default, there-by accomodating applications with differing resource requirements. Currently, *memory* is the the resource requirement supported. +* **Queue Mapping based on User or Group** - This feature allows users to map a job to a specific queue based on the user or group. + Configuration - @@ -151,6 +153,30 @@ Configuration **Note:** An *ACL* is of the form *user1*, *user2spacegroup1*, *group2*. The special value of * implies *anyone*. The special value of *space* implies *no one*. The default is * for the root queue if not specified. + * Queue Mapping based on User or Group + + The `CapacityScheduler` supports the following parameters to configure the queue mapping based on user or group: + +| Property | Description | +|: |: | +| `yarn.scheduler.capacity.queue-mappings` | This configuration specifies the mapping of user or group to aspecific queue. You can map a single user or a list of users to queues. Syntax: `[u or g]:[name]:[queue_name][,next_mapping]*`. Here, *u or g* indicates whether the mapping is for a user or group. The value is *u* for user and *g* for group. *name* indicates the user name or group name. To specify the user who has submitted the application, %user can be used. *queue_name* indicates the queue name for which the application has to be mapped. To specify queue name same as user name, *%user* can be used. To specify queue name same as the name of the primary group for which the user belongs to, *%primary_group* can be used.| +| `yarn.scheduler.capacity.queue-mappings-override.enable` | This function is used to specify whether the user specified queues can be overridden. This is a Boolean value and the default value is *false*. | + +Example: + +``` + property + nameyarn.scheduler.capacity.queue-mappings/name + valueu:user1:queue1,g:group1:queue2,u:%user:%user,u:user2:%primary_group/value + description + Here, user1 is mapped to queue1, group1 is mapped to queue2, + maps users to queues with the same name as user, user2 is mapped + to queue name same as primary group respectively. The mappings will be + evaluated from left to right, and the first valid mapping will be used. + /description + /property +``` + ###Other Properties * Resource Calculator
hadoop git commit: YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty
Repository: hadoop Updated Branches: refs/heads/trunk 8d5b01e00 - a380643d2 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/a380643d Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/a380643d Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/a380643d Branch: refs/heads/trunk Commit: a380643d2044a4974e379965f65066df2055d003 Parents: 8d5b01e Author: Jian He jia...@apache.org Authored: Tue Mar 10 10:54:08 2015 -0700 Committer: Jian He jia...@apache.org Committed: Tue Mar 10 11:15:57 2015 -0700 -- hadoop-yarn-project/CHANGES.txt | 3 +++ .../src/site/markdown/CapacityScheduler.md | 26 2 files changed, 29 insertions(+) -- http://git-wip-us.apache.org/repos/asf/hadoop/blob/a380643d/hadoop-yarn-project/CHANGES.txt -- diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt index a6dcb29..82134db 100644 --- a/hadoop-yarn-project/CHANGES.txt +++ b/hadoop-yarn-project/CHANGES.txt @@ -381,6 +381,9 @@ Release 2.7.0 - UNRELEASED YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. (Hitesh Shah via junping_du) +YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user +or group. (Gururaj Shetty via jianhe) + OPTIMIZATIONS YARN-2990. FairScheduler's delay-scheduling always waits for node-local and http://git-wip-us.apache.org/repos/asf/hadoop/blob/a380643d/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md -- diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md index 3c32cdd..1cb963e 100644 --- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md +++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md @@ -69,6 +69,8 @@ The `CapacityScheduler` supports the following features: * **Resource-based Scheduling** - Support for resource-intensive applications, where-in a application can optionally specify higher resource-requirements than the default, there-by accomodating applications with differing resource requirements. Currently, *memory* is the the resource requirement supported. +* **Queue Mapping based on User or Group** - This feature allows users to map a job to a specific queue based on the user or group. + Configuration - @@ -151,6 +153,30 @@ Configuration **Note:** An *ACL* is of the form *user1*, *user2spacegroup1*, *group2*. The special value of * implies *anyone*. The special value of *space* implies *no one*. The default is * for the root queue if not specified. + * Queue Mapping based on User or Group + + The `CapacityScheduler` supports the following parameters to configure the queue mapping based on user or group: + +| Property | Description | +|: |: | +| `yarn.scheduler.capacity.queue-mappings` | This configuration specifies the mapping of user or group to aspecific queue. You can map a single user or a list of users to queues. Syntax: `[u or g]:[name]:[queue_name][,next_mapping]*`. Here, *u or g* indicates whether the mapping is for a user or group. The value is *u* for user and *g* for group. *name* indicates the user name or group name. To specify the user who has submitted the application, %user can be used. *queue_name* indicates the queue name for which the application has to be mapped. To specify queue name same as user name, *%user* can be used. To specify queue name same as the name of the primary group for which the user belongs to, *%primary_group* can be used.| +| `yarn.scheduler.capacity.queue-mappings-override.enable` | This function is used to specify whether the user specified queues can be overridden. This is a Boolean value and the default value is *false*. | + +Example: + +``` + property + nameyarn.scheduler.capacity.queue-mappings/name + valueu:user1:queue1,g:group1:queue2,u:%user:%user,u:user2:%primary_group/value + description + Here, user1 is mapped to queue1, group1 is mapped to queue2, + maps users to queues with the same name as user, user2 is mapped + to queue name same as primary group respectively. The mappings will be + evaluated from left to right, and the first valid mapping will be used. + /description + /property +``` + ###Other Properties * Resource Calculator
