K0K0V0K commented on code in PR #6834:
URL: https://github.com/apache/hadoop/pull/6834#discussion_r1605127306
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerCgroups.md:
##########
@@ -12,51 +12,53 @@
limitations under the License. See accompanying LICENSE file.
-->
-Using CGroups with YARN
+Using Cgroups with YARN
=======================
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
-CGroups is a mechanism for aggregating/partitioning sets of tasks, and all
their future children, into hierarchical groups with specialized behaviour.
CGroups is a Linux kernel feature and was merged into kernel version 2.6.24.
From a YARN perspective, this allows containers to be limited in their resource
usage. A good example of this is CPU usage. Without CGroups, it becomes hard to
limit container CPU usage.
+Cgroups is a mechanism for aggregating/partitioning sets of tasks, and all
their future children, into hierarchical groups with specialized behaviour.
Cgroups (v1) is a Linux kernel feature and was merged into kernel version
2.6.24, while Control Group v2 is available since the kernel version 4.5. From
a YARN perspective, this allows containers to be limited in their resource
usage. A good example of this is CPU usage. Without cgroups, it becomes hard to
limit container CPU usage.
-CGroups Configuration
+Cgroups Configuration
---------------------
-This section describes the configuration variables for using CGroups.
+This section describes the configuration variables for using cgroups.
-The following settings are related to setting up CGroups. These need to be set
in *yarn-site.xml*.
+The following settings are related to setting up cgroups. These need to be set
in *yarn-site.xml*.
-|Configuration Name | Description |
-|:---- |:---- |
-| `yarn.nodemanager.container-executor.class` | This should be set to
"org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor". CGroups is
a Linux kernel feature and is exposed via the LinuxContainerExecutor. |
-| `yarn.nodemanager.linux-container-executor.resources-handler.class` | This
should be set to
"org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler".
Using the LinuxContainerExecutor doesn't force you to use CGroups. If you wish
to use CGroups, the resource-handler-class must be set to
CGroupsLCEResourceHandler. DefaultLCEResourcesHandler won't work. |
-| `yarn.nodemanager.linux-container-executor.cgroups.hierarchy` | The cgroups
hierarchy under which to place YARN proccesses(cannot contain commas). If
yarn.nodemanager.linux-container-executor.cgroups.mount is false (that is, if
cgroups have been pre-configured) and the YARN user has write access to the
parent directory, then the directory will be created. If the directory already
exists, the administrator has to give YARN write permissions to it recursively.
|
-| `yarn.nodemanager.linux-container-executor.cgroups.mount` | Whether the LCE
should attempt to mount cgroups if not found - can be true or false. |
-| `yarn.nodemanager.linux-container-executor.cgroups.mount-path` | Optional.
Where CGroups are located. LCE will try to mount them here, if
`yarn.nodemanager.linux-container-executor.cgroups.mount` is true. LCE will try
to use CGroups from this location, if
`yarn.nodemanager.linux-container-executor.cgroups.mount` is false. If
specified, this path and its subdirectories (CGroup hierarchies) must exist and
they should be readable and writable by YARN before the NodeManager is
launched. See CGroups mount options below for details. |
-| `yarn.nodemanager.linux-container-executor.group` | The Unix group of the
NodeManager. It should match the setting in "container-executor.cfg". This
configuration is required for validating the secure access of the
container-executor binary. |
+|Configuration Name | Description
|
+|:----
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `yarn.nodemanager.container-executor.class` | This should be set to
"org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor". Cgroups is
a Linux kernel feature and is exposed via the LinuxContainerExecutor.
|
+| `yarn.nodemanager.linux-container-executor.resources-handler.class` | This
should be set to
"org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler".
Using the LinuxContainerExecutor doesn't force you to use cgroups. If you wish
to use cgroups, the resource-handler-class must be set to
CGroupsLCEResourceHandler. DefaultLCEResourcesHandler won't work.
Review Comment:
```suggestion
| `yarn.nodemanager.linux-container-executor.resources-handler.class` | This
should be set to
"org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler".
Using the LinuxContainerExecutor doesn't force you to use cgroups. If you wish
to use cgroups, the resource-handler-class must be set to
CGroupsLCEResourceHandler. DefaultLCEResourcesHandler won't work. |
```
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/NodeManagerCgroups.md:
##########
@@ -12,51 +12,53 @@
limitations under the License. See accompanying LICENSE file.
-->
-Using CGroups with YARN
+Using Cgroups with YARN
=======================
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
-CGroups is a mechanism for aggregating/partitioning sets of tasks, and all
their future children, into hierarchical groups with specialized behaviour.
CGroups is a Linux kernel feature and was merged into kernel version 2.6.24.
From a YARN perspective, this allows containers to be limited in their resource
usage. A good example of this is CPU usage. Without CGroups, it becomes hard to
limit container CPU usage.
+Cgroups is a mechanism for aggregating/partitioning sets of tasks, and all
their future children, into hierarchical groups with specialized behaviour.
Cgroups (v1) is a Linux kernel feature and was merged into kernel version
2.6.24, while Control Group v2 is available since the kernel version 4.5. From
a YARN perspective, this allows containers to be limited in their resource
usage. A good example of this is CPU usage. Without cgroups, it becomes hard to
limit container CPU usage.
-CGroups Configuration
+Cgroups Configuration
---------------------
-This section describes the configuration variables for using CGroups.
+This section describes the configuration variables for using cgroups.
-The following settings are related to setting up CGroups. These need to be set
in *yarn-site.xml*.
+The following settings are related to setting up cgroups. These need to be set
in *yarn-site.xml*.
-|Configuration Name | Description |
-|:---- |:---- |
-| `yarn.nodemanager.container-executor.class` | This should be set to
"org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor". CGroups is
a Linux kernel feature and is exposed via the LinuxContainerExecutor. |
-| `yarn.nodemanager.linux-container-executor.resources-handler.class` | This
should be set to
"org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler".
Using the LinuxContainerExecutor doesn't force you to use CGroups. If you wish
to use CGroups, the resource-handler-class must be set to
CGroupsLCEResourceHandler. DefaultLCEResourcesHandler won't work. |
-| `yarn.nodemanager.linux-container-executor.cgroups.hierarchy` | The cgroups
hierarchy under which to place YARN proccesses(cannot contain commas). If
yarn.nodemanager.linux-container-executor.cgroups.mount is false (that is, if
cgroups have been pre-configured) and the YARN user has write access to the
parent directory, then the directory will be created. If the directory already
exists, the administrator has to give YARN write permissions to it recursively.
|
-| `yarn.nodemanager.linux-container-executor.cgroups.mount` | Whether the LCE
should attempt to mount cgroups if not found - can be true or false. |
-| `yarn.nodemanager.linux-container-executor.cgroups.mount-path` | Optional.
Where CGroups are located. LCE will try to mount them here, if
`yarn.nodemanager.linux-container-executor.cgroups.mount` is true. LCE will try
to use CGroups from this location, if
`yarn.nodemanager.linux-container-executor.cgroups.mount` is false. If
specified, this path and its subdirectories (CGroup hierarchies) must exist and
they should be readable and writable by YARN before the NodeManager is
launched. See CGroups mount options below for details. |
-| `yarn.nodemanager.linux-container-executor.group` | The Unix group of the
NodeManager. It should match the setting in "container-executor.cfg". This
configuration is required for validating the secure access of the
container-executor binary. |
+|Configuration Name | Description
|
+|:----
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `yarn.nodemanager.container-executor.class` | This should be set to
"org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor". Cgroups is
a Linux kernel feature and is exposed via the LinuxContainerExecutor.
|
+| `yarn.nodemanager.linux-container-executor.resources-handler.class` | This
should be set to
"org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler".
Using the LinuxContainerExecutor doesn't force you to use cgroups. If you wish
to use cgroups, the resource-handler-class must be set to
CGroupsLCEResourceHandler. DefaultLCEResourcesHandler won't work.
+| `yarn.nodemanager.linux-container-executor.cgroups.v2.enabled` | A property
to enable cgroup v2 support. Setting this to true YARN will try to use the
cgroup v2 structure and controllers. If this setting is true, but no unified
(v2) hierarchy is mounted it will automatically fall back to v1. Defaults to
false.
||
Review Comment:
```suggestion
| `yarn.nodemanager.linux-container-executor.cgroups.v2.enabled` | A
property to enable cgroup v2 support. Setting this to true YARN will try to use
the cgroup v2 structure and controllers. If this setting is true, but no
unified (v2) hierarchy is mounted it will automatically fall back to v1.
Defaults to false.
|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
