[
https://issues.apache.org/jira/browse/YARN-11669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019236#comment-18019236
]
Benjamin Teke commented on YARN-11669:
--------------------------------------
[~mkrup] yes, the "device controller" based classes are not yet implemented, at
this point we're not actively developing them. However the rest of the code is
merged, and we're using it in production, so it is entirely possible releasing
the current state as is. As for the 3.5.0 release: I haven't seen jiras/talks
about it, not sure what the content is. The first step would be to create a
jira like this: HADOOP-19565
> [Umbrella] cgroup v2 support
> ----------------------------
>
> Key: YARN-11669
> URL: https://issues.apache.org/jira/browse/YARN-11669
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: yarn
> Reporter: Ferenc Erdelyi
> Assignee: Benjamin Teke
> Priority: Major
> Fix For: 3.5.0
>
>
> cgroup v2 has some fundamental changes compared to v1. RHEL9, Ubuntu 22
> already moved to cgroup v2 as a default, hence YARN should support it. This
> umbrella tracks the required work.
> [Documentation|https://docs.kernel.org/admin-guide/cgroup-v2.html]
> A way to test the newly added features:
> # Turn on cgroup v1 based on the current
> [documentation|https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html].
> # System prerequisites:
> ## the file {{/etc/mtab}} should contain a mount path with the file system
> type {{cgroup2}}, by default this could be {{/sys/fs/cgroup}} on most OS's
> ## the {{cgroup.subtree_control}} file should contain the necessary
> controllers (update it with: {{echo "+cpu +io +memory" >
> cgroup.subtree_control}})
> ## either create the YARN hierarchy and give recursive access to the user
> running the NM on the node. The hierarchy is {{hadoop-yarn}} by default
> (controller by
> {{yarn.nodemanager.linux-container-executor.cgroups.hierarchy}}), and
> recursive mode is required, because as soon as the directory is created it
> will be filled with the controller files which YARN will try to edit.
> ### Alternatively if the NM process user has access rights on the
> {{/sys/fs/cgroup}} directory it'll try to create the hierarchy and update the
> {{cgroup.subtree_control}} file.
> # YARN configuration
> ## {{yarn.nodemanager.linux-container-executor.cgroups.mount-path}} should
> point to the directory where the cgroup2 structure is mounted and the
> {{hadoop-yarn}} hierarchy was created
> ## {{yarn.nodemanager.linux-container-executor.cgroups.v2.enabled}} should be
> set to {{true}}
> ## Enable a cgroup controller, like {{yarn. nodemanager. resource.
> cpu.enabled}}: {{true}}
> # Launch the NM and monitor the cgroup files on container launches (i.e:
> {{/sys/fs/cgroup/hadoop-yarn/container_id/cpu.weight}})
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
