[ https://issues.apache.org/jira/browse/YARN-8924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhankun Tang updated YARN-8924: ------------------------------- Summary: Refine the document or code related to Cgroups/LCE (was: Refine the document or code related to legacy CPU isolation/enforcement) > Refine the document or code related to Cgroups/LCE > -------------------------------------------------- > > Key: YARN-8924 > URL: https://issues.apache.org/jira/browse/YARN-8924 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Zhankun Tang > Assignee: Zhankun Tang > Priority: Minor > > This is to re-think the legacy configuration/code of CPU resource isolation. > In YARN-3542, we involve _CGroupsCpuResourceHandlerImpl_ based on new > _ResourceHandler_ mechanism but leaves the configuration > "yarn.nodemanager.linux-container-executor.resources-handler.class" there for > a long time. Now it seems confusing to the end user. > Check YARN-6729, one sets "_DefaultLCEResourcesHandler_" and found that give > "percentage-physical-cpu-limit" a value less than "100" doesn't work. > As far as I know, internally, the _CgroupsLCEResourcesHandler_ and > _DefaultLCEResourcesHandler are_ both deprecated. YARN won't use them anymore. > Instead, YARN uses _CGroupsCpuResourceHandlerImpl_ to do CPU isolation and > only in LCE. If we want to enforce CPU usage, we must set LCE and > CgroupsLCEResourceHandler like this: > {noformat} > <property> > <description>who will execute(launch) the containers.</description> > <name>yarn.nodemanager.container-executor.class</name> > > <value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value> > </property> > <property> > <description>The class which should help the LCE handle > resources.</description> > > <name>yarn.nodemanager.linux-container-executor.resources-handler.class</name> > > <value>org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler<value> > </property>{noformat} > Based on above settings can the CPU related settings like > "percentage-physical-cpu-limit" works as expected. > To avoid confusing like YARN-6729, we can do two things: > # More clear document about how should user configure CPU > isolation/enforcement in "NodeManagerCgroups.md" > # Make "ResourceHandlerModuler" stable and remove legacy code and update the > document to recommend new setting "yarn.nodemanager.resource.cpu.enabled" > Thoughts? [~leftnoteasy], [~vinodkv], [~vvasudev] -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org