[
https://issues.apache.org/jira/browse/YARN-6384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944421#comment-15944421
]
dengkai commented on YARN-6384:
-------------------------------
Yes, when strictResourceUsageMode is true by setting
yarn.nodemanager.resource.percentage-physical-cpu-limit true, we can limit
every container.
{quote}
if (isCpuWeightEnabled()) {
int containerVCores = containerResource.getVirtualCores();
createCgroup(CONTROLLER_CPU, containerName);
int cpuShares = CPU_DEFAULT_WEIGHT * containerVCores;
updateCgroup(CONTROLLER_CPU, containerName, "shares",
String.valueOf(cpuShares));
if (strictResourceUsageMode) {
int nodeVCores =
conf.getInt(YarnConfiguration.NM_VCORES,
YarnConfiguration.DEFAULT_NM_VCORES);
if (nodeVCores != containerVCores) {
float containerCPU =
(containerVCores * yarnProcessors) / (float) nodeVCores;
int[] limits = getOverallLimits(containerCPU);
updateCgroup(CONTROLLER_CPU, containerName, CPU_PERIOD_US,
String.valueOf(limits[0]));
updateCgroup(CONTROLLER_CPU, containerName, CPU_QUOTA_US,
String.valueOf(limits[1]));
}
}
}
{quote}
If false, we just can let cpu scheduled fairly between containers with
cpu.shares. In many cases, container use a little more cpu is reasonable, but
not be so much. The strictResourceUsageMode is inflexible by contrast.
> Add configuratin to set max cpu usage when strict-resource-usage is false
> with cgroups
> --------------------------------------------------------------------------------------
>
> Key: YARN-6384
> URL: https://issues.apache.org/jira/browse/YARN-6384
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: dengkai
>
> When using cgroups on yarn, if
> yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage is
> false, user may get very more cpu time than expected based on the vcores.
> There should be a upper limit even resource-usage is not strict, just like a
> percentage which user can get more than promised by vcores. I think it's
> important in a shared cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
