[
https://issues.apache.org/jira/browse/YARN-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931007#comment-15931007
]
Miklos Szegedi commented on YARN-5179:
--------------------------------------
[[email protected]], could you help me, what am I missing?
I ran a test and the numbers look right to me. Here is the code:
{code}
float cpuUsagePercentPerCore = pTree.getCpuUsagePercent();
float cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore /
resourceCalculatorPlugin.getNumProcessors();
// Multiply by 1000 to avoid losing data when converting to int
int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000
* maxVCoresAllottedForContainers /nodeCpuPercentageForYARN);
{code}
And here are the values. I have 4 processors, and we use about 3 processors
with our job. This is 75% of total CPU resources, so the first three numbers
look right. I chose 8192 vcores in yarn.nodemanager.resource.cpu-vcores to have
a very different number than the 4 cores. Then we prorate 75% to 8192 vcores,
so we get about 6141 number of vcores, which is 6140747 millivcores. That
sounds right. What would you expect in this case?
{code}
resourceCalculatorPlugin.getNumProcessors() = 4
cpuUsagePercentPerCore = 299.84116
cpuUsageTotalCoresPercentage = 74.96029
maxVCoresAllottedForContainers = 8192
nodeCpuPercentageForYARN = 100
milliVcoresUsed = 6140747
{code}
Approach #1 above would give us 3000, which is millirealcores. Why would we
multiply and then subtract with resourceCalculatorPlugin.getNumVcoresUsed()?
It is an interesting question though what metric is the most useful and simple
from the user point of view. I think I like the 75% the most.
> Issue of CPU usage of containers
> --------------------------------
>
> Key: YARN-5179
> URL: https://issues.apache.org/jira/browse/YARN-5179
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.7.0
> Environment: Both on Windows and Linux
> Reporter: Zhongkai Mi
>
> // Multiply by 1000 to avoid losing data when converting to int
> int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000
> * maxVCoresAllottedForContainers /nodeCpuPercentageForYARN);
> This formula will not get right CPU usage based vcore if vcores != physical
> cores.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
