[
https://issues.apache.org/jira/browse/YARN-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15873308#comment-15873308
]
Manikandan R commented on YARN-5179:
------------------------------------
[~asuresh]
Based on my understanding, resourceCalculatorPlugin.getNumProcessors() and
maxVCoresAllottedForContainers difference cause this millivcores calculation
issue. resourceCalculatorPlugin.getNumProcessors() simply return no. of logical
processors count, whereas maxVCoresAllottedForContainers has values based on
yarn.nodemanager.resource.count-logical-processors-as-cores property. If above
property is true, then maxVCoresAllottedForContainers is equal to
resourceCalculatorPlugin.getNumProcessors(), otherwise, it differs. I am
assuming yarn.nodemanager.resource.detect-hardware-capabilities has been
enabled in this case.
Can we use actual vcores being used?
Instead of
{code}float cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore /
resourceCalculatorPlugin.getNumProcessors();
int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000 *
maxVCoresAllottedForContainers /nodeCpuPercentageForYARN);{code}
Can we use this?
{code}float cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore /
resourceCalculatorPlugin.getNumVcoresUsed();
int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000 *
resourceCalculatorPlugin.getNumVcoresUsed()/nodeCpuPercentageForYARN);{code}
Please provide your thoughts.
> Issue of CPU usage of containers
> --------------------------------
>
> Key: YARN-5179
> URL: https://issues.apache.org/jira/browse/YARN-5179
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.7.0
> Environment: Both on Windows and Linux
> Reporter: Zhongkai Mi
>
> // Multiply by 1000 to avoid losing data when converting to int
> int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000
> * maxVCoresAllottedForContainers /nodeCpuPercentageForYARN);
> This formula will not get right CPU usage based vcore if vcores != physical
> cores.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
