[
https://issues.apache.org/jira/browse/YARN-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen updated YARN-8911:
-----------------------------
Description:
*UPDATE*: *per discussion below, the cpu resource utlization
(ResourceUtilzation.cpu) is incorrectly documented as a percentage number in
[0, 1.0] in the java doc. It is however, used as the number of vcores used
throughout the system. YARN-4597 is a victim of the misleading java doc.*
ResourceUtilization represents the cpu utilization with a float number in [0,
1.0], i.e. the percentage of cpu usage across the node. However, when
Containers Monitor tracks the total aggregate resource utilization of all
containers, it adds up the total number of vcores used by all running
containers.
(If you have 6 containers running, each consuming 1 vcore, you'd see the
aggregated cpu container utilization being 6.0, but it's supposed to be always
between 0 and 1.0) See [the
code|https://github.com/apache/hadoop/blob/beb850d8f7f1fefa7a6d9502df2b4a4eea372523/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java#L672]
was:
ResourceUtilization represents the cpu utilization with a float number in [0,
1.0], i.e. the percentage of cpu usage across the node. However, when
Containers Monitor tracks the total aggregate resource utilization of all
containers, it adds up the total number of vcores used by all running
containers.
(If you have 6 containers running, each consuming 1 vcore, you'd see the
aggregated cpu container utilization being 6.0, but it's supposed to be always
between 0 and 1.0) See [the
code|https://github.com/apache/hadoop/blob/beb850d8f7f1fefa7a6d9502df2b4a4eea372523/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java#L672]
> ContainerScheduler incorrectly uses percentage number as the cpu resource
> utlization
> ------------------------------------------------------------------------------------
>
> Key: YARN-8911
> URL: https://issues.apache.org/jira/browse/YARN-8911
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Major
> Attachments: YARN-8911.00.patch
>
>
> *UPDATE*: *per discussion below, the cpu resource utlization
> (ResourceUtilzation.cpu) is incorrectly documented as a percentage number in
> [0, 1.0] in the java doc. It is however, used as the number of vcores used
> throughout the system. YARN-4597 is a victim of the misleading java doc.*
>
>
> ResourceUtilization represents the cpu utilization with a float number in [0,
> 1.0], i.e. the percentage of cpu usage across the node. However, when
> Containers Monitor tracks the total aggregate resource utilization of all
> containers, it adds up the total number of vcores used by all running
> containers.
>
> (If you have 6 containers running, each consuming 1 vcore, you'd see the
> aggregated cpu container utilization being 6.0, but it's supposed to be
> always between 0 and 1.0) See [the
> code|https://github.com/apache/hadoop/blob/beb850d8f7f1fefa7a6d9502df2b4a4eea372523/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java#L672]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
