[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163573#comment-15163573 ] Karthik Kambatla commented on YARN-3304: [~djp] - did we ever file follow up JIRAs to delete these deprecated methods from trunk? If not, I would like to file them and get the trunk changes in. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391181#comment-14391181 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2100 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2100/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390654#comment-14390654 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #141 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/141/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390632#comment-14390632 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #150 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/150/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390616#comment-14390616 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2082 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2082/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390436#comment-14390436 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Yarn-trunk #884 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/884/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390424#comment-14390424 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #150 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/150/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389774#comment-14389774 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-trunk-Commit #7480 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7480/]) YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) (vinodkv: rev 7610925e90155dfe5edce05da31574e4fb81b948) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch, > YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389705#comment-14389705
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708554/YARN-3304-appendix-v4.patch
against trunk revision e428fea.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7180//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7180//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-appendix-v2.patch,
> YARN-3304-appendix-v3.patch, YARN-3304-appendix-v4.patch,
> YARN-3304-appendix.patch, YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch,
> YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389558#comment-14389558
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708519/YARN-3304-appendix-v3.patch
against trunk revision e428fea.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1154 javac
compiler warnings (more than the trunk's current 1148 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7179//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/7179//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7179//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-appendix-v2.patch,
> YARN-3304-appendix-v3.patch, YARN-3304-appendix.patch, YARN-3304-v2.patch,
> YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch,
> YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch,
> YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch,
> YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389514#comment-14389514 ] Karthik Kambatla commented on YARN-3304: Filed YARN-3427 to follow up. I am okay with the latest patch (looked only at ResourceCalculatorProcessTree), but defer to Vinod for full review. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix.patch, YARN-3304-v2.patch, > YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389415#comment-14389415
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708519/YARN-3304-appendix-v3.patch
against trunk revision e428fea.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:red}-1 eclipse:eclipse{color}. The patch failed to build with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7178//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7178//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-appendix-v2.patch,
> YARN-3304-appendix-v3.patch, YARN-3304-appendix.patch, YARN-3304-v2.patch,
> YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch,
> YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch,
> YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch,
> YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389410#comment-14389410 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- Looks good to me. Will check it in unless either of jenkins or [~kasha] say no. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix.patch, YARN-3304-v2.patch, > YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389361#comment-14389361 ] Junping Du commented on YARN-3304: -- bq. We do not need to explicitly say so for each deprecated API. That said, in practice, we don't tend to remove deprecated APIs in the next release itself. Agree. That sounds reasonable. bq. Back to the patch. Junping Du, can you address these? "Also, can we deprecate (..) getCumulativeRssmem(int) and getCumulativeVmem(int)? " Sure. [~vinodkv]. Incorporate this comments from [~kasha] in appendix-v3 patch. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, > YARN-3304-appendix-v3.patch, YARN-3304-appendix.patch, YARN-3304-v2.patch, > YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389144#comment-14389144 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2099 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2099/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/CHANGES.txt > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389077#comment-14389077 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #149 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/149/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/CHANGES.txt > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388929#comment-14388929 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- Interesting discussion. The expectation on deprecated APIs is that they are going to be deleted in the next major release. We do not need to explicitly say so for each deprecated API. That said, in practice, we don't tend to remove deprecated APIs in the next release itself. For example, we have tried this with old MR job APIs which we went back on given the very long tail. So even though we set the expectation of deprecated-APIs removal in next release, we conservatively do so a little later in practice. Back to the patch. [~djp], can you address these? bq. Also, can we deprecate (..) getCumulativeRssmem(int) and getCumulativeVmem(int)? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388824#comment-14388824
]
Karthik Kambatla commented on YARN-3304:
[~hitesh] - thanks for okay their removal in 3.0.0. The reason I was asking -
our compat policy requires us to deprecate an API for an entire major release
before being removed in the next one. This is an ambiguous case, because this
API is essentially born deprecated :)
[~djp] - can you add a blurb in the javadoc to say we are going to remove this
in 3.0.0. Also, can we deprecate and add the blurb to
{{getCumulativeRssmem(int)}} and {{getCumulativeVmem(int)}}?
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch,
> YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch,
> YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch,
> YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch,
> YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388820#comment-14388820 ] Allen Wittenauer commented on YARN-3304: 3.0.0 will likely release by the end of the year. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388817#comment-14388817 ] Hitesh Shah commented on YARN-3304: --- [~kasha] 3.0.0 is a major release. I would assume all deprecated apis should be removed. Given the length of time after which new major releases come into existence, there would be no point of deprecating apis if they are not removed in the next major release. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388802#comment-14388802 ] Karthik Kambatla commented on YARN-3304: [~djp] - just to be clear, we are adding a deprecated method to a class we are marking Public first time in 2.7. I find it weird, but can live with it as long as we have a plan/timeline to remove it? [~hitesh] - is Tez okay with YARN removing these deprecated methods in 3.0.0? If yes, I would like for the javadoc for those methods to explicitly state that and a follow-up JIRA filed and fixed against trunk. If that is not okay, we are kidding ourselves by marking it deprecated. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388624#comment-14388624 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #140 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/140/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/CHANGES.txt > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388602#comment-14388602 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2081 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2081/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388509#comment-14388509 ] Junping Du commented on YARN-3304: -- I would like to get appendix (v2) patch in not only because it fix an issue of existing patch but also it shows how collaborative we are to downstream projects, like Tez. >From what I see from discussions above, they already did what they can do for >Hadoop part - delivered a patch to make API public and wait for the commit. It >is us - a small group of people watching this JIRA earlier - decided to change >API (for more readable reason) at last minute of release. The whole story here >sounds very interesting and like a multi-threads race condition happen on real >world. Through appendix patch, we get sync on the same page again. So, please help to review and get fix in. Thanks! > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388461#comment-14388461
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708410/YARN-3304-appendix-v2.patch
against trunk revision b5a22e9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7173//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7173//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch,
> YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch,
> YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch,
> YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch,
> YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388431#comment-14388431 ] Hudson commented on YARN-3304: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #883 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/883/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388411#comment-14388411 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #149 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/149/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-appendix-v2.patch, YARN-3304-appendix.patch, > YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch, > YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch, > YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388370#comment-14388370
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708294/YARN-3304-appendix.patch
against trunk revision b5a22e9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1162 javac
compiler warnings (more than the trunk's current 1148 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7171//testReport/
Javac warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/7171//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7171//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-appendix.patch, YARN-3304-v2.patch,
> YARN-3304-v3.patch, YARN-3304-v4-boolean-way.patch,
> YARN-3304-v4-negative-way-MR.patch, YARN-3304-v4-negtive-value-way.patch,
> YARN-3304-v6-no-rename.patch, YARN-3304-v6-with-rename.patch,
> YARN-3304-v7.patch, YARN-3304-v8.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387812#comment-14387812 ] Allen Wittenauer commented on YARN-3304: If these methods were used before they were made public, I have zero sympathy. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387810#comment-14387810 ] Hitesh Shah commented on YARN-3304: --- https://issues.apache.org/jira/browse/YARN-3297 is probably relevant too. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387808#comment-14387808 ] Hitesh Shah commented on YARN-3304: --- [~aw] Thanks for putting it so bluntly. You may wish to look at the related jiras such as https://issues.apache.org/jira/browse/YARN-3296. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387786#comment-14387786 ] Allen Wittenauer commented on YARN-3304: bq. To change tez, we will need to create shims based on which version of hadoop, the plugin is being built for. The Tez community could have opted to work the Hadoop community to make this public. They didn't. That's the risks when a program hits a private interface. Tez wouldn't be the first to make such a stupid mistake and won't be the first to have to pay the consequences. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387656#comment-14387656 ] Hitesh Shah commented on YARN-3304: --- Forgot to add, we use this for resource monitoring for a task within a container. Given that we run multiple tasks within the same container, this api stability becomes more important as YARN cannot provide the resource monitoring functionality at the granularity that we need. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387645#comment-14387645 ] Hitesh Shah commented on YARN-3304: --- [~kasha] The reason I re-opened this was because now, we cannot write a common lightweight resource calculator that can work with multiple versions of hadoop. To change tez, we will need to create shims based on which version of hadoop, the plugin is being built for. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387585#comment-14387585 ] Karthik Kambatla commented on YARN-3304: [~vinodkv], [~hitesh] - given we have made this Public only in 2.7 which hasn't been released yet, is it possible that we update Tez instead of YARN? It seems like an overkill to add deprecated APIs for a previously non-Public API. This should be the last time Tez fixes this. If we remove deprecated APIs in the future, we will break Tez anyway. No? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387509#comment-14387509 ] Junping Du commented on YARN-3304: -- Let's do here. I will upload a quick patch soon. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387478#comment-14387478 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- To summarize what happened so far - We had a configuration property to ResourceCalculatorProcessTree since for ever. But it was never usable in practice as ResourceCalculatorProcessTree itself was private and so could have gone through multiple compatible/incompatible changes. - I don't know of anyone outside of Hadoop MapReduce as ResourceCalculatorProcessTree is neither public nor documented. -- Besides Tez, that is. - 2.7.0 is the first release where we are trying to make it public: YARN-3296. - The API is only for developers to extend and create a new implementation. Users could never use it directly. MapReduce used it, but I think long term CPU millis for the container should come from YARN. -- Except, Tez is also using it That said, let's do this - Recreate the old APIs and deprecate them. - Remark the getResourceCalculatorProcessTree() API as Public. Here or another JIRA. Fine both ways. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387345#comment-14387345
]
Hitesh Shah commented on YARN-3304:
---
Sigh. This broke compatibility again. Was there a reason why APIs were just
removed/renamed instead of some form of supporting 2 APIs with a way to check
at runtime whether the plugin supports old or new APIs? ( and the old ones
being deprecated ).
{code}
public int getValueFromOldAPI();
public int getValueFromNewAPI();
public boolean supportsNewAPI() { return false; }
if (supportsNewAPI()) {
getValueFromNewAPI();
} else {
getValueFromOldAPI();
}
...
{code}
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Fix For: 2.7.0
>
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch,
> YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14387021#comment-14387021 ] Hudson commented on YARN-3304: -- FAILURE: Integrated in Hadoop-trunk-Commit #7461 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7461/]) YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. (vinodkv: rev c358368f511963ad8e35f030b9babee541e1bd01) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestResourceCalculatorProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestWindowsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/WindowsBasedProcessTree.java * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestProcfsBasedProcessTree.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/CpuTimeTracker.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorProcessTree.java > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Fix For: 2.7.0 > > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386997#comment-14386997 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- Tx for the update, [~djp]. Looks good to me now, checking this in. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386847#comment-14386847 ] Junping Du commented on YARN-3304: -- Hi [~adhoot], [~kasha] and [~vinodkv], I am sure v8 patch is in good status now. Can you review it again? Thanks! > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch, > YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386823#comment-14386823
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708157/YARN-3304-v8.patch
against trunk revision ae3e8c6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7147//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7147//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304-v8.patch,
> YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385405#comment-14385405
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707993/YARN-3304-v7.patch
against trunk revision e0ccea3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7138//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7138//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304.patch,
> yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385382#comment-14385382 ] Junping Du commented on YARN-3304: -- v7 should apply the latest trunk. Manually kick off Jenkins test... > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304.patch, > yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385367#comment-14385367
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707993/YARN-3304-v7.patch
against trunk revision e0ccea3.
{color:red}-1 @author{color}. The patch appears to contain @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include new
or modified test files.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7137//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304.patch,
> yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385362#comment-14385362
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707991/YARN-3304-v6-with-rename.patch
against trunk revision 27d49e6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7135//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7135//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304-v6-with-rename.patch, YARN-3304-v7.patch, YARN-3304.patch,
> yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385352#comment-14385352 ] Anubhav Dhoot commented on YARN-3304: - The renames look good. A couple of comments getSmapBasedCumulativeRssmem should also be renamed to remove cumulative, getSmapBasedRssMemorySize? Looks like at TestWindowsBasedProcessTree line 60, getCumulativeCpuTime got replaced by getRssMemorySize. That should be reverted. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch, > YARN-3304-v6-with-rename.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385319#comment-14385319
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707987/YARN-3304-v6-no-rename.patch
against trunk revision 27d49e6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7134//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7134//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304-v6-no-rename.patch,
> YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385275#comment-14385275 ] Junping Du commented on YARN-3304: -- Wait... Sounds like getCumulativeCpuTime() is proper as it add up across time, getCumulativeMem() is just a summarize of all process which should be changed something else. Am I missing something? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385263#comment-14385263
]
Junping Du commented on YARN-3304:
--
There is an small issue for getCumulativeCpuTime() in v5 patch:
if p.getDtime() ==0, below code will return UNAVAILABLE, but it should return 0.
{code}
public long getCumulativeCpuTime() {
if (JIFFY_LENGTH_IN_MILLIS < 0) {
return UNAVAILABLE;
}
long incJiffies = 0;
for (ProcessInfo p : processTree.values()) {
if (p != null) {
incJiffies += p.getDtime();
}
}
if (incJiffies > 0 && cpuTime == UNAVAILABLE) {
cpuTime = 0L;
}
cpuTime += incJiffies * JIFFY_LENGTH_IN_MILLIS;
return cpuTime;
}
{code}
I will fix this and name (suggested by Vinod) and MR in v6 patch.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385097#comment-14385097 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- We should rename it to be getCpuWallTime(). Do it here itself? Also, [~djp], change the callers in MapReduce to act on the UNAVAILABLE value? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385064#comment-14385064 ] Anubhav Dhoot commented on YARN-3304: - ProcfsProcessTree#getCumulativeCpuTime is cumulative over time which makes sense for CPU. The getCumulativeMem is really summing up snapshot value all for processes in the tree, which is appropriate for memory. We could name it something other than cumulative to make it clearer. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385031#comment-14385031
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707943/yarn-3304-5.patch
against trunk revision 3836ad6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7133//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7133//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385018#comment-14385018
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707940/yarn-3304-5.patch
against trunk revision 3836ad6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7132//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7132//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384992#comment-14384992 ] Karthik Kambatla commented on YARN-3304: Just uploaded the v5 patch. The changes are primarily to define ResourceCalculatorProcessTree#UNAVAILABLE and use that elsewhere in the code, and javadoc changes. [~adhoot] - ProcfsProcessTree#getCumulativeCpuTime is different compared to getCumulativeMem methods in that class, and even getCumulativeCpuTime in WindowsProcfsProcessTree. It seems to continue adding up usage across multiple calls to getCumulativeCpuTime, while rest of them freshly compute on every call. Is this intentional? If not, we might want to fix that too. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch, > YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch, yarn-3304-5.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384937#comment-14384937
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707923/YARN-3304-v4-negative-way-MR.patch
against trunk revision 3836ad6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7131//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negative-way-MR.patch,
> YARN-3304-v4-negtive-value-way.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384903#comment-14384903 ] Junping Du commented on YARN-3304: -- Yes. Please go ahead. [~kasha], thanks! I will post a MR patch here too - given this a blocker, we should move really quickly. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384899#comment-14384899 ] Karthik Kambatla commented on YARN-3304: The approach in the "negative value" patch looks reasonable. Have a bunch of nit-picks. [~djp] - if you don't mind, I ll just post an update with the changes I have in mind in the interest of time. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384887#comment-14384887
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707904/YARN-3304-v4-negtive-value-way.patch
against trunk revision 3836ad6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7129//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7129//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Junping Du
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch,
> YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negtive-value-way.patch,
> YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384868#comment-14384868 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- I like the "return negative" patch - very maintainable going forward. I am good with this, would like to get it in. [~kasha]/[~adhoot], care a final look? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, > YARN-3304-v4-boolean-way.patch, YARN-3304-v4-negtive-value-way.patch, > YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384830#comment-14384830 ] Junping Du commented on YARN-3304: -- Thanks [~vinodkv] for comments! Agree that the second comment is very important, and we should put it on public interface now. For boolean API VS. negative value, sounds like more people may prefer the later one so far. I would like to hear your more comments when you see the finally code looks like. In v4 patch, I will post 2 patches, one for boolean API (with addressing 2nd comment above based on v3 patch) way, and the other is for negative value way. Honestly speaking, I don't feel very comfortable in writing patch for the later way as every return -1 sounds like a warning to external developer that to follow the implicit contract here... Also, please note that the negative value way is not a completed work which need YARN-3392 ([~adhoot] have a demo patch there) and fix in MapReduce resource counter. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384439#comment-14384439 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- I too feel the same w.r.t the code duplication. It is going to be very hard maintaining this. To me, all ideas seem to converge on either throwing an exception or return -1. Given there were no external developers using this before and this is the first release where we are making this public, I think we should go with the -1 approach. I also think we should clearly javadoc this class saying this should not be used for external users, it is more of an SPI for developers to extend and include their own process-tree implementation. (Writing it that way, I don't see this to be public at all, but we screwed up way back when we made this a user-visible configuration). For doing this, we should make the getResourceCalculatorProcessTree() an @Private method. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Junping Du >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383762#comment-14383762 ] Junping Du commented on YARN-3304: -- Thanks [~adhoot] for comments. bq. Overall the API looks usable and will fix the issues. Thanks for confirmation on this. That is important. bq. But the implementation has to do a lot of duplication of code to implement the boolean api. The CPU isAvailable is straightforward but the memory calculations seem to rely on duplicating code that does the actual calculation. I admit that there are some duplicate code (with getting CPU/Memory values) here. The reason I didn't do more refactor work is less risk of changing existing logic at this moment. I think we can refactor it later (in next release) to reuse logic for boolean API and get values. Anyway, this belongs to implementation not API and we don't have to finalize it now. bq. Wouldn't using it as -1 be easier instead of duplicating code to keep it as 0? First, replacing default memory resource from 0 to -1 also take extra effort and extra checking resource value is not -1 in calculating resource. Second, like I mentioned above, the consumer side (ContainerMetrics, MR resource counter, new TimelineService, etc.) need to handle -1 explicitly and existing MR resource counter never have negative value for memory resource before. Last but not the least, as a public API, negative resource value is always more misleading than boolean API for resource available. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383616#comment-14383616 ] Anubhav Dhoot commented on YARN-3304: - Overall the API looks usable and will fix the issues But the implementation has to do a lot of duplication of code to implement the boolean api. The CPU isAvailable is straightforward but the memory calculations seem to rely on duplicating code that does the actual calculation. Wouldn't using it as -1 be easier instead of duplicating code to keep it as 0? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382759#comment-14382759 ] Junping Du commented on YARN-3304: -- Hi [~kasha] and [~adhoot], v3 patch should be a complete and clean solution for this blocker. Can you help to review and comment? Thanks! > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381913#comment-14381913
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707398/YARN-3304-v3.patch
against trunk revision b4b4fe9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7115//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7115//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381282#comment-14381282
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707396/YARN-3212-v3.patch
against trunk revision 44809b8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.TestRMHA
org.apache.hadoop.yarn.server.resourcemanager.TestRMAdminService
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication
org.apache.hadoop.yarn.server.resourcemanager.TestMoveApplication
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7112//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7112//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381256#comment-14381256
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707398/YARN-3304-v3.patch
against trunk revision 44809b8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7113//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7113//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304-v3.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14381205#comment-14381205
]
Junping Du commented on YARN-3304:
--
I have uploaded the v3 patch with adding a new API of check availability. You
can see it is not so complicated and very handy to use. We don't even need any
changes happen in ContainerMetrics side in YARN-3392 (except fixing some race
condition there which is good).
If we go on with negative value, not even saying MR resource metrics counter
will get negative value, we also need to provide negative value to memory
resource when it is not available (now it is 0), and make some existing code
(like below) to handle negative value which doesn't sound elegant to me.
{code}
ContainerMetrics.forContainer(
containerId, containerMetricsPeriodMs).recordMemoryUsage(
(int) (currentPmemUsage >> 20));
{code}
Thoughts?
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3212-v3.patch, YARN-3304-v2.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379908#comment-14379908 ] Karthik Kambatla commented on YARN-3304: My bad. I didn't realize one has to call updateProcessTree before calling getUsage. While I don't like the fact that one needs to call at least two methods to get a value in this API, I understand changing this might be a lot more involved. Given that updateProcessTree takes care of fetching the values and getUsage or isAvailable calls only access this pre-fetched value, I am okay with a boolean method or returning -1 with a marginal preference for the latter. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378502#comment-14378502 ] Junping Du commented on YARN-3304: -- bq. Do we expect the code to be the following? That's right. bq. If so, how do we handle the usage being available at the time of calling isAvailable, and not being available at the time of calling getUsage? To avoid this issue, we could get the usage on the availability call and cache it, and the getUsage call would return this cached value? I don't quite understand this. We will always call pTree.updateProcessTree() to have a snapshot of processor info before we are consuming the CPU/Memory data there. I don't see any differences with adding an additional check for availability here. Am I missing something here? bq. But, requiring the availability call now is an even more incompatible change, no? I was thinking check availability call is not a must, but only in case we want to tell different between 0 value and not available. Previous code (like resource counters in MR) can still get value of 0 which sounds more compatible for me. bq. What is the cost of breaking compat of this previously Private API? I have a feeling it would be worth not making the API super-complicated. My concern is not only breaking previous Private API of ResourceCalculatorProcessTree, but also break the consistency of behavior (and bring odd result) to things like: MapReduce resource counter. It never get negative value previously. bq. I want to avoid fixing this in a hurry just to unblock the release. I am willing to prioritize this, chat offline if need be, and solve it the right way. +1. That's also what I wish. bq. If we think that is too slow, we could always revert YARN-3296 for 2.7. I don't prefer to do this unless we run out of time. BTW, [~adhoot], I will look at your patch in YARN-3392. Thanks! > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378354#comment-14378354
]
Karthik Kambatla commented on YARN-3304:
Thought a little more about this. If we choose to go with returning 0 and
adding boolean methods for availability, I would like to see how the
corresponding user code will look like compared to returning -1.
Do we expect the code to be the following? If so, how do we handle the usage
being available at the time of calling isAvailable, and not being available at
the time of calling getUsage? To avoid this issue, we could get the usage on
the availability call and cache it, and the getUsage call would return this
cached value? But, requiring the availability call now is an even more
incompatible change, no?
{code}
ResourceTrackerProcessTee procTree = new <>();
if (procTree.isMemoryUsageAvailable()) {
procTree.getMemoryUsage();
}
{code}
And, how is the above user code snippet different from the one below:
{code}
ResourceTrackerProcessTee procTree = new <>();
procTree.getMemoryUsage();
{code}
What is the cost of breaking compat of this previously Private API? I have a
feeling it would be worth not making the API super-complicated.
I want to avoid fixing this in a hurry just to unblock the release. I am
willing to prioritize this, chat offline if need be, and solve it the right
way. If we think that is too slow, we could always revert YARN-3296 for 2.7.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14378303#comment-14378303 ] Anubhav Dhoot commented on YARN-3304: - I have a patch available now for YARN-3392 that shows how returning -1 would help implement it. If we take the other approach it would be good to validate that its still possible by ensuring those changes are done in this jira. Specifically in this example, we should add the boolean options here to ensure we can still do YARN-3392. We can then compare the two approaches to see which one is better. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377753#comment-14377753 ] Junping Du commented on YARN-3304: -- Thanks for comments, [~kasha]! bq. In previous releases, we have never called these APIs Public even if they were intended to be sub-classed. In my mind, this is the last opportunity to decide on what the API should do? I think consistent and reasonable return values should be given a higher priority over compatibility. Agree on the priority here. However, having consistent and reasonable return values doesn't have to break compatibility (or consistent behaviors) - just like the way I proposed above, we can consistently return resource value to 0 if they are unavailable and we have an additional flag to mark if resource is available or not. bq. I am okay with adding boolean methods to capture unavailability, but that seems a little overboard. Using -1 in the ResourceCalculatorProcessTree is okay by me. My concern was with logging this -1 value in the metrics. In either case, I would like for the container usage metrics to see if the usage is available before logging the same. I agree both ways can work. However, I think adding a boolean method sounds better, at least former. More important, it doesn't break any consistent behavior of previous releases. We don't need to break it if we don't have to. Isn't it? bq. Since it is not too much work or risk, I would prefer we fix both in 2.7. This is solely wearing my Apache hat on. My Cloudera hat doesn't really mind it being in 2.8 vs 2.7. My idea is simple here: a fast-moving, regular and predictable release train could benefit our community and ecosystem in many aspects. I also have other wish list that cannot catch up 2.7. When this patch get in, I am not sure if YARN-3392 is still a blocker for 2.7 and I would also prefer a fix rather than a pending JIRA there delay the release unnecessarily. [~vinodkv], [~kasha] and [~adhoot], what do you think? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377008#comment-14377008
]
Hadoop QA commented on YARN-3304:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706767/YARN-3304-v2.patch
against trunk revision 2c238ae.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7082//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7082//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3304-v2.patch, YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376973#comment-14376973 ] Karthik Kambatla commented on YARN-3304: bq. That's an incompatible change which sounds not necessary for now. In previous releases, we have never called these APIs Public even if they were intended to be sub-classed. In my mind, this is the last opportunity to decide on what the API should do? I think consistent and reasonable return values should be given a higher priority over compatibility. bq. May be we don't have to leverage "-1" in resource usage to distinguish unavailable case? e.g. we can have some boolean value to identify the resource is available or not which sounds more correct than using odd value like Karthik Kambatla mentioned before. I am okay with adding boolean methods to capture unavailability, but that seems a little overboard. Using -1 in the ResourceCalculatorProcessTree is okay by me. My concern was with logging this -1 value in the metrics. In either case, I would like for the container usage metrics to see if the usage is available before logging the same. bq. So I propose to go patch here (after fixing a minor test failure) in 2.7 given this is a blocker and we can fix YARN-3392 later in 2.8. Thoughts? Since it is not too much work or risk, I would prefer we fix both in 2.7. This is solely wearing my Apache hat on. My Cloudera hat doesn't really mind it being in 2.8 vs 2.7. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304-v2.patch, YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376920#comment-14376920 ] Junping Du commented on YARN-3304: -- Thanks [~adhoot] for comments! bq. If we use a default of zero we cannot distinguish when its unavailable versus zero usage. That will make the future "track the improvement to handle unavailable case later" near impossible to do. May be we don't have to leverage "-1" in resource usage to distinguish unavailable case? e.g. we can have some boolean value to identify the resource is available or not which sounds more correct than using odd value like [~ka...@cloudera.com] mentioned before. bq. I propose we make all the defaults consistently -1. That's an incompatible change which sounds not necessary for now. bq. I can fix the metrics as well to use this to implement tracking unavailable case. Opened YARN-3392 for that. Agree that we should have some fix on metrics side later. But even that, with changed all default values to -1, it is still a incompatible behavior with old released version. So I propose to go patch here (after fixing a minor test failure) in 2.7 given this is a blocker and we can fix YARN-3392 later in 2.8. Thoughts? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376695#comment-14376695 ] Anubhav Dhoot commented on YARN-3304: - Hi [~djp] [~vinodkv], If we use a default of zero we cannot distinguish when its unavailable versus zero usage. That will make the future "track the improvement to handle unavailable case later" near impossible to do. I propose we make all the defaults consistently -1. I can fix the metrics as well to use this to implement tracking unavailable case. Opened YARN-3392 for that > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > Attachments: YARN-3304.patch > > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[
https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376466#comment-14376466
]
Hadoop QA commented on YARN-3304:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12706680/YARN-3304.patch
against trunk revision 6ca1f12.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.yarn.util.TestProcfsBasedProcessTree
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/7078//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7078//console
This message is automatically generated.
> ResourceCalculatorProcessTree#getCpuUsagePercent default return value is
> inconsistent with other getters
>
>
> Key: YARN-3304
> URL: https://issues.apache.org/jira/browse/YARN-3304
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Junping Du
>Assignee: Karthik Kambatla
>Priority: Blocker
> Attachments: YARN-3304.patch
>
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for
> unavailable case while other resource metrics are return 0 in the same case
> which sounds inconsistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375867#comment-14375867 ] Junping Du commented on YARN-3304: -- Hi [~vinodkv], agree that we can fix it as 0 to keep consistent with other values. We can have a separated JIRA to track the improvement to handle unavailable case later which shouldn't block 2.7 releases. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372965#comment-14372965 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- Coming back to this again. I think the problem of unavailable metrics existing for other APIs too. Let's make this 0 and consistent, and track the unavailability issues separately? [~djp] / [~ka...@cloudera.com] / [~adhoot], can we get going? It's blocking 2.7, appreciate your response. Tx. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357536#comment-14357536 ] Vinod Kumar Vavilapalli commented on YARN-3304: --- How about we simplify this and throw an explicit exception when we think it is unavailable and let higher layers handle it appropriately when that happens? IAC, I'd like us to make some progress to unblock 2.7. Tx. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353272#comment-14353272 ] Anubhav Dhoot commented on YARN-3304: - The intention of setting -1 as was for this issue (distinguishing unavailable vs actually zero). Ideally we should prevent adding the metrics to collection until they are available. One possibility is doing it at ContainerMetrics#recordCpuUsage. Suggest investigating if this ideal case is achievable, and if not i am fine with making these 0 to be consistent. > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3304) ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters
[ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352962#comment-14352962 ] Junping Du commented on YARN-3304: -- Agree that negative value sounds very odd. However, if we are really failed to get cpu usage info, does value of 0 confuse user that metrics works fine and the cpu usage is very low here? > ResourceCalculatorProcessTree#getCpuUsagePercent default return value is > inconsistent with other getters > > > Key: YARN-3304 > URL: https://issues.apache.org/jira/browse/YARN-3304 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Junping Du >Assignee: Karthik Kambatla >Priority: Blocker > > Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for > unavailable case while other resource metrics are return 0 in the same case > which sounds inconsistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
