[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662903#comment-14662903 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #281 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/281/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662906#comment-14662906 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1011 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1011/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/CHANGES.txt Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662991#comment-14662991 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2208 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2208/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662998#comment-14662998 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #270 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/270/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663017#comment-14663017 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #278 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/278/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663022#comment-14663022 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2227 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2227/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662033#comment-14662033 ] Wangda Tan commented on YARN-3966: -- Committing... Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662179#comment-14662179 ] Hudson commented on YARN-3966: -- FAILURE: Integrated in Hadoop-trunk-Commit #8278 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8278/]) YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) (wangda: rev 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java * hadoop-yarn-project/CHANGES.txt Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Fix For: 2.8.0 Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660768#comment-14660768 ] Jian He commented on YARN-3966: --- I went through the code path and cleaned up some logs as needed. Uploaded a patch to do that. Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660770#comment-14660770 ] Xuan Gong commented on YARN-3966: - Instead of removing the logs, change the log level from info to debug ? Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660820#comment-14660820 ] Wangda Tan commented on YARN-3966: -- Patch LGTM, +1. Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14660819#comment-14660819 ] Jian He commented on YARN-3966: --- The logs I removed are redundant logs with others. Apart from that, some logs are also moved from info to debug. Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He Attachments: YARN-3966.1.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[
https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14661124#comment-14661124
]
Hadoop QA commented on YARN-3966:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 16s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:red}-1{color} | tests included | 0m 0s | The patch doesn't appear
to include any new or modified tests. Please justify why no new tests are
needed for this patch. Also please list what manual steps were performed to
verify this patch. |
| {color:green}+1{color} | javac | 8m 23s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 27s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 57s | The applied patch generated 2
new checkstyle issues (total was 307, now 302). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 25s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 34s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:red}-1{color} | yarn tests | 53m 5s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 94m 9s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12749132/YARN-3966.1.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 8572a5a |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/8783/artifact/patchprocess/diffcheckstylehadoop-yarn-server-resourcemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8783/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8783/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8783/console |
This message was automatically generated.
Fix excessive loggings in CapacityScheduler
---
Key: YARN-3966
URL: https://issues.apache.org/jira/browse/YARN-3966
Project: Hadoop YARN
Issue Type: Bug
Reporter: Jian He
Assignee: Jian He
Attachments: YARN-3966.1.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639319#comment-14639319 ] Xuan Gong commented on YARN-3966: - How about that we combine two options together ? I think that potentially for all the plugin components in RM, we need to separate their logs from RM logs. Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3966) Fix excessive loggings in CapacityScheduler
[ https://issues.apache.org/jira/browse/YARN-3966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639261#comment-14639261 ] Junping Du commented on YARN-3966: -- +1. Some messages are not necessary to be in INFO level which mess up RM logs. I think we have two options here: 1. Move scheduler log message out of RM logs. 2. Downgrade some scheduler log level from INFO to DEBUG. Fix excessive loggings in CapacityScheduler --- Key: YARN-3966 URL: https://issues.apache.org/jira/browse/YARN-3966 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He -- This message was sent by Atlassian JIRA (v6.3.4#6332)
