[jira] [Commented] (HDFS-11054) Suppress verbose log message in BlockPlacementPolicyDefault
[
https://issues.apache.org/jira/browse/HDFS-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607261#comment-15607261
]
Hudson commented on HDFS-11054:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10680 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/10680/])
HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. (arp:
rev 9dce9011ad211bf35fe2538ded0f19d68f5c16d2)
* (edit)
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
> Suppress verbose log message in BlockPlacementPolicyDefault
> ---
>
> Key: HDFS-11054
> URL: https://issues.apache.org/jira/browse/HDFS-11054
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Arpit Agarwal
>Assignee: Chen Liang
> Fix For: 2.8.0, 3.0.0-alpha2
>
> Attachments: HDFS-11054.001.patch, HDFS-11054.002.patch
>
>
> We saw an instance where this log message was generated millions of times in
> a short time interval, slowing the NameNode to a crawl. It can probably be
> changed to log at DEBUG level instead.
> {code}
> if (cur == null) {
> LOG.warn("No excess replica can be found. excessTypes: "+excessTypes+
> ". moreThanOne: "+moreThanOne+". exactlyOne: "+exactlyOne+".");
> break;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11054) Suppress verbose log message in BlockPlacementPolicyDefault
[
https://issues.apache.org/jira/browse/HDFS-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606839#comment-15606839
]
Hadoop QA commented on HDFS-11054:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 2s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 84m 16s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
| | hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
| | hadoop.hdfs.web.TestWebHDFS |
| Timed out junit tests | org.apache.hadoop.hdfs.TestSmallBlock |
| | org.apache.hadoop.hdfs.TestPread |
| | org.apache.hadoop.hdfs.TestSafeMode |
| | org.apache.hadoop.hdfs.TestDFSUpgradeFromImage |
| | org.apache.hadoop.hdfs.server.mover.TestStorageMover |
| | org.apache.hadoop.hdfs.server.mover.TestMover |
| | org.apache.hadoop.hdfs.TestLeaseRecoveryStriped |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-11054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835213/HDFS-11054.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e40d78ba2bae 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29
20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 86c735b |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17284/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17284/testReport/ |
| modules | C:
[jira] [Commented] (HDFS-11054) Suppress verbose log message in BlockPlacementPolicyDefault
[
https://issues.apache.org/jira/browse/HDFS-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606816#comment-15606816
]
Xiaobing Zhou commented on HDFS-11054:
--
Thanks [~vagarychen] for the fix. +1 the patch v002.
> Suppress verbose log message in BlockPlacementPolicyDefault
> ---
>
> Key: HDFS-11054
> URL: https://issues.apache.org/jira/browse/HDFS-11054
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Arpit Agarwal
>Assignee: Chen Liang
> Attachments: HDFS-11054.001.patch, HDFS-11054.002.patch
>
>
> We saw an instance where this log message was generated millions of times in
> a short time interval, slowing the NameNode to a crawl. It can probably be
> changed to log at DEBUG level instead.
> {code}
> if (cur == null) {
> LOG.warn("No excess replica can be found. excessTypes: "+excessTypes+
> ". moreThanOne: "+moreThanOne+". exactlyOne: "+exactlyOne+".");
> break;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11054) Suppress verbose log message in BlockPlacementPolicyDefault
[
https://issues.apache.org/jira/browse/HDFS-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606761#comment-15606761
]
Hadoop QA commented on HDFS-11054:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 44m 40s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 63m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Timed out junit tests | org.apache.hadoop.hdfs.server.datanode.TestHSync |
| | org.apache.hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade |
| | org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter |
| | org.apache.hadoop.cli.TestCryptoAdminCLI |
| |
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation |
| | org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-11054 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12835211/HDFS-11054.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3ff4424b29b4 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 86c735b |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/17283/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/17283/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/17283/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT
[jira] [Commented] (HDFS-11054) Suppress verbose log message in BlockPlacementPolicyDefault
[
https://issues.apache.org/jira/browse/HDFS-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606621#comment-15606621
]
Arpit Agarwal commented on HDFS-11054:
--
+1 pending Jenkins.
Thanks for the quick fix [~vagarychen]! cc [~szetszwo] who reported this.
> Suppress verbose log message in BlockPlacementPolicyDefault
> ---
>
> Key: HDFS-11054
> URL: https://issues.apache.org/jira/browse/HDFS-11054
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Arpit Agarwal
>Assignee: Chen Liang
> Attachments: HDFS-11054.001.patch
>
>
> We saw an instance where this log message was generated millions of times in
> a short time interval, slowing the NameNode to a crawl. It can probably be
> changed to log at DEBUG level instead.
> {code}
> if (cur == null) {
> LOG.warn("No excess replica can be found. excessTypes: "+excessTypes+
> ". moreThanOne: "+moreThanOne+". exactlyOne: "+exactlyOne+".");
> break;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11054) Suppress verbose log message in BlockPlacementPolicyDefault
[
https://issues.apache.org/jira/browse/HDFS-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606623#comment-15606623
]
Xiaoyu Yao commented on HDFS-11054:
---
Thanks [~arpitagarwal] for reporting the issue and [~vagarychen] for posting
the fix. NIT: There is a minor tab/space issue in the 2nd line of the changed
log. +1 pending Jenkins.
> Suppress verbose log message in BlockPlacementPolicyDefault
> ---
>
> Key: HDFS-11054
> URL: https://issues.apache.org/jira/browse/HDFS-11054
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Arpit Agarwal
>Assignee: Chen Liang
> Attachments: HDFS-11054.001.patch
>
>
> We saw an instance where this log message was generated millions of times in
> a short time interval, slowing the NameNode to a crawl. It can probably be
> changed to log at DEBUG level instead.
> {code}
> if (cur == null) {
> LOG.warn("No excess replica can be found. excessTypes: "+excessTypes+
> ". moreThanOne: "+moreThanOne+". exactlyOne: "+exactlyOne+".");
> break;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
