[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038283#comment-16038283 ] Brahma Reddy Battula commented on HDFS-7933: bq. If you have time please kindly review HDFS-11743 done. > fsck should also report decommissioning replicas. > -- > > Key: HDFS-7933 > URL: https://issues.apache.org/jira/browse/HDFS-7933 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Jitendra Nath Pandey >Assignee: Xiaoyu Yao > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, > HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch > > > Fsck doesn't count replicas that are on decommissioning nodes. If a block has > all replicas on the decommissioning nodes, it will be marked as missing, > which is alarming for the admins, although the system will replicate them > before nodes are decommissioned. > Fsck output should also show decommissioning replicas along with the live > replicas. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038247#comment-16038247
]
Zhe Zhang commented on HDFS-7933:
-
Thanks for the feedback [~brahmareddy]. I decided to revert only the {{fsck}}
logic and keep the logic from this JIRA to differentiate {{decommissioning}}
and {{decommissioned}} replicas. If you have time please kindly review
HDFS-11743. Thanks!
> fsck should also report decommissioning replicas.
> --
>
> Key: HDFS-7933
> URL: https://issues.apache.org/jira/browse/HDFS-7933
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Jitendra Nath Pandey
>Assignee: Xiaoyu Yao
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
> Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch,
> HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch
>
>
> Fsck doesn't count replicas that are on decommissioning nodes. If a block has
> all replicas on the decommissioning nodes, it will be marked as missing,
> which is alarming for the admins, although the system will replicate them
> before nodes are decommissioned.
> Fsck output should also show decommissioning replicas along with the live
> replicas.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038243#comment-16038243
]
Brahma Reddy Battula commented on HDFS-7933:
bq.Actually can we make this new fsck output configurable, instead of reverting
this patch?
Yes,making it configurable based on extra option from {{fsck}} is better. if
you just escape the output,it will be hidden..? Sorry for delayed response.
> fsck should also report decommissioning replicas.
> --
>
> Key: HDFS-7933
> URL: https://issues.apache.org/jira/browse/HDFS-7933
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Jitendra Nath Pandey
>Assignee: Xiaoyu Yao
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
> Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch,
> HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch
>
>
> Fsck doesn't count replicas that are on decommissioning nodes. If a block has
> all replicas on the decommissioning nodes, it will be marked as missing,
> which is alarming for the admins, although the system will replicate them
> before nodes are decommissioned.
> Fsck output should also show decommissioning replicas along with the live
> replicas.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037822#comment-16037822 ] Zhe Zhang commented on HDFS-7933: - I agree with the above suggestion from [~shv]. > fsck should also report decommissioning replicas. > -- > > Key: HDFS-7933 > URL: https://issues.apache.org/jira/browse/HDFS-7933 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Jitendra Nath Pandey >Assignee: Xiaoyu Yao > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, > HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch > > > Fsck doesn't count replicas that are on decommissioning nodes. If a block has > all replicas on the decommissioning nodes, it will be marked as missing, > which is alarming for the admins, although the system will replicate them > before nodes are decommissioned. > Fsck output should also show decommissioning replicas along with the live > replicas. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035587#comment-16035587 ] Konstantin Shvachko commented on HDFS-7933: --- May be we should only fix the reporting, that is make it report as before, but keep the compute logic in place. Re-purpose HDFS-11743 to do just that? > fsck should also report decommissioning replicas. > -- > > Key: HDFS-7933 > URL: https://issues.apache.org/jira/browse/HDFS-7933 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Jitendra Nath Pandey >Assignee: Xiaoyu Yao > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, > HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch > > > Fsck doesn't count replicas that are on decommissioning nodes. If a block has > all replicas on the decommissioning nodes, it will be marked as missing, > which is alarming for the admins, although the system will replicate them > before nodes are decommissioned. > Fsck output should also show decommissioning replicas along with the live > replicas. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994057#comment-15994057
]
Zhe Zhang commented on HDFS-7933:
-
Thanks [~brahmareddy]. I filed HDFS-11743.
Actually can we make this new fsck output configurable, instead of reverting
this patch? Differentiating {{decommissioned}} and {{decommissioning}} is also
useful in HDFS-11499.
> fsck should also report decommissioning replicas.
> --
>
> Key: HDFS-7933
> URL: https://issues.apache.org/jira/browse/HDFS-7933
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Jitendra Nath Pandey
>Assignee: Xiaoyu Yao
> Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
> Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch,
> HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch
>
>
> Fsck doesn't count replicas that are on decommissioning nodes. If a block has
> all replicas on the decommissioning nodes, it will be marked as missing,
> which is alarming for the admins, although the system will replicate them
> before nodes are decommissioned.
> Fsck output should also show decommissioning replicas along with the live
> replicas.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986129#comment-15986129 ] Brahma Reddy Battula commented on HDFS-7933: I think, you should revert before 2.7.4 release. > fsck should also report decommissioning replicas. > -- > > Key: HDFS-7933 > URL: https://issues.apache.org/jira/browse/HDFS-7933 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Jitendra Nath Pandey >Assignee: Xiaoyu Yao > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, > HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch > > > Fsck doesn't count replicas that are on decommissioning nodes. If a block has > all replicas on the decommissioning nodes, it will be marked as missing, > which is alarming for the admins, although the system will replicate them > before nodes are decommissioned. > Fsck output should also show decommissioning replicas along with the live > replicas. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985310#comment-15985310 ] Zhe Zhang commented on HDFS-7933: - [~brahmareddy] Sorry missed your question. You are right, it was my mistake to backport the incompatible change. Will revert from branch-2.7. > fsck should also report decommissioning replicas. > -- > > Key: HDFS-7933 > URL: https://issues.apache.org/jira/browse/HDFS-7933 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Jitendra Nath Pandey >Assignee: Xiaoyu Yao > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, > HDFS-7933.02.patch, HDFS-7933.03.patch, HDFS-7933-branch-2.7.00.patch > > > Fsck doesn't count replicas that are on decommissioning nodes. If a block has > all replicas on the decommissioning nodes, it will be marked as missing, > which is alarming for the admins, although the system will replicate them > before nodes are decommissioned. > Fsck output should also show decommissioning replicas along with the live > replicas. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15545413#comment-15545413 ] Brahma Reddy Battula commented on HDFS-7933: As per versioning rule [here|http://hadoop.apache.org/versioning.html],I think, we should not add incompatible changes to dot and minor releases..? > fsck should also report decommissioning replicas. > -- > > Key: HDFS-7933 > URL: https://issues.apache.org/jira/browse/HDFS-7933 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Reporter: Jitendra Nath Pandey >Assignee: Xiaoyu Yao > Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1 > > Attachments: HDFS-7933-branch-2.7.00.patch, HDFS-7933.00.patch, > HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch > > > Fsck doesn't count replicas that are on decommissioning nodes. If a block has > all replicas on the decommissioning nodes, it will be marked as missing, > which is alarming for the admins, although the system will replicate them > before nodes are decommissioned. > Fsck output should also show decommissioning replicas along with the live > replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423433#comment-15423433
]
Zhe Zhang commented on HDFS-7933:
-
{{TestOpenFilesWithSnapshot}} is broken after HDFS-9696. I verified other
reported failures. I just backported the change to branch-2.7.
> fsck should also report decommissioning replicas.
> --
>
> Key: HDFS-7933
> URL: https://issues.apache.org/jira/browse/HDFS-7933
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Jitendra Nath Pandey
>Assignee: Xiaoyu Yao
> Fix For: 2.8.0, 2.7.4
>
> Attachments: HDFS-7933-branch-2.7.00.patch, HDFS-7933.00.patch,
> HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch
>
>
> Fsck doesn't count replicas that are on decommissioning nodes. If a block has
> all replicas on the decommissioning nodes, it will be marked as missing,
> which is alarming for the admins, although the system will replicate them
> before nodes are decommissioned.
> Fsck output should also show decommissioning replicas along with the live
> replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15423324#comment-15423324
]
Hadoop QA commented on HDFS-7933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m
53s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
0s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
14s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
1s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 25s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2320 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
59s{color} | {color:red} The patch 221 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 57m 42s{color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
19s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}162m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_101 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.namenode.TestCacheDirectives |
| | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot |
| JDK v1.7.0_101 Failed junit tests |
hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| |
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421884#comment-15421884
]
Hadoop QA commented on HDFS-7933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 13m
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
6s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
4s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 24s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1690 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
46s{color} | {color:red} The patch 221 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 2s{color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
22s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 48s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_101 Failed junit tests | hadoop.tools.TestJMXGet |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| JDK v1.7.0_101 Failed junit tests | hadoop.tools.TestJMXGet |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:c420dfe |
| JIRA Patch URL |
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421877#comment-15421877
]
Hadoop QA commented on HDFS-7933:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
45s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
30s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
59s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
0s{color} | {color:green} branch-2.7 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} branch-2.7 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
39s{color} | {color:green} branch-2.7 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 24s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1691 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
45s{color} | {color:red} The patch 221 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 32s{color}
| {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_101. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
21s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}136m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_101 Failed junit tests | hadoop.hdfs.TestBlockStoragePolicy |
| | hadoop.hdfs.TestDatanodeRegistration |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
| | hadoop.hdfs.server.datanode.TestIncrementalBrVariations |
| | hadoop.tools.TestJMXGet |
| JDK v1.7.0_101 Failed junit tests | hadoop.hdfs.web.TestWebHdfsTokens |
| | hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots |
\\
\\
|| Subsystem || Report/Notes
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491410#comment-14491410 ] Hudson commented on HDFS-7933: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #161 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/161/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491413#comment-14491413 ] Hudson commented on HDFS-7933: -- FAILURE: Integrated in Hadoop-Yarn-trunk #895 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/895/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491433#comment-14491433 ] Hudson commented on HDFS-7933: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2093 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2093/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491434#comment-14491434 ] Hudson commented on HDFS-7933: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #152 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/152/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491508#comment-14491508 ] Hudson commented on HDFS-7933: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #162 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/162/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491531#comment-14491531 ] Hudson commented on HDFS-7933: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2111 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2111/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491189#comment-14491189 ] Hudson commented on HDFS-7933: -- FAILURE: Integrated in Hadoop-trunk-Commit #7570 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7570/]) HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Fix For: 2.8.0 Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488267#comment-14488267
]
Hadoop QA commented on HDFS-7933:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch
against trunk revision 63c659d.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancer
org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
The following test timeouts occurred in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.TestBackupNode
org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10234//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10234//console
This message is automatically generated.
fsck should also report decommissioning replicas.
--
Key: HDFS-7933
URL: https://issues.apache.org/jira/browse/HDFS-7933
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Jitendra Nath Pandey
Assignee: Xiaoyu Yao
Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch,
HDFS-7933.02.patch, HDFS-7933.03.patch
Fsck doesn't count replicas that are on decommissioning nodes. If a block has
all replicas on the decommissioning nodes, it will be marked as missing,
which is alarming for the admins, although the system will replicate them
before nodes are decommissioned.
Fsck output should also show decommissioning replicas along with the live
replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14488277#comment-14488277 ] Chris Nauroth commented on HDFS-7933: - It looks like that test run got hurt by some of the classpath problems we've seen. I submitted another test run. fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Bug Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch, HDFS-7933.02.patch, HDFS-7933.03.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14485837#comment-14485837
]
Chris Nauroth commented on HDFS-7933:
-
Hi Xiaoyu. This patch looks good. Here are a few comments:
# In {{BlockManager#chooseSourceDatanode}}, should the {{decommissioning}}
counter be incremented by {{countableReplica}}, like for the other counters?
{code}
if ((nodesCorrupt != null) (nodesCorrupt.contains(node)))
corrupt += countableReplica;
else if (node.isDecommissionInProgress()) {
decommissioning++;
} else if (node.isDecommissioned()) {
decommissioned += countableReplica;
{code}
# Could the deprecated {{NumberReplicas#decommissionedReplicas}} be implemented
to forward the call to {{decommissionedAndDecommissioning}}? If so, then we
can eliminate the {{decommissionedReplicas}} member variable.
# The new test in {{TestFsck}} appears to create a {{StringBuilder}}, append
data to it, and then never use it for anything. Can it be removed?
fsck should also report decommissioning replicas.
--
Key: HDFS-7933
URL: https://issues.apache.org/jira/browse/HDFS-7933
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Jitendra Nath Pandey
Assignee: Xiaoyu Yao
Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch,
HDFS-7933.02.patch
Fsck doesn't count replicas that are on decommissioning nodes. If a block has
all replicas on the decommissioning nodes, it will be marked as missing,
which is alarming for the admins, although the system will replicate them
before nodes are decommissioned.
Fsck output should also show decommissioning replicas along with the live
replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388386#comment-14388386
]
Hadoop QA commented on HDFS-7933:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708365/HDFS-7933.02.patch
against trunk revision 85dc3c1.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/10126//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HDFS-Build/10126//console
This message is automatically generated.
fsck should also report decommissioning replicas.
--
Key: HDFS-7933
URL: https://issues.apache.org/jira/browse/HDFS-7933
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Jitendra Nath Pandey
Assignee: Xiaoyu Yao
Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch,
HDFS-7933.02.patch
Fsck doesn't count replicas that are on decommissioning nodes. If a block has
all replicas on the decommissioning nodes, it will be marked as missing,
which is alarming for the admins, although the system will replicate them
before nodes are decommissioned.
Fsck output should also show decommissioning replicas along with the live
replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366045#comment-14366045 ] Jitendra Nath Pandey commented on HDFS-7933: [~xyao], one of the key issues is that NumReplicas doesn't distinguish between decommissioned or decommissioning replicas, and is initialized with the sum of decommissioning and decommissioned replicas. I think fsck should report decommissioning and decommissioned replicas separately to provide the complete picture. fsck should also report decommissioning replicas. -- Key: HDFS-7933 URL: https://issues.apache.org/jira/browse/HDFS-7933 Project: Hadoop HDFS Issue Type: Bug Components: namenode Reporter: Jitendra Nath Pandey Assignee: Xiaoyu Yao Attachments: HDFS-7933.00.patch, HDFS-7933.01.patch Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned. Fsck output should also show decommissioning replicas along with the live replicas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364388#comment-14364388
]
Hadoop QA commented on HDFS-7933:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704903/HDFS-7933.00.patch
against trunk revision 2681ed9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestClientReportBadBlock
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/9905//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//console
This message is automatically generated.
fsck should also report decommissioning replicas.
--
Key: HDFS-7933
URL: https://issues.apache.org/jira/browse/HDFS-7933
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Jitendra Nath Pandey
Assignee: Xiaoyu Yao
Attachments: HDFS-7933.00.patch
Fsck doesn't count replicas that are on decommissioning nodes. If a block has
all replicas on the decommissioning nodes, it will be marked as missing,
which is alarming for the admins, although the system will replicate them
before nodes are decommissioned.
Fsck output should also show decommissioning replicas along with the live
replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.
[
https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364096#comment-14364096
]
Xiaoyu Yao commented on HDFS-7933:
--
Sample output from unit test during decommission:
{code}
Status: HEALTHY
Total size:512 B
Total dirs:0
Total files: 1
Total symlinks:0
Total blocks (validated): 1 (avg. block size 512 B)
Minimally replicated blocks: 1 (100.0 %)
Over-replicated blocks:0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor:1
Average block replication: 2.0
Corrupt blocks:0
Missing replicas: 0
DecommissioningReplicas: 1 (0.0 %)
Number of data-nodes: 2
Number of racks: 2
FSCK ended at Mon Mar 16 15:04:21 PDT 2015 in 1 milliseconds
The filesystem under path '/testfile' is HEALTHY
{code}
fsck should also report decommissioning replicas.
--
Key: HDFS-7933
URL: https://issues.apache.org/jira/browse/HDFS-7933
Project: Hadoop HDFS
Issue Type: Bug
Components: namenode
Reporter: Jitendra Nath Pandey
Assignee: Xiaoyu Yao
Attachments: HDFS-7933.00.patch
Fsck doesn't count replicas that are on decommissioning nodes. If a block has
all replicas on the decommissioning nodes, it will be marked as missing,
which is alarming for the admins, although the system will replicate them
before nodes are decommissioned.
Fsck output should also show decommissioning replicas along with the live
replicas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
