[
https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15522116#comment-15522116
]
Akira Ajisaka commented on HDFS-10826:
--------------------------------------
Mostly looks good to me. Some comments from me:
* Would you fix checkstyle warnings?
* Would you update the following comment?
{code:title=NamenodeFsck.java}
if (isMissing && !isCorrupt) {
// If the block is corrupted, it means all its available replicas are
// corrupted. We don't mark it as missing given these available replicas
// might still be accessible as the block might be incorrectly marked as
// corrupted by client machines.
{code}
* In the following code, why do you need to run fsck multiple times?
{code}
while (!outStr.contains("UNRECOVERABLE BLOCK GROUPS")) {
bStream = new ByteArrayOutputStream();
out = new PrintStream(bStream, true);
ToolRunner.run(new DFSck(conf, out), new String[]{ "/" });
outStr = bStream.toString();
}
{code}
this code can cause infinite loop, so I'm thinking we need to set a retry
limit. Maybe we can use GenericTestUtils.waitFor method for such situation.
* {{Thread.sleep}} is missing in the while loop in testFsckMissingECFile.
> The result of fsck should be CRITICAL when there are unrecoverable ec block
> groups.
> -----------------------------------------------------------------------------------
>
> Key: HDFS-10826
> URL: https://issues.apache.org/jira/browse/HDFS-10826
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Reporter: Takanobu Asanuma
> Assignee: Takanobu Asanuma
> Attachments: HDFS-10826.2.patch, HDFS-10826.WIP.1.patch
>
>
> For RS-6-3, when there is one ec block group and
> 1) 0~3 out of 9 internal blocks are missing, the result of fsck is HEALTY.
> 2) 4~8 out of 9 internal blocks are missing, the result of fsck is HEALTY.
> {noformat}
> Erasure Coded Block Groups:
> Total size: 536870912 B
> Total files: 1
> Total block groups (validated): 1 (avg. block group size 536870912 B)
> ********************************
> UNRECOVERABLE BLOCK GROUPS: 1 (100.0 %)
> ********************************
> Minimally erasure-coded block groups: 0 (0.0 %)
> Over-erasure-coded block groups: 0 (0.0 %)
> Under-erasure-coded block groups: 1 (100.0 %)
> Unsatisfactory placement block groups: 0 (0.0 %)
> Default ecPolicy: RS-DEFAULT-6-3-64k
> Average block group size: 5.0
> Missing block groups: 0
> Corrupt block groups: 0
> Missing internal blocks: 4 (44.444443 %)
> FSCK ended at Wed Aug 31 13:42:05 JST 2016 in 4 milliseconds
> The filesystem under path '/' is HEALTHY
> {noformat}
> 3) 9 out of 9 internal blocks are missing, the result of fsck is CRITICAL.
> (Because it is regarded as a missing block group.)
> In case 2), the result should be CRITICAL since the ec block group is
> unrecoverable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
