[
https://issues.apache.org/jira/browse/HDFS-10826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15534077#comment-15534077
]
Wei-Chiu Chuang commented on HDFS-10826:
----------------------------------------
Hello [~tasanuma0829] thanks for your contribution and [~ajisakaa] and
[~jingzhao] for the review. I am late to review this patch but if you don't
mind there are a little minor improvement that can be made:
in TestFsck
{code}
// make an unrecoverable ec file with corrupted blocks
for(int i = 0; i < 4; i++) {
{code}
* Instead of hardcode "4", it would be more meaningful to use parityBlocks+1 I
assume?
{code}
// Read the file to trigger reportBadBlocks
try {
IOUtils.copyBytes(fs.open(file), new
IOUtils.NullOutputStream(), conf,
true);
} catch (IOException ie) {
// Ignore exception
}
{code}
* Would it be possible to verify the exception thrown is expected when // Read
the file to trigger reportBadBlocks?
{code}
if (fs != null) {
try {
fs.close();
} catch (Exception e) {
}
}
if (cluster != null) {
cluster.shutdown();
}
{code}
You should just let the exception be thrown instead of catch and ignore it.
A better improvement could use @Before @After annotation to initialize/close
cluster and fs object. Then you do not even need to try...catch.
{code:title=waitForUnrecoverableBlockGroup()}
catch (Exception e) {
FSImage.LOG.error("Exception caught", e);
Assert.fail("Caught unexpected exception.");
}
{code}
I wonder if there is a more appropriate log object than FSImage.
> The result of fsck should be CRITICAL when there are unrecoverable ec block
> groups.
> -----------------------------------------------------------------------------------
>
> Key: HDFS-10826
> URL: https://issues.apache.org/jira/browse/HDFS-10826
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Reporter: Takanobu Asanuma
> Assignee: Takanobu Asanuma
> Attachments: HDFS-10826.2.patch, HDFS-10826.3.patch,
> HDFS-10826.4.patch, HDFS-10826.WIP.1.patch
>
>
> For RS-6-3, when there is one ec block group and
> 1) 0~3 out of 9 internal blocks are missing, the result of fsck is HEALTY.
> 2) 4~8 out of 9 internal blocks are missing, the result of fsck is HEALTY.
> {noformat}
> Erasure Coded Block Groups:
> Total size: 536870912 B
> Total files: 1
> Total block groups (validated): 1 (avg. block group size 536870912 B)
> ********************************
> UNRECOVERABLE BLOCK GROUPS: 1 (100.0 %)
> ********************************
> Minimally erasure-coded block groups: 0 (0.0 %)
> Over-erasure-coded block groups: 0 (0.0 %)
> Under-erasure-coded block groups: 1 (100.0 %)
> Unsatisfactory placement block groups: 0 (0.0 %)
> Default ecPolicy: RS-DEFAULT-6-3-64k
> Average block group size: 5.0
> Missing block groups: 0
> Corrupt block groups: 0
> Missing internal blocks: 4 (44.444443 %)
> FSCK ended at Wed Aug 31 13:42:05 JST 2016 in 4 milliseconds
> The filesystem under path '/' is HEALTHY
> {noformat}
> 3) 9 out of 9 internal blocks are missing, the result of fsck is CRITICAL.
> (Because it is regarded as a missing block group.)
> In case 2), the result should be CRITICAL since the ec block group is
> unrecoverable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
