[ 
https://issues.apache.org/jira/browse/HDFS-10975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587360#comment-15587360
 ] 

Takanobu Asanuma commented on HDFS-10975:
-----------------------------------------

Thank you for reviewing, [~jojochuang]!

In the case of a replication block, if the state of the block is 
low-redundancy(not missing), the missing replicas are counted as "Missing 
replicas".

{noformat}
/replication/missing:  Under replicated 
BP-442454012-172.16.165.209-1476839633883:blk_1073741825_1001. Target Replicas 
is 3 but found 1 live replica(s), 0 decommissioned replica(s) and 0 
decommissioning replica(s).
Status: HEALTHY
 Number of data-nodes:  1
 Number of racks:               1
 Total dirs:                    2
 Total symlinks:                0

Replicated Blocks:
 Total size:    1024 B
 Total files:   1
 Total blocks (validated):      1 (avg. block size 1024 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       1 (100.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     1.0
 Missing blocks:                0
 Corrupt blocks:                0
 Missing replicas:              2 (66.666664 %)
 {noformat}

However, if the block is missing, which means all of the replicas of the block 
are missing, they are not counted as "Missing replicas".
{noformat}
/replication/missing: MISSING 1 blocks of total size 1024 B.
Status: CORRUPT
 Number of data-nodes:  0
 Number of racks:               0
 Total dirs:                    2
 Total symlinks:                0

Replicated Blocks:
 Total size:    1024 B
 Total files:   1
 Total blocks (validated):      1 (avg. block size 1024 B)
  ********************************
  UNDER MIN REPL'D BLOCKS:      1 (100.0 %)
  dfs.namenode.replication.min: 1
  CORRUPT FILES:        1
  MISSING BLOCKS:       1
  MISSING SIZE:         1024 B
  ********************************
 Minimally replicated blocks:   0 (0.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     0.0
 Missing blocks:                1
 Corrupt blocks:                0
 Missing replicas:              0
 {noformat}

If we synchronize the fsck result of ec and replication, "Missing internal 
blocks" should not be counted when the state is unrecoverable. What do you 
think?

> fsck -list-corruptfileblocks does not report corrupt EC files
> -------------------------------------------------------------
>
>                 Key: HDFS-10975
>                 URL: https://issues.apache.org/jira/browse/HDFS-10975
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Wei-Chiu Chuang
>            Assignee: Takanobu Asanuma
>         Attachments: HDFS-10975.1.patch
>
>
> HDFS-10826 fix fsck for corrupt EC files if no parameters are specified.
> However, if I change the test case added in HDFS-10826 
> (TestFsck#testFsckCorruptECFile) to run "fsck -list-corruptfileblocks", the 
> same test test failed because fsck reports no corrupt files. 
> Interestingly, if I run "fsck -files -blocks -replicaDetails" then the test 
> passed and shows the corrupt file.
> Need to fix the discrepancy.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to