[ 
https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585942#comment-15585942
 ] 

Wei-Chiu Chuang edited comment on HDFS-10999 at 10/18/16 4:45 PM:
------------------------------------------------------------------

Just FYI, the fsck output in Hadoop 3 is of the following format. It does 
separate replicated blocks from erasure coded blocks. [~aw] how does this 
output look for you from an admin perspective?

{noformat}
FSCK started by weichiu (auth:SIMPLE) from /127.0.0.1 for path / at Tue Oct 18 
09:43:23 PDT 2016
/ <dir>
/striped <dir>
/striped/missing 393216 bytes, 1 block(s):  MISSING 1 blocks of total size 
393216 B
0. BP-406181414-172.16.1.88-1476808998435:blk_-9223372036854775792_1001 
len=393216 MISSING! Live_repl=5  
[DatanodeInfoWithStorage[127.0.0.1:56489,DS-1722aa0e-1ade-4286-a09f-fca1999b5370,DISK],
 
DatanodeInfoWithStorage[127.0.0.1:56481,DS-fc14e048-de5e-4dbd-986d-78161d37bcb2,DISK],
 
DatanodeInfoWithStorage[127.0.0.1:56485,DS-128523b5-086f-4b24-bb52-0426937debc3,DISK],
 
DatanodeInfoWithStorage[127.0.0.1:56493,DS-5dbac29b-1c82-40b2-b22b-7958915afd72,DISK],
 
DatanodeInfoWithStorage[127.0.0.1:56497,DS-a19f46a3-6357-42b7-84f3-9ac3ddf05ba1,DISK]]


Status: CORRUPT
 Number of data-nodes:  5
 Number of racks:               1
 Total dirs:                    2
 Total symlinks:                0

Replicated Blocks:
 Total size:    0 B
 Total files:   0
 Total blocks (validated):      0
 Minimally replicated blocks:   0
 Over-replicated blocks:        0
 Under-replicated blocks:       0
 Mis-replicated blocks:         0
 Default replication factor:    3
 Average block replication:     0.0
 Missing blocks:                0
 Corrupt blocks:                0
 Missing replicas:              0

Erasure Coded Block Groups:
 Total size:    393216 B
 Total files:   1
 Total block groups (validated):        1 (avg. block group size 393216 B)
  ********************************
  UNRECOVERABLE BLOCK GROUPS:   1 (100.0 %)
  CORRUPT FILES:        1
  MISSING BLOCK GROUPS: 1
  MISSING SIZE:         393216 B
  ********************************
 Minimally erasure-coded block groups:  0 (0.0 %)
 Over-erasure-coded block groups:       0 (0.0 %)
 Under-erasure-coded block groups:      0 (0.0 %)
 Unsatisfactory placement block groups: 0 (0.0 %)
 Default ecPolicy:              RS-DEFAULT-6-3-64k
 Average block group size:      5.0
 Missing block groups:          1
 Corrupt block groups:          0
 Missing internal blocks:       0 (0.0 %)
FSCK ended at Tue Oct 18 09:43:23 PDT 2016 in 2 milliseconds


The filesystem under path '/' is CORRUPT

{noformat}


was (Author: jojochuang):
Just FYI, the fsck output in Hadoop 3 is of the following format. It does 
separate replicated blocks from erasure coded blocks. [~aw] how does this 
output look for you from an admin perspective?

{noformat}
FSCK started by weichiu (auth:SIMPLE) from /127.0.0.1 for path / at Tue Oct 18 
09:37:00 PDT 2016

/striped/corrupted: CORRUPT blockpool BP-921842435-172.16.1.88-1476808612846 
block blk_-9223372036854775792

/striped/corrupted: CORRUPT 1 blocks of total size 393216 B.
Status: CORRUPT
 Number of data-nodes:  9
 Number of racks:               1
 Total dirs:                    2
 Total symlinks:                0

Replicated Blocks:
 Total size:    0 B
 Total files:   0
 Total blocks (validated):      0
 Minimally replicated blocks:   0
 Over-replicated blocks:        0
 Under-replicated blocks:       0
 Mis-replicated blocks:         0
 Default replication factor:    3
 Average block replication:     0.0
 Missing blocks:                0
 Corrupt blocks:                0
 Missing replicas:              0

Erasure Coded Block Groups:
 Total size:    393216 B
 Total files:   1
 Total block groups (validated):        1 (avg. block group size 393216 B)
  ********************************
  UNRECOVERABLE BLOCK GROUPS:   1 (100.0 %)
  CORRUPT FILES:        1
  CORRUPT BLOCK GROUPS:         1
  CORRUPT SIZE:         393216 B
  ********************************
 Minimally erasure-coded block groups:  0 (0.0 %)
 Over-erasure-coded block groups:       0 (0.0 %)
 Under-erasure-coded block groups:      0 (0.0 %)
 Unsatisfactory placement block groups: 0 (0.0 %)
 Default ecPolicy:              RS-DEFAULT-6-3-64k
 Average block group size:      5.0
 Missing block groups:          0
 Corrupt block groups:          1
 Missing internal blocks:       0 (0.0 %)
FSCK ended at Tue Oct 18 09:37:00 PDT 2016 in 2 milliseconds


The filesystem under path '/' is CORRUPT

{noformat}

> Use more generic "low redundancy" blocks instead of "under replicated" blocks
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-10999
>                 URL: https://issues.apache.org/jira/browse/HDFS-10999
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Wei-Chiu Chuang
>            Assignee: Yuanbo Liu
>              Labels: supportability
>
> Per HDFS-9857, it seems in the Hadoop 3 world, people prefer the more generic 
> term "low redundancy" to the old-fashioned "under replicated". But this term 
> is still being used in messages in several places, such as web ui, dfsadmin 
> and fsck. We should probably change them to avoid confusion.
> File this jira to discuss it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to