more strange , in my HDFS cluster ,every block has three replicas,but i find some one has ten replicas ,why?
# sudo -u hdfs hadoop fs -ls /data/hisstage/helen/.staging/job_1385542328307_0915 Found 5 items -rw-r--r-- 3 helen hadoop 7 2013-11-29 14:01 /data/hisstage/helen/.staging/job_1385542328307_0915/appTokens -rw-r--r-- 10 helen hadoop 2977839 2013-11-29 14:01 /data/hisstage/helen/.staging/job_1385542328307_0915/job.jar -rw-r--r-- 10 helen hadoop 3696 2013-11-29 14:01 /data/hisstage/helen/.staging/job_1385542328307_0915/job.split On Tue, Dec 10, 2013 at 9:15 AM, ch huang <[email protected]> wrote: > the strange thing is when i use the following command i find 1 corrupt > block > > # curl -s http://ch11:50070/jmx |grep orrupt > "CorruptBlocks" : 1, > but when i run hdfs fsck / , i get none ,everything seems fine > > # sudo -u hdfs hdfs fsck / > ........ > > ....................................Status: HEALTHY > Total size: 1479728140875 B (Total open files size: 1677721600 B) > Total dirs: 21298 > Total files: 100636 (Files currently being written: 25) > Total blocks (validated): 119788 (avg. block size 12352891 B) (Total > open file blocks (not validated): 37) > Minimally replicated blocks: 119788 (100.0 %) > Over-replicated blocks: 0 (0.0 %) > Under-replicated blocks: 166 (0.13857816 %) > Mis-replicated blocks: 0 (0.0 %) > Default replication factor: 3 > Average block replication: 3.0027633 > Corrupt blocks: 0 > Missing replicas: 831 (0.23049656 %) > Number of data-nodes: 5 > Number of racks: 1 > FSCK ended at Tue Dec 10 09:14:48 CST 2013 in 3276 milliseconds > > The filesystem under path '/' is HEALTHY > > > On Tue, Dec 10, 2013 at 8:32 AM, ch huang <[email protected]> wrote: > >> hi,maillist: >> my nagios alert me that there is a corrupt block in HDFS all >> day,but i do not know how to remove it,and if the HDFS will handle this >> automaticlly? and if remove the corrupt block will cause any data >> lost?thanks >> > >
