[
https://issues.apache.org/jira/browse/HDFS-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117575#comment-13117575
]
Brian Bockelman commented on HDFS-2251:
---------------------------------------
Hi Uma,
This is actually a super old ticket (Hadoop 0.18). I have no clue if it is
still applicable to the existing code base (it might have gotten added in the
last 3 years). However, it's a really easy way to detect a common corruption
case - the advice still applies.
Won't be offended if you just wanted to close it out until I see the issue
again.
Brian
> Namenode does not recognize incorrectly sized blocks
> ----------------------------------------------------
>
> Key: HDFS-2251
> URL: https://issues.apache.org/jira/browse/HDFS-2251
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Brian Bockelman
>
> We had a lot of file system corruption resulting in incorrectly sized blocks
> (on disk, they're truncated to 192KB when they should be 64MB).
> However, I cannot make Hadoop realize that these blocks are incorrectly
> sized. When I try to drain off the node, I get the following messages:
> 2008-10-29 18:46:51,293 WARN org.apache.hadoop.fs.FSNamesystem: Inconsistent
> size for block blk_-4403534125663454855_9937 reported from 172.16.1.150:50010
> current size is 67108864 reported size is 196608
> Here 172.16.1.150 is not the node which has the problematic block, but the
> destination of the file transfer. I propose that Hadoop should either:
> a) Upon startup, make sure that all blocks are properly sized (pro: rather
> cheap check; con: doesn't catch any truncations which happen while on disk)
> b) Upon detecting the incorrectly sized copy, Hadoop should ask the source of
> the block to perform a block verification.
> Thanks,
> Brian
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
