[
https://issues.apache.org/jira/browse/HDFS-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194872#comment-13194872
]
Daryn Sharp commented on HDFS-2848:
-----------------------------------
As best I can tell based on a cursory read of the code, append will add to the
block but not immediately update the file size until the block is committed --
block fills or the stream is closed. Client readers will only get the
committed block size and data, which means the spurious bytes are "harmless" to
a client. I think an append will seek to the end of the committed data, and
then overwrite the spurious bytes.
I'm not a DN expert, but detecting the incorrectly sized blocks is probably
something best left to fsck and/or the block scanner. It also might be
possible to have the NN issue a truncate in response to a block report that
doesn't match the NN's view of the world. Maybe hadoop already does something
like this. A DN expert should weigh in.
> hdfs corruption appended to blocks is not detected by fs commands or fsck
> -------------------------------------------------------------------------
>
> Key: HDFS-2848
> URL: https://issues.apache.org/jira/browse/HDFS-2848
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
>
> Courtesy Pat White
> {quote}
> Appears that there is a regression in corrupt block detection by both fsck
> and fs cmds like 'cat'. Testcases for
> pre-block and block-overwrite corruption of all replicas is correctly
> reporting errors however post-block corruption is
> not, fsck on the filesystem reports it's Healthy and 'cat' returns without
> error. Looking at the DN blocks themselves,
> they clearly contain the injected corruption pattern.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
