[
https://issues.apache.org/jira/browse/HADOOP-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662409#action_12662409
]
Brian Bockelman commented on HADOOP-4994:
-----------------------------------------
Hey Dhruba,
That is correct (I guess I should mention, as this is a Java project, not a
Unix project, stat is equivalent to File.length...).
This is the use case:
1) Node loses power.
2) On reboot, linux triggers an automatic fsck of hadoop's storage system
3) To clean up some discovered corruption, linux truncates one of Hadoop's
blocks
4) Hadoop starts up - reads in the metadata, and assumes the block is OK.
I would like to alter step (4) to be:
4) Hadoop starts up, reads in metadata
5) Hadoop checks to make sure block length recorded in the metadata file is the
same as the block length recorded by the ext3 filesystem.
My apologies if this is already done and I am just not understanding things
correctly.
> Datanode should verify block sizes vs metadata on startup
> ---------------------------------------------------------
>
> Key: HADOOP-4994
> URL: https://issues.apache.org/jira/browse/HADOOP-4994
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Brian Bockelman
>
> I could have sworn this bug had been reported by someone else already, but I
> can't find it on JIRA after searching.... apologies if this is a duplicate.
> The datanode, upon starting up, should check and make sure that all block
> sizes as reported via `stat` are the same as the block sizes as reported via
> the block's metadata.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
