[
https://issues.apache.org/jira/browse/HADOOP-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672476#action_12672476
]
hairong edited comment on HADOOP-5133 at 2/10/09 5:18 PM:
----------------------------------------------------------------
The logic in summary 3 should only apply to the last block of a file under
construction. A complete picture should be:
{noformat}
if the reported block is the last block of an under-construction file {
do summary 3
} else {
if the reported block is not the last block && its NN recorded length is
not equal to the preferred size
set the block's NN recorded length to be the preferred block size;
if the reported length is not equal to the NN recorded length
mark the reported block as corrupt;
}
{noformat}
was (Author: hairong):
The logic in summary 3 should only apply to the last block of a file under
construction. A complete picture should be:
if the reported block is the last block of an under-construction file {
do summary 3
} else {
if the reported block is not the last block of a file and NN recorded
length is not equal to the preferred block length, set the block's NN recorded
length to be the preferred block size;
if the reported length is not equal to the NN recorded length, mark the
reported block as corrupt;
}
> FSNameSystem#addStoredBlock does not handle inconsistent block length
> correctly
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5133
> URL: https://issues.apache.org/jira/browse/HADOOP-5133
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.2
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.18.4
>
> Attachments: inconsistentLen.patch
>
>
> Currently NameNode treats either the new replica or existing replicas as
> corrupt if the new replica's length is inconsistent with NN recorded block
> length. The correct behavior should be
> 1. For a block that is not under construction, the new replica should be
> marked as corrupt if its length is inconsistent (no matter shorter or longer)
> with the NN recorded block length;
> 2. For an under construction block, if the new replica's length is shorter
> than the NN recorded block length, the new replica could be marked as
> corrupt; if the new replica's length is longer, NN should update its recorded
> block length. But it should not mark existing replicas as corrupt. This is
> because NN recorded length for an under construction block does not
> accurately match the block length on datanode disk. NN should not judge an
> under construction replica to be corrupt by looking at the inaccurate
> information: its recorded block length.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
