[
https://issues.apache.org/jira/browse/HDFS-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493543#comment-14493543
]
Jing Zhao commented on HDFS-8120:
---------------------------------
Thanks for updating the patch, Zhe. And thanks for the review, Walter. The 002
patch looks good to me. The only comment is about the changes in
{{BlockManager#checkReplicaCorrupt}}.
{code}
+ assert reported.getBlockId() - storedBlock.getBlockId() <
+ HdfsConstants.MAX_BLOCKS_IN_GROUP;
+ byte reportedBlkIdx = (byte)
+ (reported.getBlockId() - storedBlock.getBlockId());
if (storedBlock.getGenerationStamp() != reported.getGenerationStamp())
{
final long reportedGS = reported.getGenerationStamp();
return new BlockToMarkCorrupt(new Block(reported), storedBlock,
reportedGS,
"block is " + ucState + " and reported genstamp " + reportedGS
+ " does not match genstamp in block map "
+ storedBlock.getGenerationStamp(), Reason.GENSTAMP_MISMATCH);
- } else if (storedBlock.getNumBytes() != reported.getNumBytes()) {
+ } else if (StripedBlockUtil.getInternalBlockLength(
+ storedBlock.getNumBytes(), HdfsConstants.BLOCK_STRIPED_CELL_SIZE,
+ HdfsConstants.NUM_DATA_BLOCKS, reportedBlkIdx)
+ != reported.getNumBytes()) {
{code}
More specifically,
# The assert should be only for striped blocks. Also {{MAX_BLOCKS_IN_GROUP}}
can be replaced by {{storedBlock.getTotalBlockNum()}}
# Let's use {{BlockIdManager.getBlockIndex}} to get internal block index
# The "else if" part should cover both striped and contiguous blocks
# Instead of using {{HdfsConstants.NUM_DATA_BLOCKS}}, we should use
{{storedBlock.getDataBlockNum}}
# We need to have some unit tests for this block report fix
> Erasure coding: created util class to analyze striped block groups
> ------------------------------------------------------------------
>
> Key: HDFS-8120
> URL: https://issues.apache.org/jira/browse/HDFS-8120
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-8120.000.patch, HDFS-8120.001.patch,
> HDFS-8120.002.patch
>
>
> The patch adds logic of calculating size of individual blocks in a striped
> block group.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
