[
https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527883#comment-14527883
]
Zhe Zhang commented on HDFS-7678:
---------------------------------
Thanks Jing for the helpful review. I tested 007 patch without HDFS-8281 and it
passes the tests. There must be some hidden conflicts I missed in the rebase.
{code}
int idx = (int) (((blkStartOffset - lb.getStartOffset()) / cellSize)
- % dataBlkNum);
+ % (dataBlkNum + parityBlkNum));
{code}
Good catch! The root of this bug is that
{{StripedBlockUtil#constructInternalBlock}} assigns the starting offset of a
parity block as {{bg.getStartOffset() + idxInBlockGroup * cellSize}}. The
challenge is that {{DFSInputStream#actualGetFromOneDataNode}} keeps refreshing
block location at each retry loop. I'll make this change.
> Erasure coding: DFSInputStream with decode functionality
> --------------------------------------------------------
>
> Key: HDFS-7678
> URL: https://issues.apache.org/jira/browse/HDFS-7678
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Li Bo
> Assignee: Zhe Zhang
> Attachments: BlockGroupReader.patch, HDFS-7678-HDFS-7285.002.patch,
> HDFS-7678-HDFS-7285.003.patch, HDFS-7678-HDFS-7285.004.patch,
> HDFS-7678-HDFS-7285.005.patch, HDFS-7678-HDFS-7285.006.patch,
> HDFS-7678-HDFS-7285.007.patch, HDFS-7678-HDFS-7285.008.patch,
> HDFS-7678.000.patch, HDFS-7678.001.patch
>
>
> A block group reader will read data from BlockGroup no matter in striping
> layout or contiguous layout. The corrupt blocks can be known before
> reading(told by namenode), or just be found during reading. The block group
> reader needs to do decoding work when some blocks are found corrupt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
