[
https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated HDFS-7678:
----------------------------
Attachment: HDFS-7678-HDFS-7285.009.patch
Rebased again to fix the test error in {{TestDFSStripedInputStream}}.
{code}
int idx = (int) (((blkStartOffset - lb.getStartOffset()) / cellSize)
- % dataBlkNum);
+ % (dataBlkNum + parityBlkNum));
{code}
[~jingzhao] I just realized this is not a bug. For contiguous blocks, the
{{LocatedBlock#offset}} field represents the starting offset of a block in a
file. For a striped block group, this field represents the same meaning and is
used to retrieve the group from NN. However, for an internal block, this field
is only used to identify its index in the group. So as long as we have a
consistent mapping it should work fine. We can even set {{startOffset}} as
{{bg.getStartOffset() + idxInBlockGroup}} in {{constructInternalBlock}}, and in
{{getBlockAt}} as long as we apply the reverse mapping, it should work. We can
think about how to assign the values to improve readability (maybe in
HDFS-8320).
> Erasure coding: DFSInputStream with decode functionality (pread)
> ----------------------------------------------------------------
>
> Key: HDFS-7678
> URL: https://issues.apache.org/jira/browse/HDFS-7678
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Li Bo
> Assignee: Zhe Zhang
> Attachments: BlockGroupReader.patch, HDFS-7678-HDFS-7285.002.patch,
> HDFS-7678-HDFS-7285.003.patch, HDFS-7678-HDFS-7285.004.patch,
> HDFS-7678-HDFS-7285.005.patch, HDFS-7678-HDFS-7285.006.patch,
> HDFS-7678-HDFS-7285.007.patch, HDFS-7678-HDFS-7285.008.patch,
> HDFS-7678-HDFS-7285.009.patch, HDFS-7678.000.patch, HDFS-7678.001.patch
>
>
> A block group reader will read data from BlockGroup no matter in striping
> layout or contiguous layout. The corrupt blocks can be known before
> reading(told by namenode), or just be found during reading. The block group
> reader needs to do decoding work when some blocks are found corrupt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
