[
https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14535803#comment-14535803
]
Zhe Zhang commented on HDFS-7678:
---------------------------------
{code
if (chunk.state == StripingChunk.REQUESTED && i <= dataBlkNum) {
{code}
Sorry forgot to update this in the patch. The {{i <= dataBlkNum}} logic was an
attempt to bypass the coded bug and should have been removed from the patch.
[~drankye] No worries. I think it's good that we discover the risk of solely
relying on end-to-end tests, since the striping and codec logics are fairly
complex. As discussed above I'm thinking about creating a dummy codec
algorithm, which is based on the current {{SimulatedDataset}}. Basically, given
input buffers 0, 1, 2, 4, 5, 6 (0, 1, 2, 4, 5 are data blocks, 6 is parity
block, 3 is data block and is missing), once we verify (from the content) that
the buffers map to matching ranges in blocks 0, 1, 2, 4, 5, 6, the simulated
codec algorithm will just generate the matching range of block 3.
> Erasure coding: DFSInputStream with decode functionality (pread)
> ----------------------------------------------------------------
>
> Key: HDFS-7678
> URL: https://issues.apache.org/jira/browse/HDFS-7678
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Li Bo
> Assignee: Zhe Zhang
> Attachments: BlockGroupReader.patch, HDFS-7678-HDFS-7285.002.patch,
> HDFS-7678-HDFS-7285.003.patch, HDFS-7678-HDFS-7285.004.patch,
> HDFS-7678-HDFS-7285.005.patch, HDFS-7678-HDFS-7285.006.patch,
> HDFS-7678-HDFS-7285.007.patch, HDFS-7678-HDFS-7285.008.patch,
> HDFS-7678-HDFS-7285.009.patch, HDFS-7678-HDFS-7285.010.patch,
> HDFS-7678-HDFS-7285.012.patch, HDFS-7678-HDFS-7285.013.patch,
> HDFS-7678.000.patch, HDFS-7678.001.patch
>
>
> A block group reader will read data from BlockGroup no matter in striping
> layout or contiguous layout. The corrupt blocks can be known before
> reading(told by namenode), or just be found during reading. The block group
> reader needs to do decoding work when some blocks are found corrupt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
