[
https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14534624#comment-14534624
]
Kai Zheng commented on HDFS-7678:
---------------------------------
Hi Zhe,
Thanks for your hard work here!
bq. I just spent several good hours to find out the above test failure is a
codec level issue
I'm sorry for this. Did you mean it's caused by something bad existing in raw
erasure coder? Please note current raw coder assumes all the input/output
buffers are very alike. For example, if you use ByteBuffer version
encode/decode API, the byteBuffers for inputs and outputs shall all start with
0 (position), the readable/writable content/space take the whole buffer
capacity. I know the assumption is bad and not documented clearly, I will get
this fixed and refined in HADOOP-11938, changing the tests to simulate all
kinds of buffers to ensure its fix. Hope this helps.
> Erasure coding: DFSInputStream with decode functionality (pread)
> ----------------------------------------------------------------
>
> Key: HDFS-7678
> URL: https://issues.apache.org/jira/browse/HDFS-7678
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Li Bo
> Assignee: Zhe Zhang
> Attachments: BlockGroupReader.patch, HDFS-7678-HDFS-7285.002.patch,
> HDFS-7678-HDFS-7285.003.patch, HDFS-7678-HDFS-7285.004.patch,
> HDFS-7678-HDFS-7285.005.patch, HDFS-7678-HDFS-7285.006.patch,
> HDFS-7678-HDFS-7285.007.patch, HDFS-7678-HDFS-7285.008.patch,
> HDFS-7678-HDFS-7285.009.patch, HDFS-7678-HDFS-7285.010.patch,
> HDFS-7678-HDFS-7285.012.patch, HDFS-7678.000.patch, HDFS-7678.001.patch
>
>
> A block group reader will read data from BlockGroup no matter in striping
> layout or contiguous layout. The corrupt blocks can be known before
> reading(told by namenode), or just be found during reading. The block group
> reader needs to do decoding work when some blocks are found corrupt.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
