[
https://issues.apache.org/jira/browse/HADOOP-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294915#comment-14294915
]
Vinayakumar B commented on HADOOP-11514:
----------------------------------------
bq. About why we need the 3rd version encode()/decode(), it is because in above
layer in ErasureCoder, ECChunks are extracted from blocks and then they're
passed down here for the encoding/decoding. How to get bytes or ByteBuffer from
ECChunk, it depends and therefore better have the logic centrally here.
Generally, in pure Java implementation, bytes are allocated in heap and used;
in ISA-L, better to obtain ByteBuffer from off-heap for performance
consideration.
I think, we need not include ECChunk in RawEncoder/RawDecoder. As 'Raw' refers
to 'byte' or 'ByteBuffer' let these RawEncoder and RawDecoder only handle pure
'Raw' form. Not the Wrapper over these. In fact whole ECChunk class itself is
unnecessary in Common, as this is specific to 'striped' design of the erasure
encoding, but current Jira targets concrete APIs only. ECChunk can be converted
'ByteBuffer' before passing to encode()/decode() of RawEncoder/RawDecoder in
higher levels.
Even if {{encode(ECChunk[] inputs, ECChunk[] outputs);}} is present in API, do
you think, any other implementation is possible other than calling
{{encode(ByteBuffer[] inputs, ByteBuffer[] outputs);}} after converting to
ECChunk to ByteBuffer ?
> Raw Erasure Coder API for concrete encoding and decoding
> --------------------------------------------------------
>
> Key: HADOOP-11514
> URL: https://issues.apache.org/jira/browse/HADOOP-11514
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Fix For: HDFS-EC
>
> Attachments: HDFS-7353-v1.patch, HDFS-7353-v2.patch,
> HDFS-7353-v3.patch, HDFS-7353-v4.patch, HDFS-7353-v5.patch,
> HDFS-7353-v6.patch, HDFS-7353-v7.patch
>
>
> This is to abstract and define raw erasure coder API across different codes
> algorithms like RS, XOR and etc. Such API can be implemented by utilizing
> various library support, such as Intel ISA library and Jerasure library.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
