[
https://issues.apache.org/jira/browse/HADOOP-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296444#comment-14296444
]
Vinayakumar B commented on HADOOP-11514:
----------------------------------------
Thanks [~drankye] for the clarification.
Me and [~drankye] had offline discussion about this a bit.
I am fine with keeping overloaded API with ECChunk.
bq. how to converting ECChunk to bytes or ByteBuffer is better to be located in
the central place as I put in the base.
{code}+ if (inputs[0].getBuffer().hasArray()) {
+ byte[][] inputBytesArr = ECChunk.toArray(inputs);
+ byte[][] outputBytesArr = ECChunk.toArray(outputs);
+ doDecode(inputBytesArr, erasedIndexes, outputBytesArr);
+ } else {
+ ByteBuffer[] inputBuffers = ECChunk.toBuffers(inputs);
+ ByteBuffer[] outputBuffers = ECChunk.toBuffers(outputs);
+ doDecode(inputBuffers, erasedIndexes, outputBuffers);
+ }{code}
In the above conversion, calling encode/decode API with byte array, or
ByteBuffer differs based on how ECChunk was created.
As kai said, Some implementations of coders use byte array ( ex: pure java),
some uses native ByteBuffer ( ex: ISA-L) for the better performance.
So, now since the ECChunk is created in upstream, it should be created based on
RawCoder used to get the best performance.
BTW, [~drankye] please rename the patch starting with HADOOP-11514
> Raw Erasure Coder API for concrete encoding and decoding
> --------------------------------------------------------
>
> Key: HADOOP-11514
> URL: https://issues.apache.org/jira/browse/HADOOP-11514
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: HDFS-EC
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HDFS-7353-v1.patch, HDFS-7353-v2.patch,
> HDFS-7353-v3.patch, HDFS-7353-v4.patch, HDFS-7353-v5.patch,
> HDFS-7353-v6.patch, HDFS-7353-v7.patch
>
>
> This is to abstract and define raw erasure coder API across different codes
> algorithms like RS, XOR and etc. Such API can be implemented by utilizing
> various library support, such as Intel ISA library and Jerasure library.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
