[
https://issues.apache.org/jira/browse/HADOOP-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196461#comment-15196461
]
Kai Zheng commented on HADOOP-11540:
------------------------------------
Hi [~cmccabe],
Thanks for being so quick for the review and the helpful comments!
* For doEncodeByConvertingToDirectBuffers, your point makes sense as we don't
have all the codes be presented here so far. I thought I should have placed the
function into AbstractNativeRawEncoder, because only native coders will need do
that. We'll have another native coder for the XOR codec, and it will extend
AbstractNativeRawEncoder and need the function too. So how about moving
doEncodeByConvertingToDirectBuffers to AbstractNativeRawEncoder, similarly for
doDecodeByConvertingToDirectBuffers?
* For initOutputs and resetBuffer, good catch! About this I initially thought
as you suggested, instead of having initOutputs, just letting concrete coders
to override resetBuffer, which would be most flexible. Then I realized for Java
coders, a default behavior can be provided and used; for native coders, we can
avoid having it because at the beginning of the encode() call the native coder
can {{memset}} the output buffers directly. If instead the native coder has to
provide resetBuffer, then a JNI function has to be added, which will be called
some times to reset output buffers. Considering the overhead in both
implementation and extra JNI calls, I used the initOutputs() approach.
As I'm not sure about how would you like this clarifying, please kindly let me
know if this works for you or not. Thanks.
> Raw Reed-Solomon coder using Intel ISA-L library
> ------------------------------------------------
>
> Key: HADOOP-11540
> URL: https://issues.apache.org/jira/browse/HADOOP-11540
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: HDFS-7285
> Reporter: Zhe Zhang
> Assignee: Kai Zheng
> Attachments: HADOOP-11540-initial.patch, HADOOP-11540-v1.patch,
> HADOOP-11540-v2.patch, HADOOP-11540-v4.patch, HADOOP-11540-v5.patch,
> HADOOP-11540-with-11996-codes.patch, Native Erasure Coder Performance - Intel
> ISAL-v1.pdf
>
>
> This is to provide RS codec implementation using Intel ISA-L library for
> encoding and decoding.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
