[
https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417261#comment-15417261
]
Kai Zheng commented on HDFS-8668:
---------------------------------
Thanks for adding some tests.
Oops, found some issues. Please double check. Thanks!
{code}
- sum.calculateChunkedSums(buffer.array(), 0, len, checksumBuf, 0);
+ if (buffer.isDirect()) {
+ ByteBuffer directCheckSumBuf =
+ BUFFER_POOL.getBuffer(true, checksumBuf.length);
+ sum.calculateChunkedSums(buffer, directCheckSumBuf);
+ directCheckSumBuf.get(checksumBuf);
+ } else {
+ sum.calculateChunkedSums(buffer.array(), 0, len, checksumBuf, 0);
+ }
+
{code}
{{directCheckSumBuf}} should be put back afterwards. Similarly for the
following:
{code}
- stripedWriter.getChecksum().calculateChunkedSums(
- targetBuffer.array(), 0, targetBuffer.remaining(),
- stripedWriter.getChecksumBuf(), 0);
+ if(targetBuffer.isDirect()) {
+ ByteBuffer directCheckSumBuf =
+ ByteBuffer.allocateDirect(stripedWriter.getChecksumBuf().length);
+ stripedWriter.getChecksum().calculateChunkedSums(
+ targetBuffer, directCheckSumBuf);
+ directCheckSumBuf.get(stripedWriter.getChecksumBuf());
+ }else{
+ stripedWriter.getChecksum().calculateChunkedSums(
+ targetBuffer.array(), 0, targetBuffer.remaining(),
+ stripedWriter.getChecksumBuf(), 0);
+ }
{code}
Please also fix the code formats around keywords {{if}}, {{else}}, needing a
space.
Unnecessary blank line in the following block:
{code}
void close() {
+
{code}
> Erasure Coding: revisit buffer used for encoding and decoding.
> --------------------------------------------------------------
>
> Key: HDFS-8668
> URL: https://issues.apache.org/jira/browse/HDFS-8668
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Yi Liu
> Assignee: SammiChen
> Attachments: HDFS-8668-v1.patch, HDFS-8668-v2.patch,
> HDFS-8668-v3.patch, HDFS-8668-v4.patch, HDFS-8668-v5.patch,
> HDFS-8668-v6.patch, HDFS-8668-v7.patch, HDFS-8668-v8.patch
>
>
> For encoding and decoding buffers, currently some places use java heap
> ByteBuffer, some use direct byteBUffer, and some use java byte array. If
> the coder implementation is native, we should use direct ByteBuffer. This
> jira is to revisit all encoding/decoding buffers and improve them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
