[
https://issues.apache.org/jira/browse/HADOOP-12060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633509#comment-14633509
]
Kai Zheng commented on HADOOP-12060:
------------------------------------
Hi [~jingzhao],
Thanks for the more review.
bq. ...In this way we can avoid scanning the inputs/outputs multiple times...
Good suggestion! I will get it done separately as follow-on issue.
I updated the patch again to address the missing changes in erasure coder
layer. The extra change is as follows.
Would you review one more time? Thanks.
{noformat}
---
a/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestErasureCoderBase.java
+++
b/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/coder/TestErasureCoderBase.java
@@ -65,14 +65,14 @@ protected void testCoding(boolean usingDirectBuffer) {
* The following runs will use 3 different chunkSize for inputs and
outputs,
* to verify the same encoder/decoder can process variable width of data.
*/
- performTestCoding(baseChunkSize);
- performTestCoding(baseChunkSize - 17);
- performTestCoding(baseChunkSize + 16);
+ performTestCoding(baseChunkSize, true);
+ performTestCoding(baseChunkSize - 17, false);
+ performTestCoding(baseChunkSize + 16, true);
}
- private void performTestCoding(int chunkSize) {
+ private void performTestCoding(int chunkSize, boolean usingSlicedBuffer) {
setChunkSize(chunkSize);
-
+ prepareBufferAllocator(usingSlicedBuffer);
// Generate data and encode
ECBlockGroup blockGroup = prepareBlockGroupForEncoding();
{noformat}
> Address some issues related to ByteBuffer type input/output buffers for raw
> erasure coders
> ------------------------------------------------------------------------------------------
>
> Key: HADOOP-12060
> URL: https://issues.apache.org/jira/browse/HADOOP-12060
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-12060-HDFS-7285-v1.patch,
> HADOOP-12060-HDFS-7285-v2.patch, HADOOP-12060-HDFS-7285-v3.patch
>
>
> In HDFS-8319 [~jingzhao] raised some issues about ByteBuffer type
> input/output buffers for raw erasure coders:
> * Should support ByteBuffers originated from {{ByteBuffer#slice}} calls;
> * Should clearly spec in Javadoc that no mixing of on-heap buffers and direct
> buffers are allowed, and with necessary checking codes ensuring the same type
> of buffers are used.
> In HDFS-8319 patch by [~jingzhao] there are some good refactoring codes that
> would be incorporated here.
> As discussed open this to address the issues separately.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
