[
https://issues.apache.org/jira/browse/HDFS-11943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054033#comment-16054033
]
Kai Zheng commented on HDFS-11943:
----------------------------------
Thanks [~liaoyuxiangqin] & [~Sammi] for the discussions!
bq. Do you have any idea that NativeXORRawEncoder should support direct Buffer
or not? I think the answer is yes. Correct me if it's not the case.
This is correct. Should all native coders indicate they prefer direct
bytebuffer for better performance.
bq. so i think the condition not neccessary to add
This is correct. Agree.
bq. So as you guess, the NativeXORRawEncoder doesn't indicate itself support
the direct buffer.
A good reasoning! Actually you got it, that's the cause. Ensuring to have the
following block in the base native class
{{AbstractNativeRawEncoder/AbstractNativeRawDecoder}} should assure all native
coders will behave as the thought.
{code}
@Override
public boolean preferDirectBuffer() {
return true;
}
{code}
[~liaoyuxiangqin] would you update your patch to fix this? Thanks!
> Warn log frequently print to screen in doEncode function on
> AbstractNativeRawEncoder class
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-11943
> URL: https://issues.apache.org/jira/browse/HDFS-11943
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding, native
> Affects Versions: 3.0.0-alpha4
> Environment: cluster: 3 nodes
> os:(Red Hat 2.6.33.20, Red Hat 3.10.0-514.6.1.el7.x86_64,
> Ubuntu4.4.0-31-generic)
> hadoop version: hadoop-3.0.0-alpha4
> erasure coding: XOR-2-1-64k and enabled Intel ISA-L
> hadoop fs -put file /
> Reporter: liaoyuxiangqin
> Assignee: liaoyuxiangqin
> Priority: Minor
> Attachments: HDFS-11943.002.patch, HDFS-11943.patch
>
> Original Estimate: 0.05h
> Remaining Estimate: 0.05h
>
> when i write file to hdfs on above environment, the hdfs client frequently
> print warn log of use direct ByteBuffer inputs/outputs in doEncode function
> to screen, detail information as follows:
> 2017-06-07 15:20:42,856 WARN rawcoder.AbstractNativeRawEncoder:
> convertToByteBufferState is invoked, not efficiently. Please use direct
> ByteBuffer inputs/outputs
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
