[jira] [Commented] (HDFS-11943) Warn log frequently print to screen in doEncode function on AbstractNativeRawEncoder class

[liaoyuxiangqin (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HDFS-11943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16047379#comment-16047379
 ] 

liaoyuxiangqin commented on HDFS-11943:
---------------------------------------

Thanks [~Sammi] for review on this. The RawErasureEncoder.encode call relation 
as  follows:

{code:title=RawErasureEncoder.java|borderStyle=solid}
 if (usingDirectBuffer) {
      doEncode(bbeState);
    } else {
      ByteArrayEncodingState baeState = bbeState.convertToByteArrayState();
      doEncode(baeState);  //AbstractNativeRawEncoder.doEncode
    }
{code}

After the test, i find hdfs default use heap buffer, so the usingDirectBuffer 
is false,
and call AbstractNativeRawEncoder.doEncode print log frequent. In addition, 
after i modify default value of buffer type, usingDirectBuffer  change to true 
and the call stack is change too and frequent log disappeared.

So as you guess, the NativeXORRawEncoder doesn't indicate itself support the 
direct buffe.

> Warn log frequently print to screen in doEncode function on  
> AbstractNativeRawEncoder class
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-11943
>                 URL: https://issues.apache.org/jira/browse/HDFS-11943
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: erasure-coding, native
>    Affects Versions: 3.0.0-alpha4
>         Environment: cluster: 3 nodes
> os:(Red Hat 2.6.33.20,  Red Hat 3.10.0-514.6.1.el7.x86_64, 
> Ubuntu4.4.0-31-generic)
> hadoop version: hadoop-3.0.0-alpha4
> erasure coding: XOR-2-1-64k and enabled Intel ISA-L
> hadoop fs -put file /
>            Reporter: liaoyuxiangqin
>            Assignee: liaoyuxiangqin
>            Priority: Minor
>         Attachments: HDFS-11943.002.patch, HDFS-11943.patch
>
>   Original Estimate: 0.05h
>  Remaining Estimate: 0.05h
>
>  when i write file to hdfs on above environment,  the hdfs client  frequently 
> print warn log of use direct ByteBuffer inputs/outputs in doEncode function 
> to screen, detail information as follows:
> 2017-06-07 15:20:42,856 WARN rawcoder.AbstractNativeRawEncoder: 
> convertToByteBufferState is invoked, not efficiently. Please use direct 
> ByteBuffer inputs/outputs



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org