[
https://issues.apache.org/jira/browse/HDFS-11765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001850#comment-16001850
]
Masatake Iwasaki commented on HDFS-11765:
-----------------------------------------
The patch lgtm too. I'm running HDFS tests with the patch applied. +1 if there
is no failures except for known flaky ones.
bq. I find that the write performance decreased by 13%
[~GeLiXin], can you share how you did the comparison? It would be nice if we
can reproduce the issue by TestDFSIO or similar.
BlockReceiver seems to be relevant caller of {{DataChecksum#BlockReceiver}}
with non-direct buffer. Based on the result of Crc32PerformanceTest,
NativeCrc32 is faster than CRC32 if bytes-per-checksum <= 128 on my environment.
> Fix:Performance regression due to incorrect use of DataChecksum
> ---------------------------------------------------------------
>
> Key: HDFS-11765
> URL: https://issues.apache.org/jira/browse/HDFS-11765
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: native, performance
> Affects Versions: 2.8.0, 3.0.0-alpha1
> Reporter: LiXin Ge
> Assignee: LiXin Ge
> Attachments: HDFS-11765.patch
>
>
> Recently I have upgraded my Hadoop version from 2.6 to 3.0, and I find that
> the write performance decreased by 13%. After some days comparative analysis,
> It's seems introduced by HADOOP-10865.
> Since James Thomas have done the work that native checksum can run against
> byte[] arrays instead of just against byte buffers, we may use native method
> preferential because it runs faster than others.
> [~szetszwo] and [~iwasakims] could you take a look at this to see if it make
> bad effect on your benchmark test? [~tlipcon] could you help to see if I have
> make mistakes in this patch?
> thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
