[
https://issues.apache.org/jira/browse/HADOOP-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160213#comment-14160213
]
Hudson commented on HADOOP-10681:
---------------------------------
FAILURE: Integrated in Hadoop-Yarn-trunk #703 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/703/])
HADOOP-10681. Remove unnecessary synchronization from Snappy & Zlib codecs.
Contributed by Gopal Vijayaraghavan. (acmurthy: rev
8f9ab998e273259c1e7a3ed53ba37d767e02b6bb)
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/ZlibCompressor.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/snappy/SnappyCompressor.java
*
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/compress/TestCodecPool.java
* hadoop-common-project/hadoop-common/CHANGES.txt
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/snappy/SnappyDecompressor.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/ZlibDecompressor.java
> Remove synchronized blocks from SnappyCodec and ZlibCodec buffering inner loop
> ------------------------------------------------------------------------------
>
> Key: HADOOP-10681
> URL: https://issues.apache.org/jira/browse/HADOOP-10681
> Project: Hadoop Common
> Issue Type: Bug
> Components: performance
> Affects Versions: 2.2.0, 2.4.0, 2.5.0
> Reporter: Gopal V
> Assignee: Gopal V
> Labels: perfomance
> Fix For: 2.6.0
>
> Attachments: HADOOP-10681.1.patch, HADOOP-10681.2.patch,
> HADOOP-10681.3.patch, HADOOP-10681.4.patch, compress-cmpxchg-small.png,
> perf-top-spill-merge.png, snappy-perf-unsync.png
>
>
> The current implementation of SnappyCompressor spends more time within the
> java loop of copying from the user buffer into the direct buffer allocated to
> the compressor impl, than the time it takes to compress the buffers.
> !perf-top-spill-merge.png!
> The bottleneck was found to be java monitor code inside SnappyCompressor.
> The methods are neatly inlined by the JIT into the parent caller
> (BlockCompressorStream::write), which unfortunately does not flatten out the
> synchronized blocks.
> !compress-cmpxchg-small.png!
> The loop does a write of small byte[] buffers (each IFile key+value).
> I counted approximately 6 monitor enter/exit blocks per k-v pair written.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
