[
https://issues.apache.org/jira/browse/HBASE-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phabricator updated HBASE-5387:
-------------------------------
Attachment: D1719.3.patch
mbautin updated the revision "[jira] [HBASE-5387] Reuse compression streams in
HFileBlock.Writer".
Reviewers: tedyu, Liyin, dhruba, JIRA
Breaking compression stream creation methods into multiple parts. Avoiding
using "finish-on-flush" stream in a couple of places, and deprecating the
currently used method that is always called with downstream buffer size of 0,
at least within HBase code. The method is still there for backwards
compatibility. Also, getting rid of hard-coded buffer size by delegating the
native compressor case to superclass in ReusableStreamGzipCodec.
REVISION DETAIL
https://reviews.facebook.net/D1719
AFFECTED FILES
src/main/java/org/apache/hadoop/hbase/io/hfile/Compression.java
src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java
src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java
src/main/java/org/apache/hadoop/hbase/io/hfile/ReusableStreamGzipCodec.java
src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java
> Reuse compression streams in HFileBlock.Writer
> ----------------------------------------------
>
> Key: HBASE-5387
> URL: https://issues.apache.org/jira/browse/HBASE-5387
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.94.0
> Reporter: Mikhail Bautin
> Assignee: Mikhail Bautin
> Priority: Critical
> Fix For: 0.94.0
>
> Attachments: D1719.1.patch, D1719.2.patch, D1719.3.patch,
> Fix-deflater-leak-2012-02-10_18_48_45.patch,
> Fix-deflater-leak-2012-02-11_17_13_10.patch
>
>
> We need to to reuse compression streams in HFileBlock.Writer instead of
> allocating them every time. The motivation is that when using Java's built-in
> implementation of Gzip, we allocate a new GZIPOutputStream object and an
> associated native data structure every time we create a compression stream.
> The native data structure is only deallocated in the finalizer. This is one
> suspected cause of recent TestHFileBlock failures on Hadoop QA:
> https://builds.apache.org/job/HBase-TRUNK/2658/testReport/org.apache.hadoop.hbase.io.hfile/TestHFileBlock/testPreviousOffset_1_/.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
