[
https://issues.apache.org/jira/browse/CURATOR-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714909#comment-16714909
]
ASF GitHub Bot commented on CURATOR-487:
----------------------------------------
Github user leventov commented on the issue:
https://github.com/apache/curator/pull/282
There is no peer evidence here, because we are on the optimization
forefront. See
https://github.com/apache/incubator-druid/pull/6677#discussion_r237182258 and
https://lists.apache.org/thread.html/1aff123193cec5c385821b2d745a4e846a8a5786146c047acbdf8ea3@%3Cdev.druid.apache.org%3E.
I've seen a Druid heap with more than 10k finalizable Deflater objects,
about 8k of which were already dead, awaiting in the finalization queue. They
come from `GzipCompressionProvider`.
Historically Druid uses Zookeeper somewhat wrong (not for what Zookeeper
was designed): it announces data segment placement using Zookeeper, that leads
to creation of a lot of new nodes in Zookeeper every second. It means that by
accident, Druid is a good stress test for Zookeeper (and consequently for
Curator), and we run probably the largest Druid cluster.
> GzipCompressionProvider produces a lot of finalizable objects
> -------------------------------------------------------------
>
> Key: CURATOR-487
> URL: https://issues.apache.org/jira/browse/CURATOR-487
> Project: Apache Curator
> Issue Type: Improvement
> Reporter: Roman Leventov
> Assignee: Cameron McKenzie
> Priority: Major
> Fix For: 4.1.0
>
>
> GzipCompressionProvider.compress() and decompress() methods are called a lot
> inside and outside of the framework and each such call produces a
> java.util.zip.Deflater or Inflater object that are finalizable, that is bad
> for GC.
>
> Compressing or decompressing a finite byte[] object (in contrast to
> compressing/decompressing an InputStream or an OutputStream of unknown
> length) is actually a happy case because even PhantomReference object could
> be avoided, the native resources could be created and freed in a single
> try-finally block.
>
> Curator must avoid that. [https://github.com/ymnk/jzlib] is a potential
> replacement, for example.
>
> A similar issue could be also fixed in Apache Commons Compress:
> https://issues.apache.org/jira/browse/COMPRESS-473 and Curator is made to
> depend on Commons Compress.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
