apurtell commented on a change in pull request #3748:
URL: https://github.com/apache/hbase/pull/3748#discussion_r728433970
##########
File path:
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
##########
@@ -123,4 +137,42 @@ static int getBufferSize(Configuration conf) {
return size > 0 ? size : 256 * 1024; // Don't change this default
}
+ static LoadingCache<Configuration,byte[]> CACHE = CacheBuilder.newBuilder()
Review comment:
This is definitely a concern. I the latest version of the patch I
override hashCode in CompoundConfiguration so we are doing something better
than object identity when caching the dictionaries for the store writer case.
It is kind of expensive to compute the hashCode given how CompoundConfiguration
works but at least we do not do it that often, and not in performance critical
code. Once a compressor or decompressor is created it is reused for the
lifetime of the reader or writer.
Otherwise we are using object identity. That is not the worst thing, at
least. The cache is capped at 100 and will also expire entries if they are not
used for one hour. (And those parameters can be adjusted to your taste.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
