yihua commented on code in PR #18818:
URL: https://github.com/apache/hudi/pull/18818#discussion_r3338708743
##########
hudi-common/src/main/java/org/apache/hudi/common/util/collection/BitCaskDiskMap.java:
##########
@@ -408,29 +409,47 @@ private static class CompressionHandler implements
Serializable {
private final ByteArrayOutputStream compressBaos;
private final ByteArrayOutputStream decompressBaos;
private final byte[] decompressIntermediateBuffer;
+ // Each CompressionHandler is held in a ThreadLocal,
+ // so a single Deflater/Inflater pair per worker thread is sufficient and
+ // avoids per-call construction.
+ private transient Deflater deflater;
+ private transient Inflater inflater;
CompressionHandler() {
compressBaos = new
ByteArrayOutputStream(DISK_COMPRESSION_INITIAL_BUFFER_SIZE);
decompressBaos = new
ByteArrayOutputStream(DISK_COMPRESSION_INITIAL_BUFFER_SIZE);
decompressIntermediateBuffer = new
byte[DECOMPRESS_INTERMEDIATE_BUFFER_SIZE];
}
+ private Deflater getDeflater() {
Review Comment:
@vamsikarnika Could you add the microbenchmark code as part of this PR as
well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
