bananaaggle edited a comment on issue #11816: URL: https://github.com/apache/druid/issues/11816#issuecomment-1000195848
Hi! I know what happened! I used to test ZSTD on Kafka. It was caused by commit [10123](https://github.com/apache/kafka/pull/10123/files). You can see ZstdFactory.java, there is `return new BufferedOutputStream(new ZstdOutputStreamNoFinalizer(buffer, RecyclingBufferPool.INSTANCE), 16 * 1024);` and `return new BufferedInputStream(new ZstdInputStreamNoFinalizer(new ByteBufferInputStream(buffer), RecyclingBufferPool.INSTANCE), 16 * 1024);` It used a reused buffer to reduce GC times. This buffer is a new feature in zstd 1.4.5-8. This commit is released in Kafka 2.8.0. If Druid uses any Kafka clients after 2.8.0, it should update zstd to 1.4.5-8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
