Ext3h commented on code in PR #48192:
URL: https://github.com/apache/arrow/pull/48192#discussion_r2547005014
##########
cpp/src/arrow/util/compression_zstd.cc:
##########
@@ -207,8 +226,18 @@ class ZSTDCodec : public Codec {
Result<int64_t> Compress(int64_t input_len, const uint8_t* input,
int64_t output_buffer_len, uint8_t* output_buffer)
override {
- size_t ret = ZSTD_compress(output_buffer,
static_cast<size_t>(output_buffer_len),
- input, static_cast<size_t>(input_len),
compression_level_);
+ if (!compression_context_) {
Review Comment:
It appears the issue is common to Brotli and ZSTD - both of them are
requiring more space than can fit on the stack. So those two would benefit of a
thread-local context, as their context is always heap allocated.
The others, LZ4, Snappy, BZIP etc. are all properly optimized in the sense
that they can fit the context entirely onto the stack. For them, the
single-shot interfaces work as expected and carry no (significant) overhead.
Merely `lz4` still has a fast path for context-reuse but it's in the
static-only part of the interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
