pitrou commented on code in PR #15194:
URL: https://github.com/apache/arrow/pull/15194#discussion_r1072013727
##########
cpp/src/arrow/ipc/options.h:
##########
@@ -67,6 +67,15 @@ struct ARROW_EXPORT IpcWriteOptions {
/// May only be UNCOMPRESSED, LZ4_FRAME and ZSTD.
std::shared_ptr<util::Codec> codec;
+ /// @brief Always use compression, regardless of the resulting output size
+ ///
+ /// If false, compression will be skipped for body buffers where the
expected compressed
+ /// size matches/exceeds the uncompressed size. Otherwise, compression will
be applied
+ /// indiscriminately. This option is ignored if no codec was supplied.
+ ///
+ /// This is true by default due to backwards-compatibility concerns.
+ bool compress_always = true;
Review Comment:
We could perhaps get smart and make this an optional floating-point
threshold. For example only compress if it would save at least 10% of the
buffer size.
On the flip side, compression is typically much slower than decompression,
and compression is attempted anyway, so the end-to-end cost might not vary that
much. @lidavidm @westonpace Thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
