albertlockett opened a new pull request, #8405: URL: https://github.com/apache/arrow-rs/pull/8405
# Which issue does this PR close? - Closes https://github.com/apache/arrow-rs/issues/8386 # Rationale for this change Reusing the zstd context between subsequent calls to compress_zstd in the Arrow IPC writer for performance improvement. Benchmark results: ``` arrow_ipc_stream_writer/StreamWriter/write_10/zstd time: [4.0972 ms 4.1038 ms 4.1110 ms] change: [-53.848% -53.586% -53.335%] (p = 0.00 < 0.05) Performance has improved. ``` # What changes are included in this PR? Adds a `CompressionContext` struct, which when the zstd feature is enabled contains a zstd::bulk::Compressor object. This context object is owned by the ipc `StreamWriter`/`FileWriter` objects and is passed by mutable reference through the `IpcDataGenerator` to the `CompressionCodec` where it is used when compressing the ipc bytes. # Are these changes tested? Yes the existing unit tests cover the changed code paths # Are there any user-facing changes? No, there are no code changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
