Asthenia0412 opened a new pull request, #1376: URL: https://github.com/apache/arrow-go/pull/1376
## Background `ipc.WithZstd()` turns on ZSTD compression for IPC writers but gives callers no way to choose the level. `getCompressor` always builds the encoder with `zstd.NewWriter(nil)`, defaulting to `SpeedDefault` (≈ zstd level 3). For producers streaming large IPC batches over the same network as the consumer, compression is a noticeable share of producer CPU. Since decompression speed barely depends on level, level 1 (`SpeedFastest`) cuts producer CPU with almost no reader impact — today the only way to get there is to patch arrow-go. ## What changed - Added `WithZstdLevel(level int) Option` — callers can set zstd compression level (1–22). - Extended `config`, `Writer`, `FileWriter`, and `recordEncoder` with a new `zstdLevel` field. - Updated `getCompressor(codec, zstdLevel)` to use `zstd.WithEncoderLevel(zstd.EncoderLevelFromZstd(level))` when `zstdLevel > 0`, else falls back to the previous default. ## Backwards compatibility Purely additive. Callers using `WithZstd()` without `WithZstdLevel` get the exact same default behaviour as before. ## Testing - `go build ./arrow/ipc/...` ✅ - `go test -c ./arrow/ipc/...` ✅ - `go test ./arrow/ipc/... -count=1` ✅ (all existing tests pass) ## Alignment with C++ The C++ IPC implementation already lets callers set the ZSTD compression level via `IpcWriteOptions::codec` built from `util::Codec::Create(Compression::ZSTD, level)`. This change brings Go in line with C++. Resolves #1368 Signed-off-by: yancy <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
