emecii opened a new pull request, #11051: URL: https://github.com/apache/arrow-rs/pull/11051
# Which issue does this PR close? - Closes #2282. # Rationale for this change The DELTA_BINARY_PACKED encoder currently fixes its layout at 128 values per block and four miniblocks. Workloads with uniformly distributed deltas can benefit from larger blocks or miniblocks, and the Parquet format records this layout in each encoded stream. # What changes are included in this PR? This adds validated DeltaBinaryPackedEncoderOptions and global/per-column WriterProperties setters. When DELTA_BINARY_PACKED is selected, the column writer passes the resolved layout to the encoder. Existing callers keep the current 128/4 defaults. Validation follows the Parquet format requirements: block sizes are positive multiples of 128, miniblock counts divide the block, and miniblock sizes are multiples of 32. Tests cover validation, property inheritance and per-column overrides, the serialized layout header, and encode/decode round trips. # Are these changes tested? Yes. - cargo test --locked -p parquet --all-features --lib --tests passed with the Arrow and Parquet test datasets configured. - cargo clippy --locked -p parquet --all-targets --all-features -- -D warnings passed. - cargo fmt --all -- --check passed. - git diff --check passed. # Are there any user-facing changes? Yes. This adds an additive public configuration API for the DELTA_BINARY_PACKED encoder. Existing defaults and behavior are unchanged unless users opt in. # AI assistance This PR was prepared with OpenAI Codex assistance for investigation, implementation, and test execution. The changes and public API are scoped to the issue, and the tests exercise both property resolution and the resulting encoded stream. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
