DeviousCardi opened a new pull request, #11092: URL: https://github.com/apache/arrow-rs/pull/11092
## Summary Fixes #11073. Writing a Decimal-logical-type column backed by an Arrow `BinaryArray`/`ByteArray` through `ArrowColumnWriter` produced wrong min/max column statistics, because the statistics computation used plain unsigned lexicographic `Ord` comparison on the raw two's-complement-encoded bytes. The direct `parquet::file::writer` column-writer path (and the `FixedSizeBinaryArray` path) already handled this correctly via a signed, decimal-aware comparator. ## Changes - `parquet/src/column/writer/mod.rs`: widened `compare_greater_byte_array_decimals` to `pub(crate)`; added a shared `is_decimal_descr()` helper (used by both this file's `compare_greater` and the Arrow writer) to avoid duplicating the decimal-type-detection check. - `parquet/src/arrow/arrow_writer/byte_array.rs`: `ByteArrayEncoder` now detects decimal columns at construction and uses the signed comparator for both the running min/max computation and the cross-batch min/max merge. - `truncate_statistics`/`can_truncate_value` (`column/writer/mod.rs`): `BYTE_ARRAY` decimal columns are now excluded from statistics truncation, matching the existing guard already in place for `FIXED_LEN_BYTE_ARRAY` decimals — truncating a decimal's two's-complement bytes with unsigned `increment()` could otherwise reintroduce a corrupted bound for long decimals. - Two regression tests added, based on the issue's repro and a truncation-specific case. ## Test plan - [x] `cargo test -p parquet --lib arrow_writer` — 166 passed - [x] `cargo test -p parquet --lib column::writer` — 112 passed - [x] `cargo clippy -p parquet --lib --all-targets` — clean - [x] `cargo build -p parquet` — clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
