wenn-id opened a new pull request, #10467: URL: https://github.com/apache/arrow-rs/pull/10467
Closes #5550. # Rationale for this change Arrow `Int8`, `Int16`, `UInt8`, and `UInt16` values are widened to Parquet `INT32` when written. Checking their original narrow byte representation against the serialized bloom filter therefore produces false negatives. # What changes are included in this PR? - Add an Arrow-aware array kernel that widens narrow integer arrays to their Parquet physical representation before checking `Sbbf`. - Share the narrow-integer coercion helper with `ArrowWriter`, keeping writer and checker semantics aligned. - Preserve Arrow nulls in the returned `BooleanArray` and reject unsupported Arrow types. - Add round-trip tests using bloom filters written by `ArrowWriter`, serialized, reopened, and checked for present, absent, boundary, and null values. # Are there any user-facing changes? Adds `parquet::arrow::bloom_filter::check_bloom_filter` for `Int8`, `Int16`, `UInt8`, and `UInt16` arrays. Other Arrow-to-Parquet coercions remain unsupported and return an error. # Testing - `cargo test -p parquet --lib --features arrow` (1,237 passed) - `cargo test -p parquet --lib bloom_filter --features arrow -- --nocapture` (38 passed) - `cargo clippy -p parquet --lib --all-features -- -D warnings` - `RUSTDOCFLAGS="-D warnings" cargo doc -p parquet --all-features --no-deps` - `cargo +stable fmt --all -- --check` - `git diff --check` # AI usage Hermes Agent was used to help analyze the existing code and draft the implementation and tests. I reviewed the complete diff, added independent negative-membership coverage, initialized the official test-data submodules, ran the full Parquet library test suite and quality gates above, and can maintain and debug the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
