mattfaltyn opened a new pull request, #3123: URL: https://github.com/apache/iceberg-rust/pull/3123
## Which issue does this PR close? - Closes #3122. ## What changes are included in this PR? After numeric schema evolution, Arrow row filters compare an old physical column with a literal bound to the new table schema. Casting the literal down can turn out-of-range integers into nulls or round floating-point values, silently changing which rows match. This change promotes `Int32` columns to `Int64`, `Float32` to `Float64`, and `Decimal128` to a wider precision at the same scale before comparison. The six binary comparisons and `IN`/`NOT IN` share this path; each membership predicate promotes the column once per batch. Existing scalar casts between equivalent Arrow string/binary representations remain unchanged. No public API, dependency, metadata format, or persisted-data changes. Scope: this fixes Arrow row filters, including the default reader configuration. Optional page-index pruning has a separate pre-existing numeric-promotion defect (described in #3122) and is not changed here. ## Are these changes tested? - Added an end-to-end Parquet scan regression for all supported numeric widening families, out-of-range and in-range literals, all six comparisons, `IN`/`NOT IN`, nulls, and a double literal that rounds to a stored float if narrowed. - The matrix covers projected and filter-only columns with row-group pruning enabled and disabled. - The regression fails on the base revision (`x < 2147483648` returns no rows) and passes with the fix. - `cargo test -p iceberg --lib arrow::reader --locked`: 103 passed, including the existing Arrow representation-cast tests. - Formatting, TOML formatting, and dependency-use checks passed. - Full local workspace validation and upstream CI are in progress; this PR remains draft until local validation and final review are complete. ## AI Disclosure Codex assisted with investigation, implementation, regression tests, and this description. The regression was executed before and after the fix, and the changed comparison paths were reviewed. The page-index limitation above was identified during expanded validation and is explicitly outside this PR's scope. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
