alexeigor opened a new pull request, #10958: URL: https://github.com/apache/arrow-rs/pull/10958
# Which issue does this PR close? - Closes #10955. # Rationale for this change `cast_numeric_arrays` currently sends every numeric pair through `num_cast`, including total conversions such as `Int32 -> Int64`. The safe path rebuilds validity through `unary_opt`; the unsafe path retains a fallible operation and walks valid indices. Both prevent the straight-line widening loop used by `PrimitiveArray::unary`. # What changes are included in this PR? - Identify the 35 numeric pairs for which `num_cast` is total and route them through `unary` with `AsPrimitive::as_`. - Keep every narrowing or otherwise fallible conversion on the existing checked paths. The match is fail-closed, so an omitted pair only misses an optimization. - Add a correctness matrix covering every fast-path pair, edge values, spread samples, floating-point special values, and an `i64 -> i32` negative control. - Add Criterion coverage for safe, `safe: false`, and direct `unary` paths at 0% and 10% null density over 100,000 values. Float16 conversions are intentionally left out of the fast path because they cannot use a primitive `as` cast and need separate correctness reasoning. # Are these changes tested? Yes. Criterion results on Apple arm64 with rustc 1.98.0: | nulls | path | stock | patched | Criterion change | |---:|---|---:|---:|---:| | 0% | safe | 10.91 us | 9.13 us | -18.0% to -22.5% | | 0% | `safe: false` | 12.00 us | 9.12 us | -18.6% to -23.0% | | 10% | safe | 55.92 us | 9.14 us | -83.4% to -83.7% | | 10% | `safe: false` | 56.20 us | 9.14 us | -83.8% to -84.0% | | 10% | direct `unary` | 9.36 us | 9.14 us | reference | A DataFusion 55 reproduction over 9,994,240 rows improves `SUM(Int32)` from 7.40 ms to 3.08 ms (2.40x), while the `SUM(Int64)` control stays at 2.14 ms. Validation against current `main` using its pinned Rust toolchain: - `cargo test -p arrow-cast --lib` (379 passed) - `cargo fmt --all -- --check` - `cargo clippy -p arrow-cast --lib --tests -- -D warnings` - `cargo clippy -p arrow --bench cast_kernels --features test_utils -- -D warnings` - `cargo bench -p arrow --bench cast_kernels --features test_utils --no-run` AI assistance disclosure: Codex assisted with the reproduction scaffolding, test matrix, benchmark, and initial patch. The resulting code was validated with the checks above. # Are there any user-facing changes? No API or behavior changes are intended. Infallible numeric casts become faster and retain the input null buffer instead of rebuilding it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
