neilconway opened a new pull request, #11242: URL: https://github.com/apache/arrow-rs/pull/11242
# Which issue does this PR close? - N/A # Rationale for this change Casting an array of numeric to boolean iterated over each array element, checking for validity and then appending the new value and validity bit. Using `BooleanArray::from_unary` is much faster, for two reasons: 1. It reuses the input's validity array instead of reconstructing it bit-by-bit 2. Avoiding per-element null checks makes it easier for the compiler to vectorize the loop. Casting an array of boolean to numeric also used a similar value-by-value loop. As before, we can do better by reusing the input's validity bitmap. We also play a further trick: we chunk the input bits into 64-bit words and process one chunk at a time; the fixed-size loop helps LLVM vectorize more effectively. Benchmarks: (M4 Max) - Int32 → Boolean, no nulls: 18.999 → 0.909 µs — 20.90× faster - Int32 → Boolean, ~10% nulls: 16.990 → 0.914 µs — 18.59× faster - Int32 → Boolean, all nulls: 18.379 → 0.919 µs — 20.00× faster - Boolean → Int32, no nulls: 14.333 → 0.587 µs — 24.40× faster - Boolean → Int32, ~10% nulls: 12.717 → 0.589 µs — 21.59× faster - Boolean → Int32, all nulls: 4.407 → 0.591 µs — 7.46× faster # What changes are included in this PR? * Optimize `cast_numeric_to_bool` and `cast_bool_to_numeric` as described above * Add unit tests * Add benchmarks # Are these changes tested? Yes; new tests added. # Are there any user-facing changes? No. # AI usage Developed with Codex (Astra 6), reviewed and revised with Claude Code (Opus 5.5). I reviewed and understand the resulting code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
