haohuaijin opened a new pull request, #10593: URL: https://github.com/apache/arrow-rs/pull/10593
# Which issue does this PR close? Closes #10592. # Rationale for this change `and_then_masks` runs once per predicate per row group during filter pushdown when selections are mask-backed. The current per-set-bit implementation is slow for large or dense selections. # What changes are included in this PR? - Rewrite `and_then_masks` to process the mask one 64-bit word at a time, depositing the next `k` bits of `other` onto each word's `k` set positions, with word-wise output instead of per-bit appends. - Add `mask_and_then` cases to the `row_selector` benchmark (this path previously had no coverage). # Benchmark results `row_selector::mask_and_then`, 3M rows (aarch64 macOS): | outer mask | main | this PR | change | |---|---|---|---| | random 1% selected | 349 µs | 244 µs | −30% | | random 33% selected | 4.47 ms | 0.98 ms | −78% | | random 90% selected | 9.16 ms | 1.98 ms | −78% | ClickBench (`async_object_store`, default policy): 6 of 22 queries improve up to −2.5%, rest within noise, no regressions. # Are these changes tested? Covered by existing `selection` tests (63 pass), including randomized equivalence tests. Bit-for-bit equivalent, performance only. # Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
