sunchao opened a new issue, #11379: URL: https://github.com/apache/arrow-rs/issues/11379
### Problem description `arrow_select::interleave::interleave` has specialized paths for primitive, byte, view, dictionary and nested arrays, while Boolean arrays use the `MutableArrayData` fallback. For scattered row selections this repeats descriptor/bookkeeping work around individual bitmap bits. Comet PR https://github.com/apache/datafusion-comet/pull/6430 has a top-level Boolean specialization using `BooleanBuffer::collect_bool` for values and validity. The reviewer requested moving the optimization upstream so nested Booleans also benefit through recursive interleave, and Comet can remove its copy after upgrading Arrow. ### Proposed improvement Add a Boolean specialization in arrow-select. Preserve sliced offsets, nulls, row ordering and existing bounds/error behavior. Compare scattered selections with contiguous/clustered runs before choosing dispatch, since the current fallback efficiently copies bitmap ranges. Include non-byte-aligned slices, nullable/all-null arrays, short and empty selections, and nested-array coverage. The Comet discussion includes Arrow 59.3.0 kernel and end-to-end shuffle measurements from both the author and reviewer. Those results motivate the change but are not a benchmark of current arrow-rs main. I checked the current main dispatch while filing this issue and Boolean still selects the fallback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
