mbutrovich opened a new issue, #7709: URL: https://github.com/apache/datafusion-comet/issues/7709
### Is your feature request related to a problem or challenge? `spark-expr` has three copies of the loop that visits the set bits of a 64-bit word with `trailing_zeros` and `bits &= bits - 1`. `set_bits` in `hash_funcs/utils.rs` wraps it in an iterator ([`utils.rs`](https://github.com/apache/datafusion-comet/blob/4918aa46fd136ff854fd128cc91d570ab135e361/native/spark-expr/src/hash_funcs/utils.rs#L24-L34)). `copy_rows` and `fill_rows` in `case_when.rs` write it out as `while` loops ([`copy_rows`](https://github.com/apache/datafusion-comet/blob/d5fe36abc30ba288da2aafb8b6a5f8c476f5cdfd/native/spark-expr/src/conditional_funcs/case_when.rs#L702-L706), [`fill_rows`](https://github.com/apache/datafusion-comet/blob/d5fe36abc30ba288da2aafb8b6a5f8c476f5cdfd/native/spark-expr/src/conditional_funcs/case_when.rs#L722-L725)). ### Describe the solution you'd like Move `set_bits` to one `pub(crate)` place in `spark-expr` and use it in `copy_rows` and `fill_rows`. All three users are in this crate, so it doesn't need to go in `datafusion-comet-common`. Both `case_when.rs` loops run once per sparse 64-row word. Before switching them, compare the generated code of the iterator with the `while` loops on x86-64 and aarch64 under the release profile. If it differs, keep the `while` loops unless a `case_when` benchmark shows no loss. ### Describe alternatives you've considered Leave the three copies. Each is four lines. ### Additional context This came up in review of #7691 (https://github.com/apache/datafusion-comet/pull/7691#discussion_r4645582456). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
