Dandandan opened a new issue, #6822: URL: https://github.com/apache/arrow-datafusion/issues/6822
### Is your feature request related to a problem or challenge? Also see request in arrow https://github.com/apache/arrow-rs/issues/4476 In DataFusion, a common operation is to repartition a RecordBatch by hashing one or more columns and dividing them into partition record batches using the "formula" hash % num_partitions. The current approach is to create the indices that match and use them to take the individual arrays (see BatchPartitioner in datafusion). This is relatively expensive however, as we visit the arrays num_partitions times in different places, leading to cache inefficient. ### Describe the solution you'd like Faster partitioning implementation ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
