alamb opened a new issue, #6486: URL: https://github.com/apache/arrow-datafusion/issues/6486
### Is your feature request related to a problem or challenge? RepartitionExec , when handling multiple input partitions, creates N channels for each input partition, where N is the output partition count. This results in a total of input_partition * output_partition channels. During processing, the channels are pulled for each output partition, depending on the processing time, which disrupts the order of records. This is particularly problematic when the input partition count is greater than 1, as it leads to an unpredictable order of records within the output partitions. ### Describe the solution you'd like To address this issue, a more sophisticated algorithm is needed, one that can combine the existing hash partitioner and round-robin partitioner functionalities while preserving the original order of records within partitions, even when the input partition count is greater than 1. ### Describe alternatives you've considered _No response_ ### Additional context I used the description from https://github.com/apache/arrow-datafusion/pull/6346 @baharberna above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
