tustvold opened a new issue, #5882: URL: https://github.com/apache/arrow-datafusion/issues/5882
### Is your feature request related to a problem or challenge? SortPreservingMerge currently always uses the arrow row format, this provides compelling benefits when sorting by a tuple of multiple columns. However, it is often the case that a sort is being performed on a single column. `lexsort_to_indices` which is used by `SortExec` detects this case and calls through to `sort_to_indices` which has specialised implementations for each column type. This significantly outperforms converting to the row format as it allows for fixed width comparisons, which are significantly faster ### Describe the solution you'd like I would like to be able to use a specialized, fixed width sort cursor within SortPreservingMerge ### Describe alternatives you've considered _No response_ ### Additional context https://github.com/apache/arrow-datafusion/pull/5854 contained a POC implementation of this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
