jaylmiller opened a new pull request, #5292:
URL: https://github.com/apache/arrow-datafusion/pull/5292
# Which issue does this PR close?
Closes #5230.
# Rationale for this change
Using arrow row format for SortExec should improve sorting speed.
Additionally preserving that row encoding to be used
in SortPreservingMergeExec should improve perf by minimizing the amount of
encoding needed to be done.
# What changes are included in this PR?
- Changes are all made within the sorts module
(`datafusion/core/src/physical_plan/sorts`).
- Had to change several files across the module to allow for streaming row
encoding data between SortExec and SortPreservingMergeExec
# Are these changes tested?
- Existing unit tests for SortExec/SortPreservingMergeExec cover this for
the most part.
- For the new structs defined for handling selections of rows have new test.
# Are there any user-facing changes?
Had to add a few new public exports since SortKeyCursor needs to be public
(since it is used in an integration test).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
