alamb commented on issue #10316: URL: https://github.com/apache/datafusion/issues/10316#issuecomment-2113301756
Update here: 1. @NGA-TRAN has a PR open to port this operator into DataFusion: https://github.com/apache/datafusion/pull/10490 2. We don't want to merge it into DataFusion until at least one other user could get value from it. So in my mind, what is required to move on with this ticket is: 1. Update one of the existing optimizer passes to detect when the sort keys of a `SortPreservingMerge` don't overlap in value and replace with `ProgressiveEval` 2. Tests showing it working For the "do key ranges overlap" detection code I think we can use what @suremarc added in https://github.com/apache/datafusion/pull/9593 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
