geoffreyclaude opened a new pull request, #26118: URL: https://github.com/apache/datafusion/pull/26118
## Which issue does this PR close? Related to #23276. This fixes unsafe pushdown through the existing custom-plan fallback. ## Rationale for this change Sort pushdown through a custom execution plan can remove a necessary sort when the operator changes column positions or values. For example, an operator projecting `[b, a]` is sorted underneath on `[a, b]`, then a merge requiring `[b, a]` fails `SanityCheckPlan`. Preserving row order does not establish a mapping between output and input columns. ## What changes are included in this PR? Rebuild the custom operator with candidate child orderings and check that its output properties satisfy the requested ordering before accepting pushdown. Invalid candidate expressions leave the outer sort in place. Transparent and renamed projections retain the optimization. ## What is the testing strategy for this PR? A public API regression in `custom_sort_pushdown.rs` executes six cases across two partitions: identity, renaming, reordering, generated values with the same schema, reversed values, and incompatible expression types. It checks sort placement and exact output order, with `SanityCheckPlan` enabled. The unchanged baseline fails the reordered-column case. ## Are there any user-facing changes? Queries sorting custom-operator output retain the necessary sort when the requested ordering cannot be proved. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
