SubhamSinghal opened a new pull request, #21621: URL: https://github.com/apache/datafusion/pull/21621
## Which issue does this PR close? https://github.com/apache/datafusion/issues/11900 ## Rationale for this change When a query has `ORDER BY <cols> LIMIT N` on top of an outer join and all sort columns come from the preserved side, DataFusion currently runs the full join first, then sorts and limits. We can push a copy of the `Sort(fetch=N)` to the preserved input, reducing the number of rows entering the join. **Before:** ``` Sort: t1.b ASC, fetch=3 Left Join: t1.a = t2.a Scan: t1 ← scans ALL rows Scan: t2 ``` **After:** ``` Sort: t1.b ASC, fetch=3 Left Join: t1.a = t2.a Sort: t1.b ASC, fetch=3 ← pushed down Scan: t1 ← only top-3 rows enter join Scan: t2 ``` ## What changes are included in this PR? A new logical optimizer rule `PushDownTopKThroughJoin` that: 1. Matches `Sort` with `fetch = Some(N)` (TopK) 2. Looks through an optional `Projection` to find a `Join` 3. Checks join type is LEFT or RIGHT with no non-equijoin filter 4. Verifies all sort expression columns come from the preserved side 5. Inserts a copy of the `Sort(fetch=N)` on the preserved child 6. Keeps the top-level sort for correctness ## Are these changes tested? Yes through UT ## Are there any user-facing changes? No API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
