princejain756 commented on issue #26710: URL: https://github.com/apache/datafusion/issues/26710#issuecomment-6395247132
take Reproduced on main: `ScalarSubquery(<subquery>)` survives because `PullUpCorrelatedExpr` stops at the `Sort` (and at `Limit` for non-EXISTS), so `build_join` gets `can_pull_up = false` and the subquery reaches the physical planner. Going to implement the two-shape rewrite in `ScalarSubqueryToJoin` per the sketch: - `Limit 1` over `Sort` over a correlated filter, where the filter is equalities plus a single range predicate on the sort key in the direction the sort picks, becomes an `AsOfJoin` (`MATCH_CONDITION outer.v >= inner.key` for `DESC`, mirrored for `ASC`). Non-correlated predicates stay on the inner input. - Equalities only becomes a LEFT JOIN over `row_number() OVER (PARTITION BY eq keys ORDER BY sort keys) = 1`. - Anything that does not match either shape keeps today's behavior, so plan shapes elsewhere are untouched. Starting with the ASOF shape, which covers most of the dbolympics queries. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
