crm26 opened a new pull request, #21363: URL: https://github.com/apache/datafusion/pull/21363
## Summary `DecorrelatePredicateSubquery` only handled `InSubquery` and `Exists` in `Filter` nodes. When they appeared in Projection expressions (CASE, COALESCE, bare boolean SELECT), the physical planner errored: ``` This feature is not implemented: Physical plan does not support logical expression InSubquery(...) ``` This adds a `Projection` match arm following the same pattern as `ScalarSubqueryToJoin`'s Projection handler. ## What's fixed All of these now work: ```sql -- CASE with IN subquery SELECT CASE WHEN id IN (SELECT id FROM t2) THEN 'yes' ELSE 'no' END FROM t1 -- EXISTS in CASE SELECT CASE WHEN EXISTS (SELECT 1 FROM t2 WHERE t2.id = t1.id) THEN 'found' ELSE 'missing' END FROM t1 -- NOT IN / NOT EXISTS SELECT CASE WHEN id NOT IN (SELECT id FROM t2) THEN 'keep' ELSE 'drop' END FROM t1 -- Bare boolean SELECT id IN (SELECT id FROM t2) AS is_match FROM t1 -- Multiple subqueries in one expression SELECT CASE WHEN id IN (SELECT id FROM t2) AND EXISTS (SELECT 1 FROM t3 WHERE t3.id = t1.id) THEN 'both' ELSE 'neither' END FROM t1 -- COALESCE with IN subquery SELECT COALESCE(CASE WHEN id IN (SELECT id FROM t2) THEN 'found' END, 'missing') FROM t1 ``` ## Design - Follows `ScalarSubqueryToJoin`'s established Projection handler pattern - Reuses existing `rewrite_inner_subqueries` + `mark_join` — no new decorrelation logic - Bails out when decorrelation fails (e.g., correlated subquery with LIMIT) — returns original plan unchanged - LeftMark joins already handled by all downstream optimizer rules (verified) ## Tests - 8 sqllogictest cases (`in_subquery_projection.slt`) - Plan-shape unit test for the bail-out path (undecorrelatable subquery in Projection) - 58 optimizer unit tests passing, clippy clean -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
