ryux1 opened a new pull request, #25002: URL: https://github.com/apache/datafusion/pull/25002
## Which issue does this PR close? - Closes #24960. ## Rationale for this change A groupless aggregate always emits one row unless an operator such as `HAVING` or `LIMIT 0` removes it. Correlated `EXISTS` and `IN` predicates currently decorrelate that aggregate into semi/anti joins, which loses the empty-input aggregate row. As a result, `EXISTS` drops outer rows without an inner match, `IN` misses comparisons against values such as `count(*) = 0`, and `NOT IN` can fail planning with a two-column null-aware anti join. ## What changes are included in this PR? - Recognize correlated, guaranteed-single-row subqueries through projection and alias wrappers. - Fold `EXISTS` and `NOT EXISTS` according to that cardinality. - Rewrite `IN` and `NOT IN` to scalar-subquery comparisons so the existing scalar decorrelator preserves empty-input aggregate values. - Keep `HAVING`, `LIMIT`, grouped aggregates, and uncorrelated subqueries on their existing paths. - Update the affected logical-plan snapshot. ## What is the testing strategy for this PR? A new `issue_24960.slt` regression file covers top-level `EXISTS`, `NOT EXISTS`, `IN`, and `NOT IN`, plus embedded `EXISTS`. It also verifies that `HAVING` and `LIMIT 0` are not treated as guaranteed-one-row cases. Locally passing: - `cargo test --test sqllogictests -- issue_24960.slt subquery.slt` - `cargo test -p datafusion-optimizer` (776 unit tests, 26 integration tests, doc tests) - `PATH=/home/ryu/.cargo/bin:$PATH ./dev/rust_lint.sh` ## Are there any user-facing changes? Yes. Correlated `EXISTS` and `IN` predicates over groupless aggregates now preserve SQL aggregate cardinality and return the correct rows. There are no public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
