namanjain24-sudo opened a new pull request, #25641:
URL: https://github.com/apache/datafusion/pull/25641
## Which issue does this PR close?
- Closes #25473.
- Closes #25474.
## Rationale for this change
`NOT IN (subquery)` decides whether it needs null-aware join semantics by
checking if any *column* in the join filter is nullable. That misses a NULL
constant (`NULL NOT IN (...)`) or an expression that is nullable on its own
(`CASE` without `ELSE`, `NULLIF`) even when every column it reads is `NOT
NULL`. In both cases a row that should be `UNKNOWN` (and dropped from the
result) is silently kept.
## What changes are included in this PR?
`join_keys_may_be_null` in `decorrelate_predicate_subquery.rs` now checks the
nullability of each equality operand as an expression (via `ExprSchemable`),
not just the columns it references, falling back to "may be null" when that
can't be resolved to one side.
## What is the testing strategy for this PR?
- Two optimizer unit tests: a NULL constant against a `NOT NULL` subquery
column, and a `CASE` expression over `NOT NULL` columns, both asserting the
resulting join is null-aware.
- New cases in `null_aware_anti_join.slt` covering both issues' tables,
including the `OR` mark-join form and `EXPLAIN` assertions. Confirmed each
new query fails on `main` and passes with this change.
- `cargo test -p datafusion-optimizer` and the full default `sqllogictest`
suite (521 files) pass. Checked the TPC-H Q16 plan (the only TPC-H query
using this path) is unchanged.
## Are there any user-facing changes?
`NOT IN` subqueries now return correct (fewer) rows for these two cases.
No public API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
