adriangb commented on code in PR #25338:
URL: https://github.com/apache/datafusion/pull/25338#discussion_r4020665150
##########
datafusion/optimizer/src/decorrelate_predicate_subquery.rs:
##########
@@ -576,7 +653,10 @@ fn build_join(
new_plan.display_indent()
);
- return Ok(Some(new_plan));
+ return Ok(Some(BuiltJoin {
+ plan: new_plan,
+ mark_is_three_valued_exact: mark_filter_is_hashable_only,
Review Comment:
Good catch, confirmed. With `id INT NOT NULL`, `NULLIF(id, 1) IN (SELECT
...)` returned `false` and `NULLIF(id, 1) NOT IN (SELECT ...)` returned `true`
where SQL gives `NULL`. DuckDB and PostgreSQL both give `NULL`.
Fixed in b44d847eca. `join_keys_may_be_null` now takes the equijoin key
expressions from `split_eq_and_noneq_join_predicate` and asks
`ExprSchemable::nullable` for each key against the schema of its own side. That
covers `NULLIF`, `TRY_CAST`, a `CASE` with no `ELSE`, and a key that a cast
wraps. The residual filter keeps the older column-based check, so no case is
less conservative than before. For these keys the projection path stays one
null-aware mark join, so the plan shape and the timings do not change.
The same helper feeds the `LeftAnti` path. On `main`, `SELECT id FROM nn
WHERE NULLIF(id, 1) NOT IN (SELECT id FROM r3)` keeps the `id = 1` row that it
must drop. The same commit fixes that too.
Tests: two unit tests with a nullable key expression over a non-nullable
column, one for `IN` in a projection and one for `NOT IN` in a filter, both
plan `null_aware`. A new section in `subquery_projection.slt` runs the `NULLIF`
left key, the `TRY_CAST` left key, a `NULLIF` right key, `NOT IN`, and the
`WHERE` form, and pins the single-join `EXPLAIN`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
