acbokade opened a new issue, #26284:
URL: https://github.com/apache/datafusion/issues/26284
### Describe the bug
Aggregating over a projected `CASE` expression whose `WHEN` compares two
columns fails with an internal error:
```
Internal error: Physical input schema should be the same as the one
converted from logical input schema. Differences:
- field nullability at index 0 [x]: (physical) true vs (logical) false.
This issue was likely caused by a bug in DataFusion's code. ...
```
The logical schema marks `CASE WHEN a < b THEN a ELSE 1 END` as non-nullable
(`THEN a` is only reached when `a < b` is true, so `a` isn't NULL there), but
the physical `CaseExpr` reports it as nullable. The physical planner compares
the two schemas for an aggregate's input, so queries that compute the
expression below an aggregate fail, for example `DISTINCT`, `UNION`, or an
aggregate over a subquery.
### To Reproduce
```sql
CREATE TABLE n(a INT, b INT) AS VALUES (NULL, 1), (0, 5), (7, 2);
SELECT DISTINCT CASE WHEN a < b THEN a ELSE 1 END AS x FROM n;
```
These fail the same way:
```sql
SELECT sum(x) FROM (SELECT CASE WHEN a < b THEN a ELSE 1 END AS x FROM n);
SELECT CASE WHEN a < b THEN a ELSE 1 END AS x FROM n UNION SELECT 2;
```
With `WHEN a > 0` or `WHEN a IS NOT NULL` in place of `a < b`, the
`DISTINCT` query works.
Tested on `main` at 0566b55a4.
### Expected behavior
`0` and `1` for the `DISTINCT` query, `2` for the `sum`, and `0`, `1`, `2`
for the `UNION`, as in PostgreSQL.
### Additional context
- The logical side
([`expr_schema.rs`](https://github.com/apache/datafusion/blob/0566b55a4766e280b36c73d11b0bffe9aa5a7fcb/datafusion/expr/src/expr_schema.rs#L353-L376))
uses `predicate_bounds::evaluate_bounds`, which works out that `NULL < b`
can't be true. The physical side
([`CaseExpr::nullable`](https://github.com/apache/datafusion/blob/0566b55a4766e280b36c73d11b0bffe9aa5a7fcb/datafusion/physical-expr/src/expressions/case.rs#L1286-L1307))
replaces `a` with NULL and tries to constant-fold `NULL < b`. That still
references `b`, so the result is inconclusive and the branch is treated as
nullable.
- #25369 reports the same mismatch reached through common subexpression
elimination. Its open fix, #25684, adds AND/OR/NOT handling to the physical
side; the three queries above still fail with #25684 applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
