eldenmoon opened a new pull request, #70365:
URL: https://github.com/apache/doris/pull/70365
### What problem does this PR solve?
A null test over a Variant cast silently drops rows whose Variant value is
not convertible to the
target type. For a `VARIANT NOT NULL` column `v` holding `1`, `{"a":1}`,
`"x"`, `[1,2]`:
```sql
SELECT id, CAST(v AS INT) IS NULL FROM t ORDER BY id; -- 0,1,1,1
SELECT id FROM t WHERE CAST(v AS INT) IS NULL ORDER BY id; -- empty,
expected 2,3,4
```
`_normalize_is_null_predicate` rewrites the null test on the cast into a
`NullPredicate` on the
scan slot (`v IS NULL`), but `CAST(v AS INT)` returns NULL for the non-NULL
object/string/array
values, so the pushed predicate removes rows that the expression would keep.
`IS NOT NULL` is
unaffected (`v IS NOT NULL` is a superset and the expression is still
evaluated), and
`<=> NULL` is wrong in the same way as `IS NULL` because it is rewritten to
the same null test.
`is_valid_push_down_cast` only admits casts over Variant slots, so the
rewrite is reachable
exactly for Variant casts, where `CAST(v AS T) IS NULL` is not equivalent to
`v IS NULL`.
### Fix
When the predicate's argument is a cast whose source is a Variant slot, keep
the null test in the
expression tree instead of pushing a `NullPredicate` down. The predicate
then filters with the
cast's own NULL semantics. No other pushdown is affected: the guard only
triggers for the
Variant-cast shape that `is_valid_push_down_cast` already admits.
### Tests
New regression `variant_p0/test_variant_cast_isnull_pushdown` (golden
generated by the
regression runner): INT cast values plus `IS NULL`, `IS NOT NULL` and `<=>
NULL` filters over a
`VARIANT NOT NULL` column.
Red/green was verified locally with a 4.2 build of the same code path:
- before the fix: `WHERE CAST(v AS INT) IS NULL` returns no rows, while the
projection returns
`0,1,1,1`;
- after the fix: the filter returns `2,3,4`, `<=> NULL` returns `2,3,4`, `IS
NOT NULL` returns `1`.
### Release note
Fix `CAST(variant_col AS T) IS NULL` / `<=> NULL` returning wrong (empty)
results.
### Check List (For Author)
- Test: Regression test (`test_variant_cast_isnull_pushdown`) + manual
end-to-end red/green
- Behavior changed: Yes (the filters now return the rows whose cast result
is NULL)
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
