englefly opened a new pull request, #69784:
URL: https://github.com/apache/doris/pull/69784
### What problem does this PR solve?
Issue Number: None
Related PR: #69111
Problem Summary:
Constant propagation materializes a constant above the aggregate or the
project which defines a
column. The materialized item is a literal, which is always not-nullable,
while the column it
replaces may be nullable. The item used to keep the exprId of the column in
every case, so the
rewritten plan could contain two definitions of the same exprId with
different nullability: the
not-nullable literal above, and the nullable column of the subtree below.
The plan stays valid while a node still separates the two definitions, but a
later rule may remove
that node. In the reported queries `EliminateGroupByKeyByUniform` replaces
the uniform DISTINCT
aggregate by `limit + project`, and the not-nullable literal is then
attached directly above the
nullable set operation. `AdjustNullable` has to widen the column on the way
down, and with
`fe_debug` enabled the check aborts the query with
`AdjustNullable convert slot ... from not-nullable to nullable`.
Reproduction: with #69111 reverted, both reported queries trip the check
again; the reduced form of
the second one is
```sql
select * from ((
select 'vZRgkfajsD', 'cKToNmsxok'
except (select col_char_25__undef_signed_not_null,
col_char_40__undef_signed from by54)
union distinct
select col_varchar_25__undef_signed, col_char_25__undef_signed from by53
where col_tinyint_undef_signed_not_null = 4.4929476592789594
intersect distinct
select col_varchar_100__undef_signed, col_char_40__undef_signed from by53
where col_bigint_undef_signed_not_null < 54)
except (select col_char_25__undef_signed,
col_varchar_100__undef_signed_not_null from by5)
) as t where exists (select 1);
```
The fix keeps the exprId only when
* the materialized constant does not change the nullability of the column, or
* the child does not produce the column any more, then the project item is
the only definition of the exprId.
When the child still produces the column and the constant has another
nullability, the constant gets
a fresh exprId and the references above are refreshed, so one exprId keeps
one definition. Not
renaming the columns whose nullability does not change keeps the identity of
the columns the
materialized view matching relies on, which #69111 had to trade for a
refresh of the whole plan
above.
### Release note
None
### Check List (For Author)
- Test: Unit Test / Regression test
- Unit: `ConstantPropagationTest`, `ConstantPropagationFreshExprIdTest`
(the reported queries and
the reduced form are rewritten with `fe_debug` enabled, and the
rewritten plan is checked to
have one nullability per exprId)
- Regression: the mv suites under `nereids_rules_p0/mv`
(`agg_optimize_when_uniform`,
`aggregate_with_roll_up`, `aggregate_without_roll_up`, `availability`,
`mv_ssb_test`,
`pre_write_strategy`)
- Behavior changed: No
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
