englefly opened a new pull request, #69111:
URL: https://github.com/apache/doris/pull/69111
### What problem does this PR solve?
Problem Summary:
When ConstantPropagation replaces an aggregate output by a constant, it
materializes the constant in a
project above the aggregate, and the project item reused the exprId of the
replaced output:
LogicalProject ( projects=[..., 1 AS `id`#1, ...] )
+-- LogicalAggregate ( groupByExpr=[id#1], outputExpr=[id#1] )
So one exprId had two definitions, the aggregate output slot and the
constant, whose nullable and data
type may differ. Rules which compare exprIds only then take them as the same
column. For RQG case
11000258898 EliminateUnnecessaryProject removed such a project after the
aggregate had been rewritten
into `Limit 1`, the plan above fell back to the nullable output slot of the
set operation, and the last
AdjustNullable pass reported
AdjustNullable convert slot 2004-04-28#55 from not-nullable to nullable
The same project item is created in `replaceNameExpressionConstants` for a
project item which is not a
plain slot reference (or a cast of one).
Fix:
- The constant item gets a fresh exprId and the rule records replaced exprId
-> fresh exprId.
- The references in the plan above are refreshed with ExprIdRewriter, the
same way EliminateGroupByKey
and EliminateGroupByKeyByUniform do it.
- The constant project is only materialized if the aggregate really changes,
so no exprId is registered
for a project which is not built.
- The sink also follows the fresh exprIds, otherwise it would reference a
slot which its child does not
output any more. Only the exprIds are remapped there, the name and data
type declared by the sink are
kept (that is why the rule does not replace the slots of the sink by
literals).
Reproduction: ConstantPropagationFreshExprIdTest runs this rule alone, so
the project above the aggregate
is still visible in the plan, and asserts that it does not reuse the exprId
of the aggregate output and
that the plan above references the fresh exprIds. ConstantPropagationTest
used to pin the reused exprIds
in its constant project expectations; it now asserts fresh exprIds. Both
fail before this change.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- ConstantPropagationFreshExprIdTest (new): fails before with "the
constant project reuses the exprId
of the aggregate output".
- ConstantPropagationTest: expectations updated, fails before with "the
constant project item must not
reuse the exprId of the replaced output slot".
- 31 related FE unit test classes (constant propagation, constant cte,
set operation, column pruning,
fd, uniform, sql plan suite, runtime filter, ...) rerun: 385 tests, 0
failures.
- Behavior changed: Yes. A constant materialized above an aggregate has its
own exprId and the plan above
references it, so one exprId no longer describes two different columns.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
