github-actions[bot] commented on code in PR #67940:
URL: https://github.com/apache/doris/pull/67940#discussion_r4036475716
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/UnCorrelatedApplyAggregateFilter.java:
##########
@@ -97,28 +548,1427 @@ public List<Rule> buildRules() {
return apply;
}
- // pull up correlated filter into apply node
- List<NamedExpression> newAggOutput = new
ArrayList<>(agg.getOutputExpressions());
- List<Expression> newGroupby =
- Utils.getUnCorrelatedExprs(correlatedPredicate,
apply.getCorrelationSlot());
- newGroupby.addAll(agg.getGroupByExpressions());
+ CorrelatedAggregatePredicates predicates =
+ CorrelatedAggregatePredicates.of(apply, correlatedPredicate,
+ aggregation.filtersAboveTheAggregation());
+ if (needCorrelatedAggregationOnOuter(apply, aggregation,
correlatedPredicate, predicates)) {
+ Plan aggregatedOuter = pullUpCorrelatedPredicateByAggregatingOuter(
+ apply, aggregation, unCorrelatedPredicate, predicates);
+ if (aggregatedOuter != null) {
+ return aggregatedOuter;
+ }
+ // The original rewrite is known to be not equivalent for this
subquery and the rewrite
+ // above cannot be applied safely: report the subquery as
unsupported instead of building
+ // a plan whose result is wrong.
+ throw new AnalysisException("Unsupported correlated subquery with
grouping and/or aggregation "
+ + apply.right());
+ }
+
+ // pull up correlated filter into apply node: the inner side of every
correlated predicate
+ // becomes a group by column and an output column of the aggregation
below the filter, so that
+ // the aggregation of one outer row is the aggregation of the rows of
its own key, and every
+ // aggregate above that aggregation groups the rows of its child by
the same keys (a scalar
+ // subquery keeps the rows of its aggregation through an aggregation
which SubqueryToApply adds
+ // above it, and those rows may not be mixed between two correlation
keys either)
+ List<Expression> newGroupby =
Utils.getUnCorrelatedExprs(correlatedPredicate, apply.getCorrelationSlot());
Map<Expression, Slot> unCorrelatedExprToSlot = Maps.newHashMap();
+ List<NamedExpression> newGroupbyOutputs =
Lists.newArrayListWithCapacity(newGroupby.size());
for (Expression expression : newGroupby) {
if (expression instanceof Slot) {
- newAggOutput.add((NamedExpression) expression);
+ newGroupbyOutputs.add((NamedExpression) expression);
} else {
Alias alias = new Alias(expression);
unCorrelatedExprToSlot.put(expression, alias.toSlot());
- newAggOutput.add(alias);
+ newGroupbyOutputs.add(alias);
}
}
+ // the keys which the aggregates above the deepest one group by: the
slots the keys have in
+ // the output of the aggregation below them
+ List<NamedExpression> keySlots = newGroupbyOutputs.stream()
+
.map(NamedExpression::toSlot).collect(ImmutableList.toImmutableList());
correlatedPredicate = ExpressionUtils.replace(correlatedPredicate,
unCorrelatedExprToSlot);
- LogicalAggregate newAgg = new LogicalAggregate<>(newGroupby,
newAggOutput,
-
PlanUtils.filterOrSelf(ImmutableSet.copyOf(unCorrelatedPredicate),
filter.child()));
+ Map<LogicalAggregate<?>, Plan> newAggregations = new
IdentityHashMap<>();
+ for (LogicalAggregate<?> aggregate : aggregation.aggregationChain()) {
+ boolean isTheAggregationOfTheDomain = aggregate ==
aggregation.domainAggregation();
+ List<Expression> groupBy = Lists.newArrayList(
+ isTheAggregationOfTheDomain ? newGroupby : keySlots);
Review Comment:
[P1] Preserve the empty-input row at every global aggregate in the chain.
For `EXISTS (SELECT MAX(c) FROM (SELECT COUNT(*) c FROM i WHERE i.k=o.k GROUP
BY i.g) x HAVING MAX(c) IS NULL)`, an outer key with no matching `i` rows makes
the lower grouped aggregate return no rows, but the upper global `MAX` must
still return one `NULL` row, so the original EXISTS is true. This loop also
groups that upper aggregate by the correlation key; with no lower row carrying
the key, it now emits nothing and the semi join drops the outer row. The added
nested case compares `m <= ...`, which rejects NULL and misses this boundary.
Seed/guard the key domain at each global stage, or reject chains where an upper
global aggregate can observe an empty child.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
