github-actions[bot] commented on code in PR #66898:
URL: https://github.com/apache/doris/pull/66898#discussion_r4312314272
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/rewrite/RewriteCteChildren.java:
##########
@@ -238,4 +262,62 @@ private LogicalPlan tryToConstructFilter(CascadesContext
cascadesContext, CTEId
}
return child;
}
+
+ /**
+ * Marks the slots of the producer of {@code cteAnchor} that the
estimation of its consumers uses
+ * as operative, i.e. the union over all the consumers of the slots of the
producer they demand.
+ *
+ * <p>The producer subtree alone does not tell which of its columns are
read: the demand comes
+ * from the consumers, each of which copies the slot of the producer it
reads, and the producer
+ * only forwards that slot through projections, which create no demand of
their own. Without this
+ * seeding the derivation marks the whole output of the producer as not
operative, and the
+ * statistics derivation only probes those columns from the cache: a miss
leaves them unknown and
+ * the estimation of the consumer, e.g. the local distinct inference of a
set operation over the
+ * CTE, then works on unknown statistics.
+ *
+ * <p>A slot can be replaced more than once while a statement is
rewritten, so the slot of the
+ * producer that a consumer keeps is resolved through the {@link
SlotReplacementMapping} of the
+ * statement, which follows the whole replacement chain, and not only its
first replacement.
+ */
+ static void seedProducerOperativeSlots(LogicalCTEAnchor<?, ?> cteAnchor,
+ CascadesContext cascadesContext,
OperativeColumnDerive.DeriveContext consumerDemand,
+ OperativeColumnDerive.DeriveContext deriveContext) {
+ SlotReplacementMapping replacements =
cascadesContext.getStatementContext()
+ .getSlotReplacementMapping();
+ cteAnchor.child(1).foreach(plan -> {
+ if (plan instanceof LogicalCTEConsumer) {
+ LogicalCTEConsumer consumer = (LogicalCTEConsumer) plan;
+ if (consumer.getCteId().equals(cteAnchor.getCteId())) {
+ for (Entry<Slot, Slot> consumerToProducer :
consumer.getConsumerToProducerOutputMap().entrySet()) {
+ if (consumerDemand.operativeSlotIds
+
.contains(consumerToProducer.getKey().getExprId().asInt())) {
Review Comment:
[P2] Propagate demand through dependent retained CTEs
Both CTEs remain materialized under the default settings for:
```sql
WITH c1 AS (SELECT a FROM wide),
c2 AS (SELECT a FROM c1 UNION ALL SELECT a FROM c1)
SELECT a FROM c2 INTERSECT SELECT a FROM c2
```
The reduced tree is:
```text
Anchor(c1)
Producer -> Project(a) -> Scan(wide)
Anchor(c2)
Producer -> Union ALL(Consumer(c1), Consumer(c1))
Intersect DISTINCT(Consumer(c2), Consumer(c2))
```
The first `consumerDemand` walk marks only the c2 consumer slots.
`OperativeColumnDerive` does not translate CTE-consumer IDs into producer IDs,
so this filter finds no demanded c1 consumer and leaves `wide.a` derived-empty.
`visitLogicalCTEAnchor` then runs `StatsDerive` on c1 before descending into
c2: on a cache miss, a is only probed and its producer gets UNKNOWN statistics.
Seeding c2 later cannot repair that cached producer, so
`InferSetOperatorDistinct` skips useful local deduplication even for analyzed
low-NDV a. The direct single-CTE mapping is repaired; this is the additional
dependent-anchor hop in the new helper. Resolve demand transitively across
retained producers before the first prerequisite statistics pass and cover this
two-anchor load-request path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
