englefly commented on code in PR #69817:
URL: https://github.com/apache/doris/pull/69817#discussion_r4452300523
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/stats/FilterEstimation.java:
##########
@@ -147,7 +149,13 @@ public Statistics visitOr(Or or, EstimationContext
context) {
List<Expression> children = or.children();
Set<Slot> leftInputSlots =
Sets.newHashSet(children.get(0).getInputSlots());
Statistics leftStats = children.get(0).accept(this, context);
- leftStats.normalizeColumnStatistics(context.statistics.getRowCount(),
true);
+ // A predicate can return key column statistics whose ndv is larger
than its output row count.
+ // For example `a = b` on 1000 rows with ndv 1000 estimates one row
but keeps ndv 1000 from the
+ // intersected range. The children already scaled their null counts
with their own selectivity, so
+ // only repair the validity here instead of rescaling the null counts
a second time; otherwise the
+ // overlap estimated below, which uses the left child's ndv, is far
too small and the union is
+ // overestimated.
+ leftStats.enforceValid();
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
