yujun777 opened a new pull request, #70102:
URL: https://github.com/apache/doris/pull/70102
### What problem does this PR solve?
Related PR: #43546, #50574
Sampled column statistics can report NDV=0 and numNulls=count while the
independently collected MIN/MAX are non-null. The legacy StatsCalculator
accessor treats this known sampling pattern as UNKNOWN, but the newer OLAP
table accessor bypasses that conversion.
Consequently, even an unused column can cause the NDV validation to disable
join reordering for the entire statement. A small preserved input can remain on
the probe side of a LEFT OUTER JOIN while a much larger input is built, losing
the opportunity to filter the large input with runtime filters.
Share the existing UNKNOWN conversion across optimizer table-statistics
reads, including join-reorder validation, unpruned scans, partition-pruned
scans and fallback from missing partition statistics. Normalize before changing
count or scaling null counts. Keep raw cached statistics available for
diagnostics and preserve the existing handling of other invalid statistics.
### Release note
Fix unnecessarily disabled join reordering when sampled all-null column
statistics have non-null MIN/MAX values.
### Check List (For Author)
- Test:
- [x] Unit Test: real OLAP cache access, unused-column outer-join
swapping, valid/invalid boundary cases, and partition-pruned table-statistics
fallback.
- Behavior changed:
- [x] Yes. Treat this known sampling pattern as UNKNOWN consistently in
optimizer table-statistics reads.
- Does this need documentation?
- [x] No.
Validation: `InvalidColumnStatsReorderTest,StatsCalculatorTest` passed (23
tests, no failures/errors). Standalone Maven validate with dependencies also
passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
