englefly opened a new pull request, #66853:
URL: https://github.com/apache/doris/pull/66853
### What problem does this PR solve?
Issue Number: N/A
Problem Summary: When a materialized view is selected, the scan stats use
the estimated row count derived from the mv generating plan even though the
actual row count of the mv is accurate, which hides the estimation error of the
base table stats. This change calibrates the estimated stats by the actual row
count of the mv, enabled by the new session variable
enable_materialized_view_stats_calibration (default off for gray release): the
scan row count is corrected to the actual value, and the column stats are
calibrated by the clamped ratio while keeping the internal consistency of the
estimate, e.g. the group by key ndv follows the actual row count in the single
group by key case, the aggregate function output ndv is never scaled, and the
passthrough high cardinality ndv scales with the clamped ratio. The column
classification (group by key / aggregate function output / passthrough) is
derived from the mv defining plan structure and registered to the statement
context when the mv
statistics is registered. The change also fixes an existing bug that the
shared estimated stats registered in the statement context is polluted in-place
by the missing column fill: the returned stats is a deep copy now.
### Release note
None
### Check List (For Author)
- Test: FE unit test MvStatsCalibrationTest (6 cases covering group by key
calibration, filter reduced column, high cardinality passthrough, disabled
fallback, unreported row count, shared stats non-pollution); regression suite
mv_stats_calibration; existing mv suites aggregate_with_roll_up /
any_value_roll_up / partition_mv_rewrite_dimension_2_3 all pass and the mv is
selected as before.
- Behavior changed: Yes (a new session variable
enable_materialized_view_stats_calibration, default off)
- Does this need documentation: No
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
