englefly opened a new pull request, #70480:
URL: https://github.com/apache/doris/pull/70480
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary:
Runtime filters built from a table stream's incremental scans must be
retained. Previously, `RuntimeFilterPruner` could remove them when join-column
statistics appeared non-selective or were unknown, even though base-table
statistics do not describe the incremental data.
Mark `PhysicalOlapScan` nodes backed by `OlapChangelogTableWrapper` as
`NATIVE` effective sources. Existing effective-source propagation then
preserves their runtime filters through filters, projects, aggregates, and
joins. Ordinary OLAP scans retain their existing pruning behavior.
Add tests for `MIN_DELTA` and `APPEND_ONLY` build sources with known and
unknown statistics, aggregation on the build side, and the negative case where
a stream is only the probe side.
### Release note
Preserve runtime filters built from incremental table-stream scans even when
static statistics appear non-selective or are unknown.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- `bash run-fe-ut.sh --run
org.apache.doris.nereids.processor.post.RuntimeFilterPrunerTest` — 13 tests
passed, 0 failures, 0 errors.
- `cd fe && mvn checkstyle:check -pl fe-core` — 0 violations.
- `git diff --check` — passed.
- [ ] Manual test
- [ ] No need to test or manual test.
- Behavior changed:
- [ ] No.
- [x] Yes. Runtime filters built from changelog scans are retained
regardless of estimated selectivity.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
