BiteTheDDDDt opened a new pull request, #62114:
URL: https://github.com/apache/doris/pull/62114
… is a scope range (#62027)
This pull request addresses a bug in the OLAP scan operator where `IN_LIST`
predicates could be incorrectly erased when both `MINMAX` and `IN` runtime
filters targeted the same key column, and the number of `IN` values exceeded
the maximum allowed for pushdown. The changes ensure that `IN_LIST` predicates
are preserved in such cases, preventing incorrect query results. Additionally,
a regression test is added to verify the fix.
**Bug fix in predicate handling:**
* Modified the logic in `_build_key_ranges_and_filters()` within
`olap_scan_operator.cpp` to ensure that `IN_LIST` predicates are not erased
when the key range is a scope range (e.g., `>= X AND <= Y`) and the `IN`
filter's value count exceeds
`max_pushdown_conditions_per_column`. This preserves filtering semantics
that are not captured by the scope range.
[[1]](diffhunk://#diff-3ddc75656071d9c0e6b0be450e152a1c94559f7e70ea820e7f0c80a7078e3292R972)
[[2]](diffhunk://#diff-3ddc75656071d9c0e6b0be450e152a1c94559f7e70ea820e7f0c80a7078e3292R986)
[[3]](diffhunk://#diff-3ddc75656071d9c0e6b0be450e152a1c94559f7e70ea820e7f0c80a7078e3292L986-R1013)
* Enhanced the profiling output in `_process_conjuncts()` to accurately
reflect the set of predicates that will reach the storage layer after key range
and filter construction. This helps with debugging and verification of
predicate pushdown.
**Testing and regression coverage:**
* Added a new regression test
`test_rf_in_list_not_erased_by_scope_range.groovy` to verify that `IN_LIST`
predicates are not incorrectly erased when both `MINMAX` and `IN` filters are
present and the `IN` list is too large to be absorbed into the key range.
* Added the corresponding expected output file for the new regression test.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
