feiniaofeiafei opened a new pull request, #69344:
URL: https://github.com/apache/doris/pull/69344
### What problem does this PR solve?
SEARCH combined with a NOT IN subquery inside a CTE can fail analysis after
subquery expansion introduces an Apply. SEARCH fields are initially embedded in
a DSL string, so predicate pushdown cannot see their column dependencies until
they are bound. Requiring a single-scan shape before rewriting also rejects
predicates that could reach a valid scan through a join, projection or UNION.
Bind SEARCH fields to the filter's visible input slots during analysis,
before subquery expansion and predicate pushdown. Each field must resolve to
exactly one input column. Existing rewrite rules can then substitute aliases,
map UNION branches and push predicates using their actual column dependencies.
Keep the early check that SEARCH appears only in filter predicates. After
rewriting, require every surviving SEARCH to be in a filter directly above an
OLAP scan, reference supported scan-column expressions and have the required
inverted indexes. Variant subcolumn checks use the original parent column.
SEARCH eliminated during rewriting no longer requires an index. Allow temporary
expression-child replacements used by optimizer inference, with executable
children validated by the final check.
Regression coverage includes inline, materialized and reused CTEs; NULL and
empty NOT IN results; join ownership; projected and shadowed aliases; Variant
paths; UNION mappings; and invalid field, placement and index cases.
### Release note
Fix SEARCH predicate handling with CTEs and NOT IN subqueries. Allow
unambiguous SEARCH predicates over joins, projected aliases and UNION outputs
when rewriting can push them to indexed OLAP scan columns.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [x] Manual test
Validation on this branch:
- 58 FE unit tests passed:
`./run-fe-ut.sh --run
'CheckSearchUsageTest,RewriteSearchToSlotsTest,SearchExpressionTest'`
- FE Checkstyle passed with zero violations:
`mvn checkstyle:check -pl fe-core`
- Built FE on the development host, replaced the test FE libraries and
verified the deployed JAR checksum.
- Replayed these 10 SEARCH regression suites successfully: 0 failed, 0
fatal, 0 skipped. This was a targeted run, not the entire
`regression-test/suites/search` directory.
- `test_search_cte_not_in`
- `test_search_input_binding`
- `test_search_index_check_after_rewrite`
- `test_search_usage_restrictions`
- `test_search_function`
- `test_search_multi_field`
- `test_search_null_semantics`
- `test_search_variant_subcolumn_analyzer`
- `test_search_default_field_operator`
- `test_search_vs_match_consistency`
- New expected output files were generated by the regression runner, checked
against the fixture rows and replayed.
- Behavior changed:
- [x] Yes. Bind SEARCH dependencies before pushdown and validate final
scan placement and indexes after rewriting, while retaining early
filter-position and unique-field checks.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
