airborne12 opened a new pull request, #69246:
URL: https://github.com/apache/doris/pull/69246
### What problem does this PR solve?
Issue Number: N/A
Related PR: #69053
Problem Summary:
With a five-row English-index fixture, `SEARCH('(content:spider) OR
(content:spider)')` adds both score contributions, but `(content MATCH_ANY
'spider') OR (content MATCH_ANY 'spider')` previously returned the
single-clause score. Repeated AND clauses were also collapsed. Actual
Elasticsearch 8.11.3 `bool.should`, `bool.must`, and `query_string` queries
confirm that each repeated positive clause contributes independently.
Preserve MATCH occurrence identity before analysis and Boolean deduplication
when a query requests `score()`. Preserve those distinct conjuncts when
translating to legacy plan nodes, where structural equality otherwise collapses
repeated AND clauses again. Unscored queries retain ordinary predicate
deduplication. No BE scoring multiplier is added.
On the original five rows, both SEARCH and MATCH now return scores
`1.5087616`, `1.2797759`, and `1.273334` for duplicate clauses, exactly twice
their single-clause scores. Three repetitions produce three times the score.
Absolute BM25 values are not asserted to match Elasticsearch across engines.
### Release note
Fix MATCH scoring so each repeated positive Boolean clause contributes to
`score()`, including repeated OR and AND clauses.
### Check List (For Author)
- Test
- [x] Regression test: 23 generated golden checks using the original
fixture, including OR/AND two and three times, mixed and nested branches, score
filtering, warm execution, derived queries, direct CTEs, and unscored rows.
Normal verification passes. Existing Boolean-branch and score-cache suites also
pass; each command executed one suite with zero failures, fatal errors, or
skips.
- [x] Unit Test: 37 FE tests pass, including 7 new analysis/rewrite
tests and existing expression, MATCH validation, and score projection tests.
- [x] Manual test: actual Elasticsearch 8.11.3 oracle and three rounds
of 24 Doris SQL queries with arithmetic score assertions. Full ASAN BE/FE build
and final FE rebuild succeed.
- Behavior changed:
- [x] Yes. Repeated scored MATCH predicates add one contribution per
occurrence. Unscored predicates still deduplicate.
- Does this need documentation?
- [x] No.
Validation covers direct CTEs. A separate chained-CTE score-alias planning
error also occurs with a single MATCH or SEARCH and is outside this fix; no
successful runtime validation is claimed for that query shape. Local validation
is distinct from PR CI.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
