airborne12 opened a new pull request, #69159:
URL: https://github.com/apache/doris/pull/69159
### What problem does this PR solve?
Related PR: #69051
Problem Summary: Backport #69051 to `branch-4.1`.
Legacy keyword SEARCH indexes lack norms and total-term-frequency metadata.
Scoring could use a zero average length or zero norm and return invalid scores.
Preserve whole keyword values, use unit document length when metadata is
absent, and keep WAND upper bounds conservative. Numeric BKD scoring remains
unsupported.
#### Branch adaptations
Backport the final net change from f22f16113644b4bbe254765329c92bdf8095593e.
Keep the branch-4.1 collection-statistics and predicate-collector paths and
TermInfo representation. Select the same reader for plain-field statistics and
execution when keyword and analyzed indexes coexist; retain the branch-specific
variant binding path. The branch has no SNII, so this PR includes V2/V3
coverage and omits SNII-only changes.
### Release note
SEARCH on legacy keyword indexes can return valid BM25 scores.
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test
- Behavior changed:
- [x] Yes. SEARCH on legacy keyword indexes can return valid BM25 scores.
- Does this need documentation?
- [x] No.
### Validation
- PASS: Clang 16 across the complete candidate merged with
`origin/branch-4.1` (`0993d5e5512b3d27649e6c44f38b13836e03ee97`).
- PASS: English code check for the complete PR diff.
- In progress: targeted ASAN BE unit tests on this backport.
- Queued after unit tests: ASAN BE/FE build and isolated SQL regression
replay, including `test_search_score_cache`.
This PR is a draft while target-branch execution checks are running.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
