airborne12 opened a new pull request, #69700:
URL: https://github.com/apache/doris/pull/69700
### What problem does this PR solve?
Issue Number: None
Related PR: #60331
Problem Summary:
SELECT score() FROM t WHERE c1 MATCH_PHRASE_PREFIX 'apple r' ORDER BY score()
DESC LIMIT 10 fails with "[E3] UnionTermIterator does not support scoring",
while the same filter without score() works, and 4.0.3 returned a score.
The last term of a phrase prefix is a UnionTermIterator over the terms the
prefix expands to. PhraseQuery sorts its iterators by document frequency and
scores a row with the norm of the first one. When the terms the prefix
expands
to are rarer than another term of the phrase in a segment, the union comes
first and UnionTermIterator::norm() threw. A union's document frequency is
the
sum of its terms', so whether a query fails depends on the data of each
segment. On a table holding "apple red", "apple green" and "banana yellow",
`apple r` (red in 1 row, apple in 2) and `apple g` fail; on a table where
`r*` matches more rows than apple the same query works. This regressed when
the backport of the phrase-level BM25 scoring of #60331 (#61243) started
reading the norm of the first iterator.
A norm belongs to the document, not to a term, so the sub-iterator positioned
on the row, the top of the union's queue, reports it. The union now returns
that norm.
This is the same change as the one verified on branch-4.1 (#69699), where
union_term_iterator.h is identical apart from its include paths. It is not a
cherry-pick of master, which replaced the phrase query engine (#69698).
### Release note
Fix score() over MATCH_PHRASE_PREFIX failing with "UnionTermIterator does not
support scoring" when the prefix matches fewer rows than the other terms.
### Check List (For Author)
- Test: Unit Test (not built locally)
- Adds UnionTermIteratorTest.NormIsReadFromTheSubIteratorOnTheDocument.
branch-4.0 was not built or run here; the identical change and test
pass on branch-4.1, and the regression case of that PR is not added
here because its expected output has to be generated on a branch-4.0
cluster.
- Behavior changed: Yes (the query no longer fails)
- Does this need documentation: No
### Notes for reviewers
Please rely on CI for this branch: the change was not compiled or run on
branch-4.0 locally. The 4.1 PR carries a regression suite; it can be ported
here once its `.out` is generated on a branch-4.0 cluster.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
