airborne12 opened a new pull request, #66877:
URL: https://github.com/apache/doris/pull/66877
### What problem does this PR solve?
Issue Number: None
Related PR: #66052
Problem Summary: SNII's count-only shortcut trusted the persisted dictionary
document frequency without checking it against Core document statistics. A
corrupted 10-row segment whose posting contains two documents but whose
dictionary entry records df=100 could fabricate a 100-id bitmap in both
nullable and non-nullable paths, producing an inflated count after segment
masking. Validate df against both total and indexed document counts before
fabricating the count-shaped bitmap and report the cross-section mismatch as
index corruption. Add CRC-valid nullable and non-nullable fixtures that rewrite
only df while preserving real two-document postings.
### Release note
Fix SNII count-only queries to reject corrupted document frequencies outside
the segment document domain.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- SniiIndexReaderCountFallback.* (32 tests)
- ./build.sh --be -j 192
- Behavior changed:
- [ ] No.
- [x] Yes. Corrupted SNII indexes now return an index-corruption error
instead of an inflated count.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
