bobhan1 opened a new pull request, #62256:
URL: https://github.com/apache/doris/pull/62256
### What problem does this PR solve?
Issue Number: close #DORIS-25014
Problem Summary:
During cloud schema change, compaction on the new tablet may produce rowsets
with wider version ranges (e.g. `[818-822]`) that overlap with individual SC
output rowsets (e.g. `[818],[819],...,[822]`). The MS correctly recycles these
compaction rowsets on SC commit, but the BE side did not mirror this behavior,
leaving stale rowsets visible and causing duplicate keys in MOW tables.
Root cause: PR #61089 enabled compaction on new tablets during SC. When
compaction merges versions `818-822` into a single rowset `[818-822]` before SC
finishes, and SC then produces individual rowsets `[818],[819],...,[822]`, the
BE-side `add_rowsets` overlap check (`to_add_v.contains(v)`) fails to detect
the overlap because `[818].contains([818-822])` is false. The stale compaction
rowset remains in `_rs_version_map`, and since its delete bitmap does not cover
the SC output rows, duplicate keys appear.
Fix: Before calling `add_rowsets` for SC output, delete all local rowsets in
`[2, alter_version]` from the new tablet, mirroring the MS-side recycle
behavior. A new `CloudTablet::delete_rowsets_for_schema_change` method is added
that also removes edges from the version graph, preventing the greedy capture
algorithm from preferring the wider stale compaction path over the individual
SC output rowsets.
### Release note
None
### Check List (For Author)
- Test
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] No.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
