luwei16 opened a new pull request, #67617:
URL: https://github.com/apache/doris/pull/67617
### What problem does this PR solve?
Issue Number: N/A
Problem Summary:
Versioned rowset reads add load metadata versions to the reader minimum
before compact rowsets remove covered loads from the returned set. With loads
L2[2,2]@80 and L3[3,3]@90 covered by A[2,3]@150, plus L4[4,4]@170, reading
[2,4] returns A and L4 but records a minimum of 80. Later compaction logs can
consequently retain successive compacted rowsets for a snapshot that only needs
L2/L3.
Keep each candidate's metadata Versionstamp alongside its rowset, then merge
only the final returned candidates into the existing minimum after both scans
succeed. The returned set, scan/coverage rules, snapshot bounds and error
propagation are unchanged. Previous stats and other real dependencies remain
tracked, and persisted historical logs are unchanged.
The regression exercises two real MetaService compactions and the Recycler
reference checker using MemTxnKv: snapshot@100 still protects the first log
containing L2/L3, while the later log has min_timestamp=150 and is no longer
protected by that snapshot.
### Release note
Fix unnecessary retention of later compacted rowsets when snapshots protect
older load rowsets already covered by compaction.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test
- Behavior changed:
- [ ] No.
- [x] Yes. Covered rowsets absent from compaction inputs no longer lower
the operation log minimum; actual snapshot dependencies remain protected.
- Does this need documentation?
- [x] No.
- [ ] Yes.
Validation on master `54f5c41220568ddcd1cf11c7ce2ba360cf1ebe73`:
- All four new tests fail against the unmodified implementation and pass
with the fix, including the 80-to-150 minimum and snapshot-reference decisions.
- Cloud ASAN UT build (`./run-cloud-ut.sh -j 16`): all 33 test executables
built.
- 59 relevant tests pass: `MetaReaderTest.*`, `CloneChainReaderTest.*`,
`MetaServiceJobVersionedReadTest.*`, `OperationLogRecycleCheckerTest.*`, and
`RecycleOperationLogTest.CoveredLoadDoesNotProtectLaterCompaction`.
- Changed-line clang-format 16, repository clang-tidy checks for all four
changed files, and `git diff --check` pass.
- No live FDB/S3 deletion or cluster deployment performed; the integration
regression uses the actual MS log and Recycler reference-checking code with
MemTxnKv.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
