bobhan1 opened a new pull request, #66092:
URL: https://github.com/apache/doris/pull/66092
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: During a large cloud load, lazy commit can finish its first
metadata phase but fail to complete the background promotion before the commit
RPC returns. FE previously treated this response like a fully published
transaction and notified BEs to make temporary rowsets visible immediately. A
BE compaction could then consume rowsets that were not yet visible in
meta-service, racing the lazy commit metadata update.
This change adds a partial-commit marker to `CommitTxnResponse`.
Meta-service keeps the marker set until the lazy commit task completes
successfully. FE skips the BE fast-path rowset promotion for marked responses,
allowing BEs to observe the final rowset metadata from meta-service.
### Release note
Fix a cloud metadata consistency issue between large transaction lazy commit
and BE compaction.
### Check List (For Author)
- Test
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test
- [ ] No need to test
Test commands:
```shell
./run-fe-ut.sh --run
org.apache.doris.cloud.transaction.CloudGlobalTransactionMgrTest
BUILD_TYPE=ASAN ./run-cloud-ut.sh --run
--filter=txn_lazy_commit_test:TxnLazyCommitTest.ConcurrentCommitTxnEventuallyCase4Test
-j 100
BUILD_TYPE=ASAN ./run-cloud-ut.sh --run
--filter=txn_lazy_commit_test:TxnLazyCommitTest.CommitTxnEventuallyWithAbortAfterCommitTest
-j 100
./build.sh --fe
build-support/check-format.sh
```
- Behavior changed:
- [ ] No
- [x] Yes. FE no longer sends BE rowset promotion tasks for partial lazy
commits.
- Does this need documentation?
- [x] No
- [ ] Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
