bobhan1 opened a new pull request, #69266:
URL: https://github.com/apache/doris/pull/69266
### What problem does this PR solve?
Related PR: #67989, #67813, #68104
This follow-up checks commit inputs before publication and checks partition
version information before returning a successful response. The two checks have
different outcomes:
- **Tablet input completeness:** FE sends the number of persisted temporary
rowsets expected in each partition. MS checks these counts before the first
commit, separately for each sub-transaction when applicable. A mismatch returns
the existing `INVALID_ARGUMENT` status before any publication writes. Already
`COMMITTED` transactions skip the full input check because concurrent lazy
publication may have removed part of the input.
- **Partition response completeness:** a retry can scan only the partitions
still awaiting lazy publication and later return success with an incomplete
version map. FE sends the expected partition count; MS clears the version
results and related table statistics when the count differs, allowing FE to
invalidate its version caches.
BE reports whether a tablet actually persisted rowset metadata. This
distinguishes skipped empty rowsets from persisted zero-row rowsets and
preserves field presence through the Protobuf/Thrift reporting chain. Unknown
counts from older BE reporting paths omit the tablet check. Ordinary loads,
per-sub-transaction counts, and row-binlog tablet reports use the same presence
semantics; optional fields support MS → FE → BE rolling upgrades and a shared
MS serving older FE/BE versions.
#### Relationship to #67989
This branch preserves all three existing commits unchanged. The first two
commits are the reproducer and stale temporary-key fix in #67989. The third
adds **both** partition response validation and tablet input validation. #67989
is limited to its original two commits so its MS-only fix can be merged
independently. The incremental change in this PR is the third commit,
`6f4fef750b4ecdb23d7e96ca5137af36c8e3c387`.
#### Validation
The branch preserves the exact previously validated source tree. Existing
results for this revision:
- Real FDB, ASAN: 13 focused MS tests passed, including successful
concurrent lazy/immediate retries, empty-rowset cases, and three rejection
tests that compare all KV metadata before and after failure.
- FE: all 59 `CloudGlobalTransactionMgrTest` tests passed.
- BE: four metadata-reporting/presence tests passed in the preceding
validation; those BE sources are unchanged.
- Protobuf C++/Java generation and absent, present-empty, multi-partition,
and sub-transaction count round trips passed.
The preceding full MS test run passed 40 active tests with one existing
disabled test. It preceded the final field-number/error-code cleanup. No SQL
end-to-end test or clang-tidy run is claimed. These are prior runs, not reruns
for the branch split.
### Release note
Detect incomplete tablet commit inputs before publication when counts are
available, and invalidate FE version caches when successful commit responses
contain incomplete partition information.
### Check List (For Author)
- Test
- [x] Unit Test
- Behavior changed:
- [x] Yes. Reject incomplete first-commit inputs and invalidate
incomplete response caches when optional counts are present.
- Does this need documentation?
- [x] No. No new user-facing setting or syntax.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
