jaredmdobson opened a new pull request, #70726:
URL: https://github.com/apache/doris/pull/70726

   ### Background
   
   In cloud mode the FE keeps no transaction state of its own: 
`CloudGlobalTransactionMgr` asks the meta service. `getTransactionId(dbId, 
label)` resolves a label to its latest transaction and backs `SHOW TRANSACTION 
WHERE LABEL = '...'` (which reports `null` as "transaction with label ... does 
not exist") and the broker-load label check. `getTransactionIdByLabel(dbId, 
label, statuses)` backs stream load 2PC commit and abort by label. The meta 
service answers `get_txn` and `get_txn_id` with `TXN_ID_NOT_FOUND` when the 
label has no transaction, and with other codes (`KV_TXN_GET_ERR`, 
`MAX_QPS_LIMIT`, ...) or an RPC failure when the lookup itself failed.
   
   ### What problem does this PR solve?
   
   Issue Number: #70586
   Related PR: #41718 (introduced `getTransactionId` for cloud SHOW TRANSACTION)
   PR Type: Bugfix
   
   Problem Summary:
   
   In cloud mode, `SHOW TRANSACTION FROM db WHERE LABEL = 'l'` can answer 
`transaction with label l does not exist` for a label whose transaction exists 
and is VISIBLE: `getTransactionId` catches every lookup failure and returns 
`null`. We saw this on 4.2.0-rc03 for labels a few minutes old, while `GET 
/api/<db>/get_load_state?label=l` returned VISIBLE for the same labels. A 
loader that trusts the answer resubmits the label and gets 
`LABEL_ALREADY_EXISTS`. Stream load 2PC by label has the same flaw. After this 
PR only the meta service's `TXN_ID_NOT_FOUND` means "not found"; every other 
failure is reported as an error with the meta service's message.
   
   ### Details
   
   - **Frequency:** Intermittent. Triggers whenever the meta-service lookup 
fails or is throttled (RPC failure after the proxy's retries, `KV_TXN_GET_ERR`, 
`MAX_QPS_LIMIT`, `KV_TXN_CONFLICT` after three tries, ...), and, for a label 
still in flight, when the label's status changes between the two calls 
`getTransactionId` made. We hit it twice in one evening on a stream load 
workload that checks labels often; the FE log had rotated, so which failure it 
was is inferred from the code, not observed.
   - **Root cause:** `CloudGlobalTransactionMgr.getTransactionId` called 
`getLabelState` and then `getTransactionIdByLabel` inside `try { ... } catch 
(UserException e) { return null; }`, so a failed lookup and an absent label 
both came back as `null`. `getTransactionIdByLabel` turned an `RpcException` 
and every non-OK `get_txn_id` code into `TransactionNotFoundException`.
   - **Why existing tests missed it:** `CloudGlobalTransactionMgrTest` has no 
case for `getTransactionId`, `getLabelState` or `getTransactionIdByLabel`. Its 
other cases mock `MetaServiceProxy`, but only on the commit, begin and abort 
paths. `query_p0/show/test_show_transaction` checks only the success path 
against a healthy meta service, and no regression suite can make the meta 
service fail a lookup on demand.
   - **Fix:** `getTransactionId` reads the label with one `get_txn` call (the 
same call `getLabelState` makes; its `TxnInfoPB` already carries the latest 
transaction id), returns `null` only for `TXN_ID_NOT_FOUND`, and throws 
`AnalysisException` with the meta service's message otherwise. The interface 
method already declares `throws AnalysisException`, and `SHOW TRANSACTION` and 
`BrokerLoadJob` already handle it. `getTransactionIdByLabel` keeps 
`TransactionNotFoundException` for `TXN_ID_NOT_FOUND` and throws 
`UserException` for other failures. In the meta service, `get_txn` by label now 
answers `TXN_ID_NOT_FOUND` for a label that holds only its version stamp: 
`begin_txn` commits the label that way before it writes the label's first 
transaction, and `get_txn` answered `PROTOBUF_PARSE_ERR` in that window, where 
`get_txn_id` already answers `TXN_ID_NOT_FOUND`. Without it, the FE change 
would turn that normal window into an error. `BrokerLoadJob.findSelfTxnByLabel` 
still catches and logs
  any lookup error, so broker load behaves as before.
   - **Behavior change:** a failed lookup is now an error with the meta 
service's message, where it was `transaction with label ... does not exist` 
(SHOW TRANSACTION) or `transaction not found, label=...` (2PC by label). A 
missing label answers as before.
   - **Other paths and branches:** `getTransactionState(dbId, txnId)` has the 
same pattern: it returns `null` on any lookup failure, and commit reports that 
as a missing transaction (#70586). SHOW TRANSACTION and 2PC call it after 
resolving the label, so a failure there can still read as "does not exist". It 
is left for a follow-up because its `null` contract is shared with 
shared-nothing mode and many callers. Shared-nothing `GlobalTransactionMgr` 
reads local memory and is not affected. `getTransactionId` has swallowed errors 
since #41718 (3.0.3). Checked: branch-3.1, branch-4.0, branch-4.1 and 
branch-4.2 all have the bug.
   
   ### Testing
   
   New FE unit tests in `CloudGlobalTransactionMgrTest`, with 
`MetaServiceProxy` mocked:
   - `testGetTransactionIdByLabelFailsOnLookupError` (`KV_TXN_GET_ERR`, 
`MAX_QPS_LIMIT`, `INVALID_ARGUMENT`, `UNDEFINED_ERR`, `RpcException`): expects 
`AnalysisException`. Before the fix `getTransactionId` returned `null`.
   - `testGetTransactionIdByMissingLabelReturnsNull`: `TXN_ID_NOT_FOUND` still 
gives `null`.
   - `testGetTransactionIdReadsTxnIdFromGetTxn` (PREPARED, PRECOMMITTED, 
COMMITTED, VISIBLE, ABORTED): the id comes from one `get_txn` call by label, 
and `get_txn_id` is never called.
   - `testGetTransactionIdByLabelWithStatusReportsOnlyMissingAsNotFound`: only 
`TXN_ID_NOT_FOUND` gives `TransactionNotFoundException`.
   
   New meta-service unit test `MetaServiceTest.GetTxnByLabelTest`: from the 
`begin_txn:after:commit_txn:1` sync point (the label committed with only its 
version stamp), `get_txn` and `get_txn_id` by label both answer 
`TXN_ID_NOT_FOUND`; before the fix `get_txn` answered `PROTOBUF_PARSE_ERR`. 
After `begin_txn`, `get_txn` by label returns the new transaction.
   
   None of these were run locally (no local FE or cloud build); this PR relies 
on the FE UT and cloud UT pipelines.
   
   ### Release note
   
   In cloud mode, `SHOW TRANSACTION ... WHERE LABEL = '...'` and stream load 
2PC by label now report a failed meta-service lookup as an error, instead of 
saying the transaction does not exist.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details)
   - Does this need documentation: No
   - Pre-submission status: Partially verified: the FE and meta-service unit 
tests have not been run locally (no local build); CI runs them.
   
   ### AI Disclosure
   
   - Tool: Claude Code (Claude Opus 5.5)
   - Generated: code, tests, this description
   - Human review: pending; the author reviews the generated code and tests 
before this PR leaves draft
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to