jaredmdobson opened a new pull request, #70764:
URL: https://github.com/apache/doris/pull/70764

   ## Design summary
   
   - **New MetaService RPC, `scan_txn_label`.** It range-scans the database's 
`txn_label` keys under the literal prefix of the pattern. It then batch-reads 
the `txn_info` of every attempt recorded under each label. The scan stays 
inside the database's key range and never reaches another database or instance.
   - **Bounded per call.** Each call is limited by:
     - the label keys it reads (`max_scan_txn_label_keys`, 2,000);
     - the `txn_info` keys it reads (same 2,000 cap);
     - the rows it returns (`max_scan_txn_label_limit`, 1,000);
     - a time budget (`scan_txn_label_max_duration_ms`, 2,000 ms);
     - the remaining key allowance the FE sends.
   
     The call stops when it hits any of these and returns an opaque `next_key`.
   - **Pagination that always advances.** A cursor points either at the next 
unread attempt inside a label (so a label with many retried attempts can span 
pages) or just past a finished label. Every OK page returns rows, moves the 
cursor forward, or ends the scan. A FoundationDB timeout or `TXN_TOO_OLD` after 
some progress returns the rows read so far and a cursor to the first unread 
item. The same error before any progress is returned as an error. The server 
rejects malformed cursors with `INVALID_ARGUMENT`.
   - **FE behavior.**
     - `CloudGlobalTransactionMgr.getDbTransInfoByLabelMatch` derives the 
literal prefix from the regex `ShowTransactionCommand` builds, and uses an 
empty prefix whenever pruning would be unsafe.
     - It pages through the scan, applies the regex, and keeps the newest 
`cloud_show_txn_label_match_max_rows` (1,000) matches by transaction ID, newest 
first.
     - The statement is limited to `cloud_show_txn_label_match_max_scan_keys` 
(100,000) keys and to `cloud_show_txn_label_match_timeout_ms` (30 s) or 
`query_timeout`, whichever is lower. Both are checked between pages, along with 
connection kill.
     - If the scan cannot finish within those limits, the statement fails and 
asks for a longer literal prefix. It never returns a silently partial list.
   - **Compatibility.** The proto changes are additive. An FE that calls an 
older MetaService gets `UNIMPLEMENTED` (bRPC maps an unknown method to it), 
which surfaces as "Please upgrade MetaService …". Upgrade MetaService before FE.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   
   Related PR: None
   
   Problem Summary:
   
   In cloud mode, `SHOW TRANSACTION FROM db WHERE LABEL LIKE 'x%'` fails with 
"Not supported in cloud mode". Shared-nothing serves it from the FE's in-memory 
transaction manager. In cloud mode transactions live in MetaService, and no RPC 
could list them by label. This PR adds a bounded, paged label scan to 
MetaService and uses it from the cloud transaction manager.
   
   ### Details
   
   - **Change map:** about 440 production lines and 600 test lines.
     - MetaService: `meta_service_txn.cpp` (`scan_txn_label`), 
`meta_service.h`, `meta_service_helper.h`, `keys.{h,cpp}` 
(`txn_label_key_prefix`), `bvars`, `config.h`. Proto: `cloud.proto` 
(`ScanTxnLabelRequest`/`Response`).
     - FE: `CloudGlobalTransactionMgr.java`, `MetaServiceClient.java`, 
`MetaServiceProxy.java`, `Config.java`.
   - **Scope:** cloud mode, `SHOW TRANSACTION … WHERE LABEL LIKE`. `WHERE 
STATUS =` stays unsupported in cloud mode.
   - **Default behavior:** shared-nothing is unchanged. Cloud returns at most 
the newest 1,000 matches. Shared-nothing has no row cap.
   - **Known limits:**
     - If the statement key budget runs out exactly when a range read fills its 
limit, the statement can fail even though no keys are left. Raise 
`cloud_show_txn_label_match_max_scan_keys` or narrow the pattern.
     - Keys read by MetaService's internal retries are not charged across 
attempts.
     - `KILL QUERY` is noticed between pages, not during an in-flight RPC.
   - **Privileges:** unchanged. The existing database-level check in 
`ShowTransactionCommand` applies.
   
   ### Testing
   
   UT Job on a Linux ARM64 build of the PR head:
   
   | Step | Result |
   |---|---|
   | FE `CloudGlobalTransactionMgrTest` | 73 run, 0 failed |
   | FE Checkstyle | passed |
   | Cloud `meta_service_test`, all enabled tests except `Fdb/*`, which needs a 
real FoundationDB | 313 run, 0 failed, including `ScanTxnLabel`, 
`ScanTxnLabelFinishesAtExactBudget` and 
`ScanTxnLabelResumesAfterSecondRangeGetFails` |
   | `doris_cloud` link | passed |
   | clang-format 16 on changed C++ | passed |
   
   New coverage:
   
   - MetaService:
     - prefix ranges, including escaped bytes and adjacent database IDs;
     - labels with several attempts, and labels holding only a versionstamp;
     - recycled `txn_info`;
     - limit clamping and per-request key allowance;
     - an exact-budget finish;
     - cursors within a label and between labels, checked to always advance 
(128/130/8 pages);
     - malformed, truncated, overflowing and legacy cursors;
     - injected timeout and `TXN_TOO_OLD` on the batch read and on the second 
range read, draining all 1,001 IDs in order with no duplicates.
   - FE:
     - prefix derivation;
     - regex filter;
     - newest-N across pages;
     - fixed page size;
     - scan-key budget and timeout errors;
     - unchanged-cursor rejection;
     - the old-MetaService error.
   - Regression: `cloud_p0/test_cloud_show_txn_label_like.groovy`. It runs two 
stream loads that share a label prefix and one that doesn't, and checks the 
result and its ordering. It was not run in a cloud regression environment for 
this PR.
   
   ### Release note
   
   Cloud mode supports `SHOW TRANSACTION FROM db WHERE LABEL LIKE '...'`, 
returning up to the newest 1,000 matching transactions. Upgrade MetaService 
before FE.
   
   ### Check List (For Author)
   
   - Behavior changed: Yes (see Details)
   - Does this need documentation: Yes. The SHOW TRANSACTION reference should 
state cloud support and the row cap. A doris-website PR will follow.
   - Pre-submission status: Partially verified. The unit tests, checkstyle, 
link and format passed (see Testing). The cloud regression suite was not run.
   
   ### AI Disclosure
   
   - Tool: Codex (implementation and tests), Claude Code (specs, reviews and 
verification).
   - Generated: Code, tests and this description.
   - Human review: Pending. The author reviews before this PR leaves draft.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to