jaredmdobson opened a new pull request, #70764:
URL: https://github.com/apache/doris/pull/70764
## Design summary
- **New MetaService RPC, `scan_txn_label`.** It range-scans the database's
`txn_label` keys under the literal prefix of the pattern. It then batch-reads
the `txn_info` of every attempt recorded under each label. The scan stays
inside the database's key range and never reaches another database or instance.
- **Bounded per call.** Each call is limited by:
- the label keys it reads (`max_scan_txn_label_keys`, 2,000);
- the `txn_info` keys it reads (same 2,000 cap);
- the rows it returns (`max_scan_txn_label_limit`, 1,000);
- a time budget (`scan_txn_label_max_duration_ms`, 2,000 ms);
- the remaining key allowance the FE sends.
The call stops when it hits any of these and returns an opaque `next_key`.
- **Pagination that always advances.** A cursor points either at the next
unread attempt inside a label (so a label with many retried attempts can span
pages) or just past a finished label. Every OK page returns rows, moves the
cursor forward, or ends the scan. A FoundationDB timeout or `TXN_TOO_OLD` after
some progress returns the rows read so far and a cursor to the first unread
item. The same error before any progress is returned as an error. The server
rejects malformed cursors with `INVALID_ARGUMENT`.
- **FE behavior.**
- `CloudGlobalTransactionMgr.getDbTransInfoByLabelMatch` derives the
literal prefix from the regex `ShowTransactionCommand` builds, and uses an
empty prefix whenever pruning would be unsafe.
- It pages through the scan, applies the regex, and keeps the newest
`cloud_show_txn_label_match_max_rows` (1,000) matches by transaction ID, newest
first.
- The statement is limited to `cloud_show_txn_label_match_max_scan_keys`
(100,000) keys and to `cloud_show_txn_label_match_timeout_ms` (30 s) or
`query_timeout`, whichever is lower. Both are checked between pages, along with
connection kill.
- If the scan cannot finish within those limits, the statement fails and
asks for a longer literal prefix. It never returns a silently partial list.
- **Compatibility.** The proto changes are additive. An FE that calls an
older MetaService gets `UNIMPLEMENTED` (bRPC maps an unknown method to it),
which surfaces as "Please upgrade MetaService …". Upgrade MetaService before FE.
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
In cloud mode, `SHOW TRANSACTION FROM db WHERE LABEL LIKE 'x%'` fails with
"Not supported in cloud mode". Shared-nothing serves it from the FE's in-memory
transaction manager. In cloud mode transactions live in MetaService, and no RPC
could list them by label. This PR adds a bounded, paged label scan to
MetaService and uses it from the cloud transaction manager.
### Details
- **Change map:** about 440 production lines and 600 test lines.
- MetaService: `meta_service_txn.cpp` (`scan_txn_label`),
`meta_service.h`, `meta_service_helper.h`, `keys.{h,cpp}`
(`txn_label_key_prefix`), `bvars`, `config.h`. Proto: `cloud.proto`
(`ScanTxnLabelRequest`/`Response`).
- FE: `CloudGlobalTransactionMgr.java`, `MetaServiceClient.java`,
`MetaServiceProxy.java`, `Config.java`.
- **Scope:** cloud mode, `SHOW TRANSACTION … WHERE LABEL LIKE`. `WHERE
STATUS =` stays unsupported in cloud mode.
- **Default behavior:** shared-nothing is unchanged. Cloud returns at most
the newest 1,000 matches. Shared-nothing has no row cap.
- **Known limits:**
- If the statement key budget runs out exactly when a range read fills its
limit, the statement can fail even though no keys are left. Raise
`cloud_show_txn_label_match_max_scan_keys` or narrow the pattern.
- Keys read by MetaService's internal retries are not charged across
attempts.
- `KILL QUERY` is noticed between pages, not during an in-flight RPC.
- **Privileges:** unchanged. The existing database-level check in
`ShowTransactionCommand` applies.
### Testing
UT Job on a Linux ARM64 build of the PR head:
| Step | Result |
|---|---|
| FE `CloudGlobalTransactionMgrTest` | 73 run, 0 failed |
| FE Checkstyle | passed |
| Cloud `meta_service_test`, all enabled tests except `Fdb/*`, which needs a
real FoundationDB | 313 run, 0 failed, including `ScanTxnLabel`,
`ScanTxnLabelFinishesAtExactBudget` and
`ScanTxnLabelResumesAfterSecondRangeGetFails` |
| `doris_cloud` link | passed |
| clang-format 16 on changed C++ | passed |
New coverage:
- MetaService:
- prefix ranges, including escaped bytes and adjacent database IDs;
- labels with several attempts, and labels holding only a versionstamp;
- recycled `txn_info`;
- limit clamping and per-request key allowance;
- an exact-budget finish;
- cursors within a label and between labels, checked to always advance
(128/130/8 pages);
- malformed, truncated, overflowing and legacy cursors;
- injected timeout and `TXN_TOO_OLD` on the batch read and on the second
range read, draining all 1,001 IDs in order with no duplicates.
- FE:
- prefix derivation;
- regex filter;
- newest-N across pages;
- fixed page size;
- scan-key budget and timeout errors;
- unchanged-cursor rejection;
- the old-MetaService error.
- Regression: `cloud_p0/test_cloud_show_txn_label_like.groovy`. It runs two
stream loads that share a label prefix and one that doesn't, and checks the
result and its ordering. It was not run in a cloud regression environment for
this PR.
### Release note
Cloud mode supports `SHOW TRANSACTION FROM db WHERE LABEL LIKE '...'`,
returning up to the newest 1,000 matching transactions. Upgrade MetaService
before FE.
### Check List (For Author)
- Behavior changed: Yes (see Details)
- Does this need documentation: Yes. The SHOW TRANSACTION reference should
state cloud support and the row cap. A doris-website PR will follow.
- Pre-submission status: Partially verified. The unit tests, checkstyle,
link and format passed (see Testing). The cloud regression suite was not run.
### AI Disclosure
- Tool: Codex (implementation and tests), Claude Code (specs, reviews and
verification).
- Generated: Code, tests and this description.
- Human review: Pending. The author reviews before this PR leaves draft.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]