zhaohai666 opened a new pull request, #10826:
URL: https://github.com/apache/rocketmq/pull/10826
# [RIP-2] Implement Proxy Admin Standardized Management Interface on the
Proxy
## Summary
This PR implements **RIP-2: Proxy Admin Standardized Management Interface**
— a
dedicated, independent gRPC Admin service on the RocketMQ Proxy, isolated
from the
data-plane `MessagingService`, with a stable backward-compatible proto
contract and
fine-grained ACL 2.0 authorization.
It closes the observability gap introduced by RocketMQ 5.0's stateless Proxy
architecture: gRPC clients attached to a Proxy were previously invisible to
operations tools that rely on broker-side `ConsumerManager` and Remoting-era
admin
commands. RIP-2 gives the control plane a standard server-side interface to
query
online clients, subscriptions, runtime config, connection control,
diagnostics,
route topology, and broker-facing ops — all served from the Proxy itself.
### Motivation
- **RIP-1** (Control Plane 5.0 dashboard, requirement `CLIENT-01`) depends
on a
standard server-side interface to read complete gRPC client data.
- Operators currently cannot answer *"which SDK clients are online, what do
they
subscribe to, are they healthy?"* without indirect metrics heuristics.
- There is no least-privilege authorization model for admin operations on the
Proxy, and no cluster-wide client view.
## What This PR Adds
### 1. Dedicated Admin gRPC Server (D1)
A second gRPC server started by `ProxyStartup`, on its own port
(`adminGrpcPort`, default **8083**), with its own interceptor chain
(metrics → auth → standard pipeline). It reuses the data plane's
`GrpcChannelManager` / `GrpcClientSettingsManager` so online clients are
visible.
The admin server intentionally does NOT expose channelz or proto reflection
(control-plane attack surface kept minimal). A global kill switch
`proxyAdminEnabled` disables the whole surface.
### 2. Two gRPC Services
**`ProxyAdminServiceGrpcService`** (extends `ProxyAdminServiceGrpc`) — the
complete
`ProxyAdminService` surface:
| Milestone | RPCs |
|-----------|------|
| M1 — online client query | `ListClients`, `DescribeClient`,
`ListClientsByGroup`, `ListClientsByTopic` |
| M2 — runtime config & connection control | `DescribeProxyConfig`,
`UpdateProxyConfig`, `KickClient`, `DisconnectChannel` |
| M2 — quota visualization | `DescribeQuota`, `UpdateQuota` |
| M3/M4 — diagnostics | `DescribePopReceiptHandles`,
`DescribeBatchConsumeDiagnostics` |
| Route observation | `SubscribeRouteEvents` (server-streaming),
`DescribeRouteTopology` |
**`ProxyAdminGrpcService`** (extends `AdminGrpc`) — broker-facing operations
served
through the Proxy's own managed broker client (`AdminService` gateway), never
opening a direct link to the broker:
`GetProxyRuntimeStats`, `DescribeTopicStatus`, `QueryMessage`,
`QueryTimeSpan`,
`GetConsumerRunningInfo`, `ListConsumerConnection`, `ListSubscription`,
`DescribeSubscription`, `DescribeGroupAccumulation`, `ResetGroupOffset`,
`DeleteSubscription`, `AdminSendMessage`, `PrintThreadStackTrace`,
`VerifyMessage`, `ChangeLogLevel`, `GetTopicRoute`.
### 3. Dedicated `proxy.admin.*` ACL 2.0 Authorization (D2)
`ProxyAdminAuthInterceptor` maps every RPC to exactly one `(resource,
action)`
pair across six resources:
- `proxy.admin.client` — online client query & diagnostics (`Get`/`List`)
- `proxy.admin.config` — runtime config query & hot update (`Get`/`Update`)
- `proxy.admin.connection` — kick/disconnect, telemetry (`Update`, high
privilege)
- `proxy.admin.quota` — quota query & adjustment (`Get`/`Update`, high
privilege)
- `proxy.admin.route` — route topology & event stream (`Get`/`List`)
- `proxy.admin.ops` — broker-facing ops (`Get`/`List` for queries;
`Update`/`Delete`/`Pub` for mutations)
High-privilege RPCs (`KickClient`, `DisconnectChannel`, `ResetGroupOffset`,
`DeleteSubscription`, `AdminSendMessage`) can **never** be authorized by a
read-only grant. A fail-closed `proxyAdminRequireAuth` mode rejects requests
without verifiable credentials even when cluster-wide auth is off. Every
served
RPC writes a `[PROXY-ADMIN-AUDIT]` log (subject / method / resource / action
/
sourceIp).
### 4. Multi-Proxy Cluster Aggregation (D3)
`ProxyAdminPeerClient` fans `PROXY_SCOPE_ALL_PROXIES` queries out to
configured
peer admin endpoints in parallel, merges per-node local views, and
deduplicates by
`client_id` (local view wins on duplicates). Every result is tagged with
`proxy_endpoint` + monotonic `epoch` for attribution. Peer failures degrade
gracefully — an unreachable peer is skipped with a warning.
### 5. Stable Cursor Pagination (D4)
Client listings use cursor-based `next_token` (base64-opaque, clientId-sorted
position of the last element) so page boundaries stay stable while clients
connect/disconnect between calls. Diagnostic snapshots use offset pagination
(`page_num`/`page_size`, max 100) because they are bounded, point-in-time
views.
### 6. Observability (Acceptance Criteria #4)
`ProxyAdminMetricsManager` / `ProxyAdminMetricsInterceptor` export two
OpenTelemetry instruments honoring the proxy's metrics exporter
configuration:
- `rocketmq_proxy_admin_rpc_total{rpc_method, status, error_type?}` — error
rate
- `rocketmq_proxy_admin_rpc_latency{rpc_method, status}` (ms histogram) — RT
P50/P99
### 7. Route Change Streaming
`RouteChangeNotifier` detects route changes from the proxy's topic route
cache
refreshes and streams them to admin subscribers. Event types:
`ROUTE_SNAPSHOT`
(replayed on subscribe), `TOPIC_CREATE`, `TOPIC_DELETE`, `QUEUE_SCALE`,
`BROKER_ONLINE`, `BROKER_OFFLINE`.
### 8. Protocol-Pure Architecture
`AdminModelConverter` is the **only** class that imports both the broker's
internal wire types (`org.apache.rocketmq.remoting.*`) and the RIP-2 gRPC
protocol (`apache.rocketmq.v2.*`). The gRPC admin services stay protocol-pure
(v2 only); the broker gateway (`DefaultAdminService`) stays remoting-pure.
## Files Changed
**34 files changed, +6,453 / −61 lines** (8 commits over `develop`).
### New Documentation
- `docs/rip-2-proxy-admin.md` — full RIP-2 proposal (motivation, goals,
design decisions, proto contract, observability, configuration, milestones,
acceptance criteria)
- `docs/rip-2-least-privilege.md` — least-privilege configuration guide with
role templates (read-only observer, on-call operator, admin)
### New Source — `proxy/grpc/admin/` (11 files)
| File | Lines | Responsibility |
|------|-------|----------------|
| `ProxyAdminServiceGrpcService.java` | 862 | `ProxyAdminService` surface:
M1 client query, M2 config/connection/quota, M3/M4 diagnostics, route
observation |
| `ProxyAdminGrpcService.java` | 716 | `AdminService` surface: broker-facing
ops via the Proxy's managed client |
| `AdminModelConverter.java` | 332 | Bridge between broker wire types and v2
proto (only class importing both worlds) |
| `ProxyAdminConfigSupport.java` | 344 | Runtime config hot update + quota
registry |
| `ProxyAdminDiagnosticsSupport.java` | 296 | POP receipt-handle &
batch-consume diagnostics from proxy state |
| `RouteChangeNotifier.java` | 332 | Route change detection &
server-streaming notification |
| `ProxyAdminAuthInterceptor.java` | 280 | Per-RPC ACL 2.0 authorization
over `proxy.admin.*` resources |
| `ProxyAdminPeerClient.java` | 264 | D3 cluster-wide fan-out & merge |
| `ProxyAdminMetricsManager.java` | 252 | OpenTelemetry RT & error-rate
metrics |
| `ProxyAdminMetricsInterceptor.java` | 52 | Per-RPC metrics recording |
| `ProxyAdminConfigSupport.java` | 344 | (listed above) |
### New Tests — 7 files, ~1,840 lines
- `AdminModelConverterTest.java`, `ProxyAdminAuthInterceptorTest.java`,
`ProxyAdminConfigSupportTest.java`, `ProxyAdminGrpcServiceTest.java`,
`ProxyAdminServiceGrpcServiceTest.java`, `RouteChangeNotifierTest.java`,
`DefaultAdminServiceTest.java` (enhanced)
### Modified Source
- `ProxyStartup.java` — starts the dedicated admin gRPC server, wires shared
channel/settings managers, peer client, route notifier, metrics
- `ProxyConfig.java` — 6 new config keys (`adminGrpcPort`,
`proxyAdminEnabled`, `proxyAdminRequireAuth`, `proxyAdminPeerEndpoints`,
`proxyAdminPeerTimeoutMillis`, `proxyAdminHeartbeatHistorySize`)
- `AdminService.java` / `DefaultAdminService.java` — broker-facing gateway
methods (offsets, consume stats, reset, delete subscription, query message,
topic config/route)
- `TopicRouteService.java` — route refresh listener hook for
`RouteChangeNotifier`
- `GrpcClientChannel.java` — heartbeat history & auth-status tracking
- `ClientActivity.java` — heartbeat/telemetry hooks feeding `DescribeClient`
- `ReceiptHandleManager.java` / `DefaultReceiptHandleManager.java` —
diagnostic accessors for POP handle inspection
- `GrpcConverter.java`, `GrpcChannelManager.java`,
`GrpcMessagingApplication.java`, `DefaultGrpcMessagingActivity.java`,
`DefaultMessagingProcessor.java`, `ReceiptHandleProcessor.java` —
shared-component exposure
## Configuration
| Key | Default | Meaning |
|-----|---------|---------|
| `proxyAdminEnabled` | `true` | Kill switch; `false` = admin server not
started |
| `adminGrpcPort` | `8083` | Dedicated admin gRPC port (`≤0` disables) |
| `proxyAdminRequireAuth` | `false` | Fail-closed credential enforcement |
| `proxyAdminPeerEndpoints` | `[]` | Peer admin endpoints for ALL_PROXIES
fan-out |
| `proxyAdminPeerTimeoutMillis` | `3000` | Per-peer fan-out timeout (ms) |
| `proxyAdminHeartbeatHistorySize` | `16` | Heartbeat records kept per
client |
## Build Prerequisite
> ⚠️ **The `rocketmq-apis` 2.3.0 artifact is consumed as a local development
> dependency.** The proto contract (`ProxyAdminService` + `AdminService` in
> `admin.proto`) lives in the
[rocketmq-apis](https://github.com/apache/rocketmq-apis)
> repository, branch `feature/rip-2-proxy-admin-grpc`. Before building this
branch,
> install the artifact into your local Maven repository:
>
> ```bash
> cd rocketmq-apis # branch feature/rip-2-proxy-admin-grpc
> mvn clean install
> ```
>
> The `rocketmq-apis` folder is intentionally **not** vendored or submoduled
into
> this repository.
### Known Issue: `rocketmq-proto.version` property
The last commit on this branch (`4945b82c`) reverted
`rocketmq-proto.version` in
the root `pom.xml` back from `2.3.0` to `2.1.2`. The RIP-2 admin code imports
types (`ProxyAdminServiceGrpc`, `AdminGrpc`, etc.) that only exist in the
**2.3.0**
artifact. The pom property should be restored to `2.3.0` before merge, or the
build will fail against the standard Maven Central artifact. This is tracked
for
follow-up.
## How to Test
1. Install the rocketmq-apis 2.3.0 artifact locally (see Build Prerequisite
above).
2. Start a Proxy with `proxyAdminEnabled=true` (default) and
`adminGrpcPort=8083`.
3. Connect gRPC clients to the data-plane port (8081).
4. Call `ListClients` / `DescribeClient` on the admin port (8083) — verify
the
connected clients appear with correct subscriptions, heartbeat history,
and
auth status.
5. Test `PROXY_SCOPE_ALL_PROXIES` with `proxyAdminPeerEndpoints` configured
across
two proxies — verify the merged, deduplicated cluster view.
6. Verify `proxy.admin.*` ACL enforcement: a read-only user can query but not
kick; a high-privilege user can kick / reset offset / delete subscription.
7. Verify metrics: `rocketmq_proxy_admin_rpc_total` and
`rocketmq_proxy_admin_rpc_latency` are exported.
8. Run the unit test suite:
```bash
mvn test -pl proxy -Dtest='org.apache.rocketmq.proxy.grpc.admin.*Test'
```
## Acceptance Criteria Mapping
| Criterion | Status |
|-----------|--------|
| RIP document + stable backward-compatible proto contract |
`docs/rip-2-proxy-admin.md` + rocketmq-apis `admin.proto` |
| Client query RPCs merged; pagination scales with connection churn | D4
stable cursor; page cost O(pageSize) after sort |
| Independent ACL control, read-only/high-risk separation, least-privilege
doc | D2 resources/actions + `docs/rip-2-least-privilege.md` |
| RPC RT & error-rate metrics | `ProxyAdminMetricsManager` instruments |
| E2E with RIP-1 dashboard | Contract frozen for dashboard CLIENT-01
integration (cross-repo) |
## Compatibility
- **Backward compatible**: additive-only proto field evolution, all new
fields
optional, `ProxyScope` defaults to local, no field number reuse.
- **No impact on data plane**: the admin server is a separate gRPC server on
a
separate port; if `proxyAdminEnabled=false` (or `adminGrpcPort≤0`), the
proxy
behaves exactly as before.
- **ACL 2.0**: no changes to the auth core engine — admin resources are
modeled as
cluster-typed literals with reserved names.
## Commits
| Hash | Message |
|------|---------|
| `59607fdb9` | feat(proxy): implement RIP-2 admin gRPC service on the proxy
|
| `5fa8c4bcf` | refactor(proxy): depend on rocketmq-proto 2.3.0 instead of
vendored v2 sources |
| `4c50b9a0b` | feat(proxy): implement RIP-2 ProxyAdminService M1 RPCs |
| `e1e8fbfcd` | fix dedicated admin gRPC server (control plane), isolated
from the data plane |
| `d992dbd3d` | feat(proxy): complete RIP-2 ProxyAdminService surface per
gap audit |
| `d43fbc8e4` | remove submodule |
| `99343ee02` | fix(build): finish removing the rocketmq-apis submodule,
keep proto 2.3.0 |
| `4945b82c4` | Fix signature-algorithm calibration |
## Related
- Issue: [RIP-2] Proxy Admin Standardized Management Interface
(`docs/rip-2-issue.md`)
- RIP-1 Control Plane 5.0 dashboard (requirement `CLIENT-01`)
- Proto contract: [rocketmq-apis](https://github.com/apache/rocketmq-apis),
branch `feature/rip-2-proxy-admin-grpc`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
