Kris20030907 opened a new issue, #431:
URL: https://github.com/apache/rocketmq-dashboard/issues/431
## Background
RocketMQ Studio currently contains the initial metrics and dashboard
scaffolding, but it does not yet provide real observability data:
- `PrometheusMetricsSource` generates random samples instead of querying
Prometheus.
- The current metrics response represents only one `List<long[]>`, which
loses Prometheus series labels and decimal precision.
- The dashboard is backed by a separate stub provider.
- Data source connection testing currently returns a fixed success result.
This issue proposes integrating RocketMQ Studio with the Prometheus HTTP API
and gradually replacing the mock observability path with real RocketMQ metrics.
Related community initiative: apache/rocketmq#10600.
This work complements the Prometheus alert rules work in #425. Alert rule
provisioning and Alertmanager integration are not included in this proposal.
## Proposed architecture
```text
RocketMQ 4.x
-> rocketmq-exporter
-> Prometheus
-> RocketMQ Studio
RocketMQ 5.1+
-> native Prometheus metrics endpoint
-> Prometheus
-> RocketMQ Studio
```
RocketMQ Studio will query Prometheus through:
- `/api/v1/query`
- `/api/v1/query_range`
Studio will not scrape Broker or Exporter endpoints directly.
## Goals
1. Add a real Prometheus HTTP query adapter.
2. Preserve multiple time series, labels, decimal sample values, result
types, and warnings.
3. Add query validation, connection/query timeouts, and explicit Prometheus
error mapping.
4. Support optional HTTP Basic and Bearer token authentication without
exposing credentials in API responses or logs.
5. Provide version-aware semantic metric mappings for:
- RocketMQ 4.x with `rocketmq-exporter`
- RocketMQ 5.1+ native metrics
6. Replace the dashboard mock provider with real Prometheus-backed data.
7. Provide loading, empty, partial failure, stale data, and refresh states
in the frontend.
## Compatibility and validation targets
The initial compatibility targets are:
- RocketMQ 4.9.4 with `rocketmq-exporter`
- RocketMQ 5.5.0 native Prometheus metrics
- Prometheus HTTP API v1
A read-only validation against an existing RocketMQ 5.5.0 Prometheus data
source has already confirmed:
- multi-series matrix responses;
- labels such as `cluster`, `node_type`, and `node_id`;
- decimal rate values;
- message, throughput, consumer lag, connection, topic/group count, and RPC
latency metrics.
A self-contained Podman environment will also be used for controlled
end-to-end validation.
## Proposed delivery plan
To follow the contribution requirements in apache/rocketmq#10600,
implementation will be split into atomic PRs, with approximately 5–10 changed
files per PR.
### PR 1: Prometheus query adapter
- Prometheus HTTP client and configuration
- correct multi-series response model
- query validation
- timeout and error handling
- unit tests using a mock Prometheus server
### PR 2: Data source integration
- connect the Settings data source APIs to the Prometheus adapter
- implement real connection testing
- support default/selected data sources
- align frontend and backend data source contracts
### PR 3: RocketMQ metric profiles
- RocketMQ 4.x exporter metric mappings
- RocketMQ 5.1+ native metric mappings
- unified semantic metrics for TPS, throughput, lag, latency, and health
### PR 4: Real dashboard
- remove the dashboard stub provider
- add time range and refresh controls
- display real cluster and Broker metrics
- handle loading, empty, error, partial, and stale states
Each PR will be independently testable and revertible.
## Non-goals
The following are intentionally excluded from this issue:
- deploying or managing users' Prometheus infrastructure;
- embedding Grafana dashboards;
- managing Prometheus alert rules;
- Alertmanager integration;
- implementing every observability page in one PR;
- silently falling back to randomly generated data.
## Acceptance criteria
- No random metrics are returned when Prometheus is unavailable.
- Range queries preserve all returned series and labels.
- Decimal values are not truncated.
- Prometheus 4xx/5xx responses, timeouts, and malformed responses produce
explicit errors.
- Authentication secrets are not logged or returned to the frontend.
- RocketMQ 4.9.4 exporter and RocketMQ 5.5.0 native metrics are covered by
end-to-end validation.
- Each implementation PR contains the relevant unit or integration tests.
## Questions for the community
1. Should a Prometheus data source be bound to each RocketMQ cluster, or
should Studio initially use one global default data source?
2. Should the low-level PromQL query API remain an internal backend
abstraction, or be exposed for future custom dashboards?
3. Is the proposed split into four atomic PRs aligned with the expected
Track 1 contribution scope?
I would like to work on this issue and submit the implementation as atomic
PRs to the `rocketmq-studio` branch after the scope is agreed upon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
