xuanskyer opened a new pull request, #191:
URL: https://github.com/apache/rocketmq-exporter/pull/191
## What is the purpose of the change
Closes #190.
All consumer-side metrics are aggregated to broker granularity today, so a
single lagging queue is invisible in Prometheus — `rocketmq_group_diff` only
shows that the whole group is behind, and one has to fall back to the console
to see per-queue offsets. This PR exposes the queue dimension that the
collector already has in hand.
`MetricsCollectTask#collectConsumerOffset` iterates
`consumeStats.getOffsetTable()`, which is a `Map<MessageQueue, OffsetWrapper>`,
and already computes `lagTime` one queue at a time. Both loops then fold the
values into a `HashMap<brokerName, Long>` and discard `queueId`. Publishing the
per-queue values therefore needs **no additional admin call** —
`queryMsgByOffset` is still invoked exactly once per queue, as before.
Note that the label list for this already exists in `RMQMetricsCollector`
(`GROUP_PULL_LATENCY_LABEL_NAMES`, containing `queueid`) but has never been
referenced by any metric.
## Brief changelog
New metrics, all labelled `cluster, broker, topic, group, queueid`:
| metric | meaning |
| --- | --- |
| `rocketmq_queue_group_diff` | per-queue unconsumed messages (`brokerOffset
- consumerOffset`) |
| `rocketmq_queue_group_get_latency_by_storetime` | per-queue consume
latency in ms (reuses the already computed `lagTime`) |
| `rocketmq_queue_consumer_offset` | per-queue consumer offset |
Queue-level series multiply the consumer series count by the number of
queues per broker, so they are **disabled by default** and gated by an explicit
topic whitelist:
```yaml
rocketmq:
config:
queueLevelTopics: "" # empty = disabled (default); "topic-a,topic-b" =
whitelist; "*" = all topics
```
Changes:
* `RMQConfigure` — new `queueLevelTopics` property, parsed once in the
setter into a `Set`, plus `isQueueLevelTopic(String)`.
* `ConsumerQueueMetric` (new) — metric key, i.e. `ConsumerMetric` plus
`queueId`.
* `RMQMetricsCollector` — three caches reusing the existing
`outOfTimeSeconds` expiry, `collectQueueGroupNums()` and three `addXxxMetric()`
methods, following the existing `groupDiff` pattern.
* `MetricsCollectTask` — publishes the per-queue values inside the two
existing loops, guarded by the whitelist. The broker-level aggregation is left
exactly as is, so existing dashboards and alerts are unaffected.
* `application.yml` — documents the new option.
The change is purely additive: no existing line of behaviour was modified.
The second commit fixes a pre-existing checkstyle violation (`if(` missing a
space, `MetricsCollectTask.java`) that currently makes `mvn clean install` fail
on master regardless of this PR — without it the build command below cannot
pass. Happy to split it out if you prefer.
## Verifying this change
* `mvn -B clean install -DskipITs` — **BUILD SUCCESS**, `Tests run: 5,
Failures: 0, Errors: 0`
* `mvn -B clean apache-rat:check checkstyle:checkstyle` — RAT summary
`Unapproved: 0, unknown: 0, approved: 39`; checkstyle clean after the second
commit
* New unit test `RMQConfigureQueueLevelTest` covers the whitelist parsing:
disabled by default, blank stays disabled, comma list is trimmed and matched
**exactly** (`topic-a` must not enable `topic-a-extra`, otherwise the series
budget silently blows up), `*` wildcard, and resetting back to disabled. The
test was also verified in reverse — replacing the exact match with `startsWith`
makes it fail as expected.
Note that `findbugs:findbugs` from the checklist was not run: the plugin
does not support the JDK used locally (17). `spotbugs` would be the modern
replacement, but that seemed out of scope for this PR.
One thing worth mentioning for the maintainers: the existing test sources
are JUnit 4, while surefire 3.2.2 auto-selects the JUnit Platform provider, so
`mvn test` currently reports `Tests run: 0` and no existing test actually
executes. The new test is therefore written against JUnit 5 so that it really
runs. Adding `junit-vintage-engine` would revive the existing ones, but again
felt out of scope here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
