zmuxuny opened a new issue, #5026:
URL: https://github.com/apache/rocketmq-dashboard/issues/5026
## Problem
A temporary failure to read one consumer group's queue progress can falsely
resolve an active per-topic backlog alert. The dashboard may then show a
recovery and enqueue a recovery notification even though the backlog was not
measured.
## Reproduction
1. Configure a native `topic.backlog.total > 10` rule for instance `local`,
consumer group `orders`, and topic `orders-topic`. Let a sample labeled
`{consumerGroup=orders, topic=orders-topic}` enter `FIRING`.
2. On the next collection, make `InstanceProvider.getGroupProgress("local",
"orders")` fail while consumer-group discovery succeeds.
3. `ApacheRocketMqBusinessMetricsCollector` emits an `UNAVAILABLE`
`topic.backlog.total` sample labeled only `{consumerGroup=orders}`, because it
cannot enumerate topics.
4. `NativeAlertProcessor.processSuccessfulCollection` does not classify that
nonempty-label sample as a whole-scope failure. The previous per-topic
fingerprint is absent from `presentKeys`, so reconciliation advances it to
`RESOLVED`.
A focused regression test against the current `rocketmq-studio` branch
reproduces the incorrect state write:
`NativeAlertProcessorTest.doesNotResolveTopicAlertWhenItsConsumerGroupProgressIsUnavailableTest`
fails because `AlertStateRepository.save` is invoked with `status=RESOLVED`.
## Expected behavior
An unavailable sample for a group should preserve active fingerprints for
that metric whose labels fall under the failed group. Other groups should still
reconcile normally. Once the group's progress is collected successfully, a
genuinely disappeared topic may resolve.
## Scope and related work
The fix belongs in native alert reconciliation: account for a non-available
sample whose labels are broader than a previously active fingerprint. Keep
metric and instance scoping so unrelated failures do not block recovery.
This is distinct from the whole-scope failure guard introduced for #2679 and
the missing Broker metric samples fixed in #4519: the collector already emits a
failure sample, but it has a narrower scope than the entire collection and
fewer labels than the active per-topic alert.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
