I am trying to identify an alert for an increasing Kafka Lag. The metrics
are in prometheus and I use Grafana 9.4 for visualisation.
In Grafana I have a dasboard where I can see that the lag is increasing or
decreasing. I have choosen to display the delta in the legend and it shows
a number that does not really make sense, but it shows something.
This is the base promql:
avg by(consumergroup, topic, cluster)
(kafka_consumergroup_lag{namespace="ns-kafka-int",
consumergroup=~".*$container", cluster="$cluster"})
When I add delta to this query I get no data:
avg by(consumergroup, topic, cluster)
(delta(kafka_consumergroup_lag{namespace="ns-kafka-int",
consumergroup=~".*$container", cluster="$cluster"}[$__interval]))
So adding delta gives me no result even though when I exclude delta I can
see that the metric has been changing.
The metric is a gaugue.
Question:
Am I doing something wrong?
Can I trust the delta from the Grafana dashboard(someone might know)?
Can I expect a positive or negative value to identify and increase?
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com.
