Grafana defaults to a "minimum step" of 15 seconds. I bet your scrape
interval is longer than 15 seconds, so "$__interval" is matching no data.
You can either adjust the datasource minimum step to your shortest scrape
interval, or set the "min step" in the query options to match the scrape
interval of your metric.
On Sat, Jul 1, 2023 at 3:04 PM Helena Lundell <helu...@gmail.com> wrote:
> I am trying to identify an alert for an increasing Kafka Lag. The metrics
> are in prometheus and I use Grafana 9.4 for visualisation.
>
> In Grafana I have a dasboard where I can see that the lag is increasing or
> decreasing. I have choosen to display the delta in the legend and it shows
> a number that does not really make sense, but it shows something.
>
> This is the base promql:
> avg by(consumergroup, topic, cluster)
> (kafka_consumergroup_lag{namespace="ns-kafka-int",
> consumergroup=~".*$container", cluster="$cluster"})
>
> When I add delta to this query I get no data:
> avg by(consumergroup, topic, cluster)
> (delta(kafka_consumergroup_lag{namespace="ns-kafka-int",
> consumergroup=~".*$container", cluster="$cluster"}[$__interval]))
>
> So adding delta gives me no result even though when I exclude delta I can
> see that the metric has been changing.
>
> The metric is a gaugue.
>
> Question:
> Am I doing something wrong?
> Can I trust the delta from the Grafana dashboard(someone might know)?
> Can I expect a positive or negative value to identify and increase?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com
> <https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/CABbyFmo%2BbDyXHOB-AkBN%3D25j3nKXTxSJyHCkcCaPbSFZU_0HKA%40mail.gmail.com.
