Thanks for the fast reply and sorry for my late response.
We have a scraping on 15 seconds. My query interval is 30 min. So I should
have data. And the Grafana dashboard below where I add difference(changed
from delta) on legend values I see what I want.
This difference from the Grafana panel is exactely what I am looking for
and thought delta would give me.
Is there an alternative to delta? Like first-last?
On Saturday, July 1, 2023 at 3:07:22 PM UTC+2 Ben Kochie wrote:
> Grafana defaults to a "minimum step" of 15 seconds. I bet your scrape
> interval is longer than 15 seconds, so "$__interval" is matching no data.
>
> You can either adjust the datasource minimum step to your shortest scrape
> interval, or set the "min step" in the query options to match the scrape
> interval of your metric.
>
> On Sat, Jul 1, 2023 at 3:04 PM Helena Lundell <hel...@gmail.com> wrote:
>
>> I am trying to identify an alert for an increasing Kafka Lag. The metrics
>> are in prometheus and I use Grafana 9.4 for visualisation.
>>
>> In Grafana I have a dasboard where I can see that the lag is increasing
>> or decreasing. I have choosen to display the delta in the legend and it
>> shows a number that does not really make sense, but it shows something.
>>
>> This is the base promql:
>> avg by(consumergroup, topic, cluster)
>> (kafka_consumergroup_lag{namespace="ns-kafka-int",
>> consumergroup=~".*$container", cluster="$cluster"})
>>
>> When I add delta to this query I get no data:
>> avg by(consumergroup, topic, cluster)
>> (delta(kafka_consumergroup_lag{namespace="ns-kafka-int",
>> consumergroup=~".*$container", cluster="$cluster"}[$__interval]))
>>
>> So adding delta gives me no result even though when I exclude delta I can
>> see that the metric has been changing.
>>
>> The metric is a gaugue.
>>
>> Question:
>> Am I doing something wrong?
>> Can I trust the delta from the Grafana dashboard(someone might know)?
>> Can I expect a positive or negative value to identify and increase?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-use...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/prometheus-users/0f37aa72-dd90-4329-8e19-5587460b80c4n%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/63cf733e-87bd-4530-9531-5a3a29556d8an%40googlegroups.com.
