I am struggling with PromQL over an issue dealing with a metric that changes less frequently than the scrape interval. I am trying to use Prometheus as a pseudo event tracker and hoping to get some advice on how to best try and accomplish my goal.
I have a random job that runs at different intervals depending on the situation. Some instances of the job run every five minutes and some run only once an hour or once a day. The job creats a node_exporter textfile snippet that gets scraped on 30 second interval. Below is an example of a metric that changes only every five minutes with the lesser scrape interval. In this scenario all the points with same value are from the same job run. I really only care about one of those. I have no way to know what the interval is between sets for all my different jobs. All I know is that when the value changes, a new set is in play. What I want to do in "reduce" my dataset to deal with only distinct values. I want to collapse these 27 entries below into 3 by taking either the first or last value of each "set". I can not find a PromQL function/operator that does what I want. Maybe I need to use recording rules? All and any help is greatly appreciated. *metric_name{instance="hostname.example.net", job="external/generic", mode="pull", name="snafu"}9973997301 <http://voice.google.com/calls?a=nc,%2B19973997301>@1673997343.7749973997301 @1673997373.7649973997301 @1673997403.7649973997301 @1673997433.7649973997301 @1673997463.7649973997301 @1673997493.7649973997301 @1673997523.7649973997301 @1673997553.7649973997301 @1673997583.7649973997601 @1673997613.7649973997601 @1673997643.7649973997601 @1673997673.7649973997601 @1673997703.7749973997601 @1673997733.7649973997601 @1673997763.7649973997601 @1673997793.7649973997601 @1673997823.7649973997601 @1673997853.8639973997901 @1673997913.7649973997901 @1673997943.7679973997901 @1673997973.7649973997901 @1673998003.7649973997901 @1673998033.7649973997901 @1673998063.7649973997901 @1673998093.7649973997901 @1673998123.7649973997901 @1673998153.764* -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/99bec6df-8c64-4cb2-95e7-f7673418ce25n%40googlegroups.com.