With this approach, how do the users know the truth? Why did Prometheus invoke query look-back? Is it due to Prometheus Target unavailability/unreachability or Prometheus unavailability?
On Thursday, January 20, 2022 at 12:45:46 PM UTC+1 [email protected] wrote: > Thanks for the explanation, I thought staleness is applicable only to > Prometheus Targets, haven't imagined this concept to Prometheus restarts > and unavailability. So, you say 'statelessness' is also applied to > Prometheus availability. > > Is there a way to know if a metric or series is marked stale through the > general PromQL curl command and also through the expression browser? > > After reading a few Prometheus docs and Q&A, is it still a good practice > to keep scraping intervals around 2 minutes and not change the default > query look-back delta flag? > On Wednesday, January 19, 2022 at 9:57:40 AM UTC+1 Brian Candler wrote: > >> I believe the following is true as well: >> >> - if prometheus does a scrape and a metric is missing (in the exporter >> output) which was present in a previous scrape, it's immediately marked as >> "stale", i.e. a staleness marker is inserted into the timeseries. >> >> - however in your case, you're turning off the prometheus server, so >> there's no scraping taking place. You just get a point in the timeseries >> at the time of the last scrape before prometheus shut down, and then at the >> first scrape after prometheus starts up. There is no indication within the >> timeseries data itself that anything is missing; therefore, PromQL queries >> will look back up to 5 minutes. >> >> On Wednesday, 19 January 2022 at 07:59:57 UTC Brian Candler wrote: >> >>> Yes. Google "Prometheus Staleness". >>> >>> In short: the value of a metric at time T is the most recent value >>> recorded *on or before time T*, and prometheus will look backwards in >>> time up to --query.lookback-delta (default 5 minutes) to find this value. >>> >>> On Tuesday, 18 January 2022 at 23:54:07 UTC [email protected] wrote: >>> >>>> Hi, >>>> >>>> I just run Prometheus (2.32.0-beta.0) without changing any CLI flags. >>>> If I shut it down for <5 Mins, I don't see a gap in my graphs. If I shut >>>> it >>>> down for > 5Mins, I see a gap. Is this valid? >>>> >>>> Thanks, >>>> Teja >>>> [image: ppk.PNG] >>> >>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/ecb94345-b5af-40cf-b4f5-f2f90c170cc3n%40googlegroups.com.
