> I understand that there can be some interpolation at the boundaries, but
> the value is not changing around the boundaries, it only changes in the
> middle of the time range. Scrap is done every 15s and the value of the
> metric is constant more than 1 minute before and after the boundaries. I
> deliberately chose a range with constant values around the boundaries to
> prevent any mis-interpolation.
>
> I'm using promtool to check values, here are my results:
[...]
> promtool query range --start "$(date -d'2024-01-18 14:00:00 UTC' +%s)"
> --end "$(date -d'2024-01-18 14:15:00 UTC' +%s)" $url 'metric'
> metric 9732212 @[1705586400]
> ...
> metric 9848219 @[1705587300]
> --> it returns 302 samples
>
> promtool query instant --time "$(date -d'2024-01-18 14:15:00 UTC' +%s)"
> $url 'metric[15m]'
> 9732212 @[1705586407.092]
> 9732212 @[1705586422.092]
> 9732212 @[1705586437.092]
> 9732212 @[1705586452.092]
> 9732212 @[1705586467.092]
> 9732212 @[1705586482.092]
> 9732212 @[1705586497.092]
> ...
> 9848219 @[1705587142.092]
> 9848219 @[1705587157.092]
> 9848219 @[1705587172.092]
> 9848219 @[1705587187.092]
> 9848219 @[1705587202.092]
> 9848219 @[1705587217.092]
> 9848219 @[1705587232.092]
> 9848219 @[1705587247.092]
> 9848219 @[1705587262.092]
> 9848219 @[1705587277.092]
> 9848219 @[1705587292.092]
> --> it returns 61 samples
> The timestamps are a bit different but values are right. But it returns way
> less samples than the range query. I would expect the same number of
> samples returned by the 2 queries.
This is a (natural) misunderstanding of what range queries do. A range
query evaluates your query term at every step through the range from
start to end, and returns a list of those results, each one with the
timestamp it was evaluated at. If you don't provide a step, Prometheus
works out a default one based on the time range, and experimentally the
default step for a 15 minute time range is 3 seconds (you can see this
in the Prometheus web interface), which gives about the right number of
answers as you get in your range query. A range query explicitly does
not restrict itself to the number of time series points that are
actually in the time range; it will freely re-use the same points across
multiple instant queries within the range.
(This is commonly visible if you use rate() with a time range larger
than the step size. If you query for 'rate(metric[2m])' for a range
query with a step of 3s, you will get a lot of duplicate results.)
The instant query for a time range gives a range vector as the result,
which contains the true set of time series points with their true
timestamps. Since you're querying a fifteen minute time range for a
metric scraped every 15 seconds, 61 samples is about right for what
you'd expect as the time series points scraped over that amount of time.
We can also see here that the first point was collected at 14:00:07 UTC
and the last one at 14:14:52. This means that a delta() of this same
range will extrapolate out to cover an additional 15 seconds (covering 7
missing seconds from the start and 8 missing seconds from the end).
> now let's compute the delta:
> promtool query instant --time "$(date -d'2024-01-18 14:15:00 UTC' +%s)"
> $url 'metric - metric offset 15m'
> {} => 116007 @[1705587300]
> --> this matches to 9848219 - 9732212
>
> promtool query instant --time "$(date -d'2024-01-18 14:15:00 UTC' +%s)"
> $url 'delta(metric [15m])'
> {} => 117973.22033898304 @[1705587300]
> --> this does not match
This delta() result is actually very close to the result that 'bc -l'
gives me for a manual extrapolation of those extra 15 seconds. The true
difference between the first and last points within the range vector is
116007, the entire range vector covers 15 minutes less fifteen seconds,
or 885 seconds, and we're extrapolating it to 15 minutes:
$ bc -l
(15*60) / 885
1.01694915254237288135
(( 15 * 60 ) / 885 ) * 116007
117973.22033898305084676945
It feels weird that a total of a fifteen second gap at the start and end
of the range can have such a big effect, but as we can see the extra
time is not trivial at the level of this calculation. If the absolute
numbers are smaller the absolute difference between them will also be
smaller, but the relative difference would always be the same.
- cks
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/330135.1705683201%40apps0.cs.toronto.edu.
