This is a classic use case for Federation.
You have a high frequency scrape server that keeps the raw data for some
amount of time. I would probably keep 1 month around, say 35 days of raw
data. It depends a bit on how much you're scraping.
Then for long-term history, you have a set of recording rules like this:
groups:
- name: network bandwidth
interval: 1m
rules:
- record: instance:IfInOctets:rate1m
expr: rate(IfInOctets[1m])
- record: instance:IfOutOctets:rate1m
expr: rate(IfOutOctets[1m])
This will give you 1 minute averaged data. The Federation server can then
scrape only these rules and store them for much longer periods of time.
There's no need to adjust --storage.tsdb.min-block-duration, the latest
Prometheus release (2.19.0) now writes out head chunk data via mmap,
eliminating the memory penalty for higher frequency scrapes.
All of that said, you might consider looking at Thanos. It provides tiered
downsampling and resolution retention. This allows you to keep raw data for
up to some amount of time, but it will transparently provide downsample
data for much longer time ranges. Providing the best of both worlds, fast
raw scrapes, and forever retention of raw counter data.
On Thu, Jun 18, 2020 at 12:10 AM Виталий Ковалев <[email protected]>
wrote:
> Hello. I use prometheus + snmp_exporter to monitor network swithes(D-Link
> and Huawei) with snmp.
> I have an issue with huawei switches - some of them can't give ports
> utilization and i have to calculate it manualy.
> As i know prometheus can't setup per job data retention(Is it rigth?), and
> i decided to start another one prometheus server which will collect
> IfInOctets/IfOutOctets every 5 seconds, and calculate utilization, when
> first server will scrape second server.
> I dont need to keep IfInOctets/IfOutOctets too long,and i decided setup
> storage.tsdb.min-block-duration to 30 m and storage.tsdb.retention.time to
> 1d.
> So, the questions are:
> Is it right way?
> What difference between storage.tsdb.min-block-duration and
> storage.tsdb.max-block-duration?
> Will prometheus calculate utilization for metrics which still in memory?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/08d2ca92-8f2a-4bcc-a98a-3e6d4530e3d4o%40googlegroups.com
> <https://groups.google.com/d/msgid/prometheus-users/08d2ca92-8f2a-4bcc-a98a-3e6d4530e3d4o%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/CABbyFmpYgKLjWh%2BFm%3Dv1rfpgkoeKFk9iegucRw338XXcRjjkFA%40mail.gmail.com.
