This is mostly unnecessary in Prometheus because it uses compression in the TSDB samples. What would take up a lot of space in an RRD file takes up very little space in Prometheus.
A basic nearline 20TB HDD can easily store 600,000 series for 10 years at full 15s resolution. This is possible because the average sample point size in Prometheus is about 1.5 bytes per sample. So 1.5 bytes * 5760 samples/day * 365 days * 10 years =~ 30MiB. So for your example, looking up the data for a single metric over a long period of time is still pretty cheap. What's actually more difficult is doing all the index loads for this long period of time. But Prometheus uses mmap to opportunistically access the data on disk. On Tue, Feb 21, 2023 at 4:29 AM Christoph Anton Mitterer <cales...@gmail.com> wrote: > Hey. > > I wondered whether one can to with Prometheus something similar that is > possible with systems using RRD (e.g. Ganlia). > > Depending on the kind of metrics, like for those from the node exporter, > one may want a very high sample resolution (and thus short scraping > interval) for like the last 2 days,... but the further one goes back the > less interesting those data becomes, at least in that resolution (ever > looked a how much IO a server had 2 years ago per 15s)? > > What one may however want is a rough overview of these metrics for those > time periods longer ago, e.g. in order to see some trends. > > > For other values, e.g. the total used disk space on a shared filesystem or > maybe a tape library, one may not need such high resolution for the last 2 > days, but therefore want the data (with low sample resolution, e.g. 1 > sample per day) going back much longer, like the last 10 years. > > > With Ganglia/RRD it one would then simply use multiple RRDs, each for > different time spans and with different resolutions... and RRD would > interpolate it's samples accordingly. > > > Can anything like this be done with Prometheus? Or is that completely out > of scope? > > > I saw that one can set the retention period, but that seems to affect > everything. > > So even if I have e.g. my low resolution tape library total size, which I > could scrape only every hour or so, ... it wouldn't really help me. > In order to keep data for that like the last 10 years, I'd need to set the > retention time to that. > > But then the high resolution samples like from the node exporter would > also be kept that long (with full resolution). > > > Thanks, > Chris. > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to prometheus-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/36e3506c-1fba-48e4-b3d9-ead908767cf2n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/36e3506c-1fba-48e4-b3d9-ead908767cf2n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmqSw%3DQH-0Lx7GuXAUQawKX8omgyKGC6FPG3%2BcC3hW%3DyKA%40mail.gmail.com.
