Again, it depends, I don't know what you want to collect and store. It
could be enough, it could not be enough. Only you can capacity plan this
with the knowledge of your internal requirements.
On Tue, Feb 20, 2024 at 8:09 PM Puneet Singh <singh.punit1...@gmail.com>
wrote:
> Hi Ben,
> Thank you for the response. So for scraping and storing/writing the data
> for this setup, prometheus would need ~30G memory .
>
> i have a query on 2 Data Query scenarios-
> scenario1: We run a query to get 3 years of load via grafana as -
> node_load15{instance="$node",job="$job"}[$__rate_interval]
> scenario2: we decide to query the system load worth 3 years for a single
> server from the grafana, which uses following query:
> avg_over_time(node_load15{instance="$node",job="$job"}[$__rate_interval])
> * 100 / on(instance) group_left sum by
> (instance)(irate(node_cpu_seconds_total{instance="$node",job="$job"}[$__rate_interval]))
>
> this involves 2 series, plus 3 functions (avg_over_time,sum by , irate)
> and a division operation
>
> Is there a way to get a rough estimate of the amount of CPU and RAM
> required for the queries mentioned in scenario1 and scenario2?
>
Again, it depends on how exactly many series and samples you need to load.
Figure maybe 20MiB of memory per series with 15s scrape intervals.
That query is a bit nonsensical. Load average is not really a useful metric
to look at, and I don't understand why you're dividing it by CPU seconds.
For things that you want to graph over long periods of time, you can use
recording rules to generate pre-computed data that is easier to query.
https://prometheus.io/docs/prometheus/latest/configuration/recording_rules/
>
> Regards,
> Puneet
>
> On Tuesday 20 February 2024 at 23:17:19 UTC+5:30 Ben Kochie wrote:
>
>> Prometheus needs a minimum of about 4KiB per "active series". Retention
>> policy doesn't affect the memory usage very much.
>>
>> If you have 10,000 metrics per server and 300 servers, that's 3 million
>> series.
>>
>> 3 million * 4KiB = 11.4GiB of memory.
>>
>> Of course, you will also need some page cache and such. Usually 2x is
>> more than good enough, but it depends on your query load. So 20-30GiB of
>> memory should be enough.
>>
>> But of course, it highly depends on how many metrics per server you have.
>>
>> On Tue, Feb 20, 2024 at 6:24 PM Puneet Singh <singh.p...@gmail.com>
>> wrote:
>>
>>> HI All,
>>> I am planning to store 3 years of data from 300 server in a single
>>> prometheus server. The data will primarily consist of default exporter
>>> metrics and the server has 500G memory and 80 cores.
>>>
>>> I'd like to ensure that my solution is optimal in terms of resource
>>> utilization, query performance and scalability. Is there a general
>>> recommendation about the amount of resources i should be having on this
>>> setup?
>>>
>>> Regards,
>>> Puneet
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Prometheus Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to prometheus-use...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/prometheus-users/2735babe-1905-45e0-90e7-2316c2f35f8bn%40googlegroups.com
>>> <https://groups.google.com/d/msgid/prometheus-users/2735babe-1905-45e0-90e7-2316c2f35f8bn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-users/31c10035-a36d-473b-95cb-016cd5060dd3n%40googlegroups.com
> <https://groups.google.com/d/msgid/prometheus-users/31c10035-a36d-473b-95cb-016cd5060dd3n%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to prometheus-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/CABbyFmo%3DZ9H_w%3DwS5gkXGDxbhsfaefB3f3DCOYL8fo4rM4kHiA%40mail.gmail.com.
