On 29 Feb 15:38, Nur Kholis Majid wrote: > Hi, > > I've test prometheus to monitoring node_exporter on 400 instances. With > default configuration, in just two months tsdb size reach +- 450GB and > memory size +- 135GB. Query become slow and unuseable. > > [image: photo_2020-03-01_06-33-51.jpg] > > [image: photo_2020-03-01_06-34-00.jpg]
Can we know what you mean by default configuration? Is it default or documented one? What are your startup parameters? How many series do you have? max_over_time(prometheus_tsdb_head_series[1d]) Do you have lots of different disks/devices per machines ? lots of network interfaces? I recommend you read https://www.robustperception.io/how-much-ram-does-prometheus-2-x-need-for-cardinality-and-ingestion to better understand this. > > > Question: > 1. How many maximum node_exporter instances can handle by prometheus with > acceptable query duration? > 2. Is there any special prometheus configuration for huge amount of > instances? > > Thank you > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/7da6b213-02d0-4beb-83fb-e943701b2422%40googlegroups.com. -- (o- Julien Pivotto //\ Open-Source Consultant V_/_ Inuits - https://www.inuits.eu -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/20200229234428.GA3785%40oxygen.
signature.asc
Description: PGP signature
