Probably not. Prometheus will write scraped data to the WAL and if that fails it won't be stored anywhere. Equally if a file is only partially written (either a full block or something in the WAL) you may end up losing all the data in that file (Prometheus will check for corruption on startup).
On 20 February 2020 08:25:14 GMT, adi garg <[email protected]> wrote: >Thanks Stuart. Is it possible to restart the prometheus without data >loss >from that point? > >On Thursday, February 20, 2020 at 1:39:03 PM UTC+5:30, Stuart Clark >wrote: >> >> If you are meaning the disk storage, Prometheus would stop working >and you >> might also encounter some corruption. >> >> You can control disk usage by setting the retention period or by >setting >> the maximum space to remain available. >> >> I'd strongly suggest putting the storage on a separate mount and also > >> using the node exporter with alerts to ensure you don't run out of >space. >> >> On 20 February 2020 05:18:03 GMT, adi garg <[email protected] >> <javascript:>> wrote: >>> >>> Thanks Julius and Julien. Awesome answers. This is related to RAM, >but >>> what will happen if the secondary storage is not sufficient to take >the >>> metrics. What will happen in that case? >>> >>> On Wednesday, February 19, 2020 at 10:32:43 PM UTC+5:30, Julius Volz > >>> wrote: >>>> >>>> While an explicit memory limit is not configurable, there are a >number >>>> of knobs in Prometheus that one can configure that limit resource >usage >>>> along certain dimensions, for example >>>> https://www.robustperception.io/limiting-promql-resource-usage. >>>> >>>> There's also a setting that prevents a maximum number of samples >>>> ingested per scrape. >>>> >>>> On Wed, Feb 19, 2020 at 5:34 PM Julien Pivotto ><[email protected]> >>>> wrote: >>>> >>>>> On 19 Feb 08:24, adi garg wrote: >>>>> > So what will happen if prometheus crosses the RAM limit, will it >die? >>>>> Or is >>>>> > it gonna affect the other processes running on the system? >>>>> >>>>> >>>>> It is the operating system that will chose. Prometheus will >probably be >>>>> terminated by the operating system. >>>>> >>>>> > >>>>> > On Wednesday, February 19, 2020 at 9:41:36 PM UTC+5:30, Julien >>>>> Pivotto >>>>> > wrote: >>>>> > > >>>>> > > On 19 Feb 08:08, adi garg wrote: >>>>> > > > Hello experts, >>>>> > > > >>>>> > > > Is there any way to give memory specifications by ourselves >in >>>>> > > prometheus? >>>>> > > >>>>> > > Hello, >>>>> > > >>>>> > > No, this is not possible. Prometheus will use the memory it >needs. >>>>> > > >>>>> > > Regards, >>>>> > > >>>>> > > > >>>>> > > > -- >>>>> > > > You received this message because you are subscribed to the >>>>> Google >>>>> > > Groups "Prometheus Developers" group. >>>>> > > > To unsubscribe from this group and stop receiving emails >from it, >>>>> send >>>>> > > an email to [email protected] > >>>>> > > <javascript:>. >>>>> > > > To view this discussion on the web visit >>>>> > > >>>>> >https://groups.google.com/d/msgid/prometheus-developers/0fb6ec18-c2a8-437f-9d2b-5cc2b9f5b330%40googlegroups.com. > >>>>> >>>>> > > >>>>> > > >>>>> > > >>>>> > > -- >>>>> > > (o- Julien Pivotto >>>>> > > //\ Open-Source Consultant >>>>> > > V_/_ Inuits - https://www.inuits.eu >>>>> > > >>>>> > >>>>> > -- >>>>> > You received this message because you are subscribed to the >Google >>>>> Groups "Prometheus Developers" group. >>>>> > To unsubscribe from this group and stop receiving emails from >it, >>>>> send an email to >[email protected]. >>>>> > To view this discussion on the web visit >>>>> >https://groups.google.com/d/msgid/prometheus-developers/df7bdc80-b95f-4f02-88d1-664d3cf3c651%40googlegroups.com >>>>> . >>>>> >>>>> >>>>> -- >>>>> (o- Julien Pivotto >>>>> //\ Open-Source Consultant >>>>> V_/_ Inuits - https://www.inuits.eu >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google > >>>>> Groups "Prometheus Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, >send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> >https://groups.google.com/d/msgid/prometheus-developers/20200219163423.GA6819%40oxygen >>>>> . >>>>> >>>> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. >> > >-- >You received this message because you are subscribed to the Google >Groups "Prometheus Developers" group. >To unsubscribe from this group and stop receiving emails from it, send >an email to [email protected]. >To view this discussion on the web visit >https://groups.google.com/d/msgid/prometheus-developers/0b15e598-42ef-4d21-b080-4448a955d463%40googlegroups.com. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/45E04138-BFC5-4758-95AF-ABFAFCFDEAB9%40Jahingo.com.
