Thanks for the quick & detailed response ! On Tuesday, 2 February 2021 at 18:09:07 UTC+2 [email protected] wrote:
> There have been some small improvements to the WAL, reducing the amount > needed for startup. But you will have to upgrade to the latest release. > This will not help the current situation you have, as the improvements come > after upgrade. > > Either way, there is almost never any choice about allocating more memory > when you're hitting an OOM. Prometheus always uses the minimum amount of > memory it can for operating. If it needs more memory, you need to decrease > your metric cardinality or increase the allocated memory. There's no magic > fix. > > On Tue, Feb 2, 2021 at 3:44 PM hamisa <[email protected]> wrote: > >> thanks for the quick reply. >> >> is there any way for me to disable it, or at least reduce the amount of >> memory it consumes? >> or for a change reduce the data capacity that being loaded from the wal? >> >> thanks in advance. >> On Tuesday, 2 February 2021 at 16:25:06 UTC+2 [email protected] wrote: >> >>> You will need to give the instance more memory in order to complete the >>> recovery. >>> >>> On Tue, Feb 2, 2021 at 2:32 PM hamisa <[email protected]> wrote: >>> >>>> Hi there, >>>> I'm running prometheus 2.17 on my openshift 3.9.43 cluster. >>>> when the prometheus pod collapse (deleted, evicted ...) the newly >>>> created pod start reading the wal segments in addiotion to run prometheus >>>> again. >>>> in that process the prometheus' pod container get into OOM state, and >>>> recreated just afterwards which causes CrashLoopBack to the pod. >>>> >>>> is there any solution to this problem? >>>> >>>> thanks in advance. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Prometheus Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-users/b967175e-9658-4d6e-ac27-bb62a5a2f70bn%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/prometheus-users/b967175e-9658-4d6e-ac27-bb62a5a2f70bn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/95931435-aab7-4859-94b1-aee7743841bfn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-users/95931435-aab7-4859-94b1-aee7743841bfn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/247a0a18-8243-49eb-a08a-5396e67591edn%40googlegroups.com.
