Hey, Unless there is some bug on the receiving side (maybe your front proxy masking the actual status code) or Cortex - both Cortex and Thanos Receive in cases of not accepting write for reasons like this (something that there is no point retrying for) returns the status code that tells Prometheus to drop those requests and not retry.
Kind Regards, Bartek Płotka (@bwplotka) On Sat, 12 Sep 2020 at 22:33, Ruben Papovyan <[email protected]> wrote: > Hi team, > What are the options to disable remote write retry ? > Can I use following config to disable remote write retry ? > ``` > remote_write: > url: http://cortex.local.int > queue_config: > min_backoff: 2h > max_backoff: 2h > ``` > or if I need to retry 4 times can I use config ? > ``` > remote_write: > url: http://cortex.local.int > queue_config: > min_backoff: 30m > max_backoff: 2h > ``` > > What are recommendations ? > > My guess here that after 2h WAL will be compacted and data will not be > resend ? > Movement for this that i had network outage and cortex will not accept > metrics(sample timestamp out of order) and it end up where prometheus > ddosed cortex. > > > Thank you, > Ruben > > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/ac8567aa-de1d-4e34-8074-8e8a924a9c30n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/ac8567aa-de1d-4e34-8074-8e8a924a9c30n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAMssQwbHs63%3D-EndnYD0NYF%3DgwqoKyy4OtNr1%2BbpQjhO4faOrA%40mail.gmail.com.
