Hey Julius, morning Found a few warn logs from thanos-receive, maybe a possible root cause?
- "Error on ingesting samples with different values but same timestamp" - "Error on ingesting out-of-order samples" Em terça-feira, 30 de maio de 2023 às 18:02:26 UTC-3, Julius Volz escreveu: > Nothing in the query itself should be able to cause a duplicate > labelset in a normal PromQL execution, so I assume that something is wrong > with the underlying Thanos setup or code. Maybe it somehow manages to > select the same duplicate time series from multiple Prometheus servers (or > the long-term storage), because something around the external labels > normally used for differentiating series isn't configured or working > properly? > > On Tue, May 30, 2023 at 10:28 PM Adso “Cho” Castro <koori....@gmail.com> > wrote: > >> Hey, been getting the error below when querying: >> >> *execution: vector cannot contain metrics with the same labelset* >> >> The query: >> >> sum(rate(istio_requests_total{reporter="source", >> destination_workload="dummy-data", response_code=~"2.."}[$_range])) by >> (destination_workload) / sum(rate(istio_requests_total{reporter="source", >> destination_workload="dummy-data"}[$_range])) by (destination_workload) >> >> Funny thing is, this is a Thanos installation. I have a Cortex >> installation with the same data and this query works fine. >> >> Thanks >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to prometheus-use...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/b08f43a8-fd11-4775-838e-350f144df8e8n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-users/b08f43a8-fd11-4775-838e-350f144df8e8n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Julius Volz > PromLabs - promlabs.com > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/36531430-934f-4334-a78c-66abb1b16aa7n%40googlegroups.com.