On 17.01.25 08:49, Mike W wrote: > > Background: We have a metric named "order_total" that has a very high > cardinality label "store_number" slowing down our queries and causing > massive resource usage that we need to drop during/before ingestion.
In general, you cannot reduce the cardinality by just removing the label or by changing the label value to the same. The many series will not disappear by that, they will just collide, so that their values will all be written into the same series in the TSDB, creating a nonsensical series with possibly many duplicate timestamp errors, too. What you are looking for is a feature to actually aggregate a set of high cardinality series into fewer series, but vanilla Prometheus does not have such a feature (yet). It is offered by some of the cloud vendors ingesting Prometheus metrics. For ideas and work in Prometheus itself, see the following: - https://github.com/prometheus/prometheus/issues/394 - https://github.com/prometheus/prometheus/pull/10529 - https://docs.google.com/document/d/11LC3wJcVk00l8w5P3oLQ-m3Y37iom6INAMEu2ZAGIIE/edit?tab=t.0#bookmark=id.hs3vj63cj7uk -- Björn Rabenstein [PGP-ID] 0x851C3DA17D748D03 [email] bjo...@rabenste.in -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/prometheus-users/Z4%2B5Yyff%2B0yh%2BFEg%40mail.rabenste.in.
