forzamehlano opened a new issue, #14638: URL: https://github.com/apache/druid/issues/14638
We're using the prometheus-emitter for metrics on druid 26.0.0. This is all working OK, metrics are flowing and all is well with the world (almost). Query metrics are showing up as expected, but the prometheus endpoint never appears to timeout the metrics. For example, we'll run a query, the metric for that query will show up (druid_query_wait_time as an example) and be scraped by prometheus. A minute/hour/day later, that same query metric is seemingly still being exposed by the prometheus emitter. Adding in some of the dimensions for query metrics (id being one), the cardinality is increased and it's causing the number of metrics to grow rapidly and causing stability issues. It's all pretty standard prometheus-emitter deployments using the exporter strategy, so I'm wondering whether I'm missing something (maybe a configuration option for the emitter to reset the metrics it's exposing after a pre-defined period). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
