schernic opened a new issue, #14327: URL: https://github.com/apache/druid/issues/14327
### Description Provide a way to disable collection (not only emission) of query metrics. Alternatively, introduce sampling to the metric collection or make the collected metric types and dimensions configurable. ### Motivation For certain workloads, we have experienced an unusually high CPU load on our historical nodes. Often times the load increased to the point that queries started to time out and Zookeeper considered the node unavailable. Out of curiosity I used a profile to look into the CPU hot spots and found that metric generation was causing most of that load. As we are not even emitting metrics I was looking for a way to disable metric collection altogether but found no way to configure that. Nevertheless, checking the code I found a way to disable metric collection during runtime using JMX in order to check if that would really help with the high CPU load. It reduced the load on that historical by 80% which makes me believe that some features around making it possible to disable collection of query metrics or dimension could be valuable addition for others as well. Actually, I don't see why metric collection would be enabled by default if no emitter was configured. Sampling or rate limiting of metric collections would also be useful options in cases like ours. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
