divick commented on issue #12885:
URL: https://github.com/apache/druid/issues/12885#issuecomment-1273460452
@holyachon To make it easier to debug and change configuration I have
provisioned druid in k8s using druid-operator instead of logging to each server
separately (master, data, query) and changing and restarting them. With that I
can see the metrics __are__ getting emitted. I am guessing that may be it was
not emitted earlier because metricsDimensions.json was not specified? Anyhow
given now I can see some metrics being emitted but now I see other issues that
it complain of unmapped metrics:
```
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/timeout/count]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/threads/started]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/threads/finished]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/threads/live]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/threads/liveDaemon]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/threads/livePeak]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/max]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/capacity]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/used]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/init]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/max]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/capacity]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/used]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/init]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/max]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/capacity]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/used]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/init]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/max]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/capacity]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/used]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/gc/mem/init]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/heapAlloc/bytes]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[avatica/server/AvaticaProtobufHandler/Handler/RequestTimings]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[avatica/server/AvaticaJsonHandler/Handler/RequestTimings]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[avatica/remote/ProtobufHandler/Handler/Serialization]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[avatica/remote/JsonHandler/Handler/Serialization]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/cpu/user]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/cpu/total]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/cpu/sys]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jvm/cpu/percent]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/delta/put/ok]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/delta/put/error]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/delta/put/oversized]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/total/put/ok]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/total/put/error]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/total/put/oversized]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/caffeine/delta/requests]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/caffeine/total/requests]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/caffeine/delta/loadTime]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/caffeine/total/loadTime]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/caffeine/delta/evictionBytes]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[query/cache/caffeine/total/evictionBytes]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/numOpenConnections]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/total]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/idle]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/isLowOnThreads]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/min]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/max]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/queueSize]
[MonitorScheduler-0] org.a.d.e.p.PrometheusEmitter - Unmapped metric
[jetty/threadPool/busy]
```
This is seen on broker and other nodes too. As per docs even though, I have
extended the metricDimensions.json so that these metrics are mapped. As an
example to enable jvm threads metrics I have added
org.apache.druid.java.util.metrics.JvmThreadsMonitor in the list of
druid.monitoring.monitors list and extended the metricsDimensions as below:
```
"jvm/threads/started" : { "dimensions" : [], "type" : "gauge" },
"jvm/threads/finished" : { "dimensions" : [], "type" : "gauge" },
"jvm/threads/live" : { "dimensions" : [], "type" : "gauge" },
```
But then also I see these metrics being unmapped in the logs. Any thoughts?
Should I create a separate issue on github for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
