hangc0276 commented on pull request #6814:
URL: https://github.com/apache/pulsar/pull/6814#issuecomment-636474961
@sijie I try to use bookkeeper prometheus provider lib the export the
bookie client metric, but failed. The exception is
```
21:49:06.142 [prometheus-stats-43-1] ERROR
org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught
java.lang.ExceptionInInitializerError: null
at
org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet.lambda$doGet$0(PrometheusMetricsServlet.java:70)
~[org.apache.pulsar-pulsar-broker-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at
org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32)
~[org.apache.pulsar-managed-ledger-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
at
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
[org.apache.bookkeeper-bookkeeper-common-4.10.0.jar:4.10.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_92]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_92]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_92]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_92]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_92]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_92]
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[io.netty-netty-common-4.1.48.Final.jar:4.1.48.Final]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92]
Caused by: java.lang.IllegalArgumentException: Collector already registered
that provides name: process_cpu_seconds_total
at
io.prometheus.client.CollectorRegistry.register(CollectorRegistry.java:54)
~[io.prometheus-simpleclient-0.5.0.jar:?]
at io.prometheus.client.Collector.register(Collector.java:139)
~[io.prometheus-simpleclient-0.5.0.jar:?]
at io.prometheus.client.Collector.register(Collector.java:132)
~[io.prometheus-simpleclient-0.5.0.jar:?]
at
io.prometheus.client.hotspot.DefaultExports.initialize(DefaultExports.java:23)
~[io.prometheus-simpleclient_hotspot-0.5.0.jar:?]
at
org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsGenerator.<clinit>(PrometheusMetricsGenerator.java:61)
~[org.apache.pulsar-pulsar-broker-2.6.0-SNAPSHOT.jar:2.6.0-SNAPSHOT]
... 11 more
```
The reason is bookkeeper lib `PrometheusMetricsProvider` will export the
follow metrics by default and can't turn off.
```
this.registerMetrics(new StandardExports());
this.registerMetrics(new MemoryPoolsExports());
this.registerMetrics(new GarbageCollectorExports());
this.registerMetrics(new ThreadExports());
this.registerMetrics(Gauge.build("jvm_memory_direct_bytes_used",
"-").create().setChild(new Child() {
public double get() {
return PrometheusMetricsProvider.directMemoryUsage != null ?
(double)PrometheusMetricsProvider.directMemoryUsage.longValue() : 0.0D / 0.0;
}
}, new String[0]));
this.registerMetrics(Gauge.build("jvm_memory_direct_bytes_max",
"-").create().setChild(new Child() {
public double get() {
return (double)PlatformDependent.maxDirectMemory();
}
}, new String[0]));
```
However the pulsar broker prometheus metric also export those metrics to
prometheus, some of the metrics registered twice, which will lead to metric
export failed.
Could you give me some ideas ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
