flowchartsman opened a new issue #9177:
URL: https://github.com/apache/pulsar/issues/9177
Seeing errors in my log for go functions refusiing a metrics connection.
```
15:59:47.285 [pulsar-web-43-1] INFO org.eclipse.jetty.server.RequestLog -
10.13.2.61 - - [10/Jan/2021:15:59:47 +0000] "GET /metrics HTTP/1.1" 302 0 "-"
"Prometheus/2.15.2" 0
15:59:47.289 [prometheus-stats-44-1] WARN
org.apache.pulsar.functions.worker.FunctionsStatsGenerator - Failed to collect
metrics for function instance tenantname/namespacename/samplertest:0
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
~[?:1.8.0_275]
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[?:1.8.0_275]
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[?:1.8.0_275]
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[?:1.8.0_275]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
~[?:1.8.0_275]
at java.net.Socket.connect(Socket.java:607) ~[?:1.8.0_275]
at java.net.Socket.connect(Socket.java:556) ~[?:1.8.0_275]
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
~[?:1.8.0_275]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
~[?:1.8.0_275]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
~[?:1.8.0_275]
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
~[?:1.8.0_275]
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
~[?:1.8.0_275]
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
~[?:1.8.0_275]
at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
~[?:1.8.0_275]
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
~[?:1.8.0_275]
at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
~[?:1.8.0_275]
at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
~[?:1.8.0_275]
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
~[?:1.8.0_275]
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
~[?:1.8.0_275]
at
org.apache.pulsar.functions.runtime.RuntimeUtils.getPrometheusMetrics(RuntimeUtils.java:427)
~[org.apache.pulsar-pulsar-functions-runtime-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
at
org.apache.pulsar.functions.runtime.process.ProcessRuntime.getPrometheusMetrics(ProcessRuntime.java:320)
~[org.apache.pulsar-pulsar-functions-runtime-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
at
org.apache.pulsar.functions.worker.FunctionsStatsGenerator.generate(FunctionsStatsGenerator.java:71)
~[org.apache.pulsar-pulsar-functions-worker-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
at
org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsGenerator.generate(PrometheusMetricsGenerator.java:96)
~[org.apache.pulsar-pulsar-broker-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
at
org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet.lambda$doGet$0(PrometheusMetricsServlet.java:66)
~[org.apache.pulsar-pulsar-broker-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
at
org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32)
[org.apache.pulsar-managed-ledger-2.8.0-SNAPSHOT.jar:2.8.0-SNAPSHOT]
at
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
[org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_275]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_275]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
[?:1.8.0_275]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_275]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_275]
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
[io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
15:59:47.291 [prometheus-stats-44-1] INFO
org.eclipse.jetty.server.RequestLog - 10.13.2.61 - - [10/Jan/2021:15:59:47
+0000] "GET /metrics/ HTTP/1.1" 200 63238 "http://10.13.2.236:8080/metrics";
"Prometheus/2.15.2" 5
```
The function appears to start a listener on startup
`021/01/10 15:57:16.848 log.go:46: [info] Serving InstanceCommunication on
port 33915`
However its counts DO eventually go up, so perhaps it's intermittant?
Running off of 2.8.0-SNAPSHOT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
