Bowen Li created FLINK-7405: ------------------------------- Summary: Reduce excessive warning logging from DatadogHttpReporter Key: FLINK-7405 URL: https://issues.apache.org/jira/browse/FLINK-7405 Project: Flink Issue Type: Improvement Affects Versions: 1.3.2 Reporter: Bowen Li Assignee: Bowen Li Priority: Minor Fix For: 1.4.0, 1.3.3
DatadogHttpReporter is logging too much when there's a connection timeout, and we need to reduce the amount of logging noise. The excessive logging looks like: {code:java} 2017-08-07 19:30:54,408 WARN org.apache.flink.metrics.datadog.DatadogHttpReporter - Failed reporting metrics to Datadog. java.net.SocketTimeoutException: timeout at org.apache.flink.shaded.okio.Okio$4.newTimeoutException(Okio.java:227) at org.apache.flink.shaded.okio.AsyncTimeout.exit(AsyncTimeout.java:284) at org.apache.flink.shaded.okio.AsyncTimeout$2.read(AsyncTimeout.java:240) at org.apache.flink.shaded.okio.RealBufferedSource.indexOf(RealBufferedSource.java:344) at org.apache.flink.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:216) at org.apache.flink.shaded.okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.java:210) at org.apache.flink.shaded.okhttp3.internal.http1.Http1Codec.readResponseHeaders(Http1Codec.java:189) at org.apache.flink.shaded.okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:75) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at org.apache.flink.shaded.okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:45) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at org.apache.flink.shaded.okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at org.apache.flink.shaded.okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at org.apache.flink.shaded.okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at org.apache.flink.shaded.okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at org.apache.flink.shaded.okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:185) at org.apache.flink.shaded.okhttp3.RealCall.execute(RealCall.java:69) at org.apache.flink.metrics.datadog.DatadogHttpClient.send(DatadogHttpClient.java:85) at org.apache.flink.metrics.datadog.DatadogHttpReporter.report(DatadogHttpReporter.java:142) at org.apache.flink.runtime.metrics.MetricRegistry$ReporterTask.run(MetricRegistry.java:381) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketException: Socket closed at java.net.SocketInputStream.read(SocketInputStream.java:204) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940) at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) at org.apache.flink.shaded.okio.Okio$2.read(Okio.java:138) at org.apache.flink.shaded.okio.AsyncTimeout$2.read(AsyncTimeout.java:236) ... 29 more {code} Let's reduce it to something as concise as this: {code:java} 2017-08-07 19:30:54,408 WARN org.apache.flink.metrics.datadog.DatadogHttpReporter - Failed reporting metrics to Datadog. java.net.SocketTimeoutException: timeout {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)