[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16220104#comment-16220104
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user asfgit closed the pull request at:

https://github.com/apache/flink/pull/4661


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218439#comment-16218439
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4661
  
merging.


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217084#comment-16217084
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user yew1eb commented on the issue:

https://github.com/apache/flink/pull/4661
  
@zentol thanks a lot for your review. I have updated the PR according to 
your comments.


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216758#comment-16216758
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4661#discussion_r146533152
  
--- Diff: flink-metrics/flink-metrics-slf4j/pom.xml ---
@@ -0,0 +1,65 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd;>
+
+   4.0.0
+
+   
+   org.apache.flink
+   flink-metrics
+   1.4-SNAPSHOT
+   ..
+   
+
+   flink-metrics-slf4j
+   flink-metrics-slf4j
+
+   
+   
+   org.apache.flink
+   flink-annotations
+   ${project.version}
+   provided
+   
+   
+   org.apache.flink
+   
flink-runtime_${scala.binary.version}
+   ${project.version}
+   provided
+   
+
+   
+   org.apache.flink
+   flink-metrics-core
+   ${project.version}
+   provided
+   
+
+   
+
+   
+   org.apache.flink
+   flink-test-utils-junit
--- End diff --

add `test


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216757#comment-16216757
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4661#discussion_r146533089
  
--- Diff: flink-metrics/flink-metrics-slf4j/pom.xml ---
@@ -0,0 +1,65 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
+xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd;>
+
+   4.0.0
+
+   
+   org.apache.flink
+   flink-metrics
+   1.4-SNAPSHOT
+   ..
+   
+
+   flink-metrics-slf4j
+   flink-metrics-slf4j
+
+   
+   
+   org.apache.flink
+   flink-annotations
+   ${project.version}
+   provided
+   
+   
+   org.apache.flink
+   
flink-runtime_${scala.binary.version}
+   ${project.version}
+   provided
--- End diff --

move to test dependency block and set scope to `test`


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216756#comment-16216756
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4661#discussion_r139702401
  
--- Diff: 
flink-metrics/flink-metrics-slf4j/src/main/java/org/apache/flink/metrics/slf4j/Slf4jReporter.java
 ---
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.metrics.slf4j;
+
+import org.apache.flink.annotation.VisibleForTesting;
+import org.apache.flink.metrics.Counter;
+import org.apache.flink.metrics.Gauge;
+import org.apache.flink.metrics.Histogram;
+import org.apache.flink.metrics.HistogramStatistics;
+import org.apache.flink.metrics.Meter;
+import org.apache.flink.metrics.Metric;
+import org.apache.flink.metrics.MetricConfig;
+import org.apache.flink.metrics.reporter.AbstractReporter;
+import org.apache.flink.metrics.reporter.MetricReporter;
+import org.apache.flink.metrics.reporter.Scheduled;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Map;
+
+/**
+ * {@link MetricReporter} that exports {@link Metric Metrics} via SLF4J 
{@link Logger}.
+ */
+public class Slf4jReporter extends AbstractReporter implements Scheduled {
+   private static final Logger LOG = 
LoggerFactory.getLogger(Slf4jReporter.class);
+   private static final String lineSeparator = System.lineSeparator();
+
+   @VisibleForTesting
+   public Map getGauges() {
+   return gauges;
+   }
+
+   @VisibleForTesting
+   public Map getCounters() {
+   return counters;
+   }
+
+   @VisibleForTesting
+   public Map getHistograms() {
+   return histograms;
+   }
+
+   @VisibleForTesting
+   public Map getMeters() {
+   return meters;
+   }
+
+   @Override
+   public void open(MetricConfig metricConfig) {
+
+   }
+
+   @Override
+   public void close() {
+
+   }
+
+   @Override
+   public void report() {
+   StringBuilder builder = new StringBuilder();
+   
builder.append(lineSeparator).append("=== Starting 
metrics report ===").append(lineSeparator);
+   builder.append("-- Counters 
---").append(lineSeparator);
+   for (Map.Entry metric : counters.entrySet()) {
+   builder.append(metric.getValue()).append(": 
count=").append(metric.getKey().getCount()).append(lineSeparator);
+   }
+
+   builder.append(lineSeparator).append("-- Gauges 
-").append(lineSeparator);
+   for (Map.Entry metric : gauges.entrySet()) {
+   builder.append(metric.getValue()).append(": 
value=").append(metric.getKey().getValue()).append(lineSeparator);
+   }
+
+   builder.append(lineSeparator).append("-- Histograms 
-").append(lineSeparator);
+   for (Map.Entry metric : 
histograms.entrySet()) {
+   HistogramStatistics stats = 
metric.getKey().getStatistics();
+   builder.append(metric.getValue()).append(": 
count=").append(stats.size())
+   .append(", 
min=").append(stats.getMin()).append(", max=").append(stats.getMax())
+   .append(", 
mean=").append(stats.getMean()).append(", stddev=").append(stats.getStdDev())
+   .append(", 
p50=").append(stats.getQuantile(0.50)).append(", 

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216754#comment-16216754
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4661#discussion_r146535744
  
--- Diff: 
flink-metrics/flink-metrics-slf4j/src/test/resources/log4j-test.properties ---
@@ -0,0 +1,24 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+log4j.rootLogger=INFO, testlogger
--- End diff --

This will cause a bit of noise in the logs that I would like to avoid.

Instead, can we, If the log level is configured to OFF, remove all other 
appenders and set it to INFO? This will make the test run without affecting log 
files etc.


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216755#comment-16216755
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on a diff in the pull request:

https://github.com/apache/flink/pull/4661#discussion_r139702445
  
--- Diff: 
flink-metrics/flink-metrics-slf4j/src/test/java/org/apache/flink/metrics/slf4j/Slf4jReporterTest.java
 ---
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.metrics.slf4j;
+
+import org.apache.flink.api.common.JobID;
+import org.apache.flink.configuration.ConfigConstants;
+import org.apache.flink.configuration.Configuration;
+import org.apache.flink.configuration.MetricOptions;
+import org.apache.flink.metrics.Gauge;
+import org.apache.flink.metrics.SimpleCounter;
+import org.apache.flink.runtime.metrics.MetricRegistry;
+import org.apache.flink.runtime.metrics.MetricRegistryConfiguration;
+import org.apache.flink.runtime.metrics.groups.TaskManagerJobMetricGroup;
+import org.apache.flink.runtime.metrics.groups.TaskManagerMetricGroup;
+import org.apache.flink.runtime.metrics.groups.TaskMetricGroup;
+import org.apache.flink.util.AbstractID;
+import org.apache.flink.util.TestLogger;
+
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.hamcrest.Matchers.equalTo;
+import static org.junit.Assert.assertThat;
+import static org.junit.Assert.assertTrue;
+
+/**
+ * Test for {@link Slf4jReporter}.
+ */
+
--- End diff --

remove empty line


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208652#comment-16208652
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user yew1eb commented on the issue:

https://github.com/apache/flink/pull/4661
  
CC @zentol 


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou UTC+8
>Priority: Minor
>  Labels: easyfix, starter
> Fix For: 1.4.0
>
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178598#comment-16178598
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user yew1eb commented on the issue:

https://github.com/apache/flink/pull/4661
  
@zentol Please let me know if this is good :)


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou_UTC+8
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16178585#comment-16178585
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user yew1eb commented on the issue:

https://github.com/apache/flink/pull/4661
  
cc @zentol


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou_UTC+8
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161582#comment-16161582
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

Github user zentol commented on the issue:

https://github.com/apache/flink/pull/4661
  
Really nice that you figured out to rename the package to slf4j :)

I'll check this out in more detail next week.


> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16159094#comment-16159094
 ] 

ASF GitHub Bot commented on FLINK-4831:
---

GitHub user yew1eb opened a pull request:

https://github.com/apache/flink/pull/4661

[FLINK-4831] Implement a slf4j metric reporter

## What is the purpose of the change

Implement a slf4j metric reporter For debugging.

## Brief change log

## Verifying this change
  - *Added unit test Slf4jReporterTest*
  - *Build a local flink cluster test Slf4jReporter, it works well*

## Does this pull request potentially affect one of the following parts:

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: (no)
  - The serializers: (no)
  - The runtime per-record code paths (performance sensitive): (no)
  - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)

## Documentation

  - Does this pull request introduce a new feature? (yes)
  - If yes, how is the feature documented? (docs)



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yew1eb/flink FLINK-4831

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/flink/pull/4661.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4661


commit 79804a6ae554b93d41e5c8748c6442c2aa330fa2
Author: yew1eb 
Date:   2017-09-06T15:09:13Z

Implement a slf4j metric reporter




> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[Chesnay Schepler (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154076#comment-16154076
 ] 

Chesnay Schepler commented on FLINK-4831:
-

It's a good question what is missing, let's see. For one we're missing a null 
check when accessing the value of a gauge, and the whole branch should be 
rebased to the latest master. Documentation is also missing. Something to 
investigate would be whether we can ensure that the report is written as one 
big lock, instead of interleaved with other logging messages.

It is true that we can extend AbstractReporter to simplify things.

> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[Hai Zhou (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153866#comment-16153866
 ] 

Hai Zhou commented on FLINK-4831:
-

I recently read *dropwizard metric*, i seem to have some ideas.

> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Hai Zhou
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[Hai Zhou (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151947#comment-16151947
 ] 

Hai Zhou commented on FLINK-4831:
-

Hi [~Zentol],
I checked your code, it's very good. 
but, I do not know what else need to be modified?
I just found via *extends AbstractReporter implements Scheduled*, the 
implementation would be easier. 

> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[Chesnay Schepler (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136531#comment-16136531
 ] 

Chesnay Schepler commented on FLINK-4831:
-

The relevant interfaces are the {{Metric}} classes, {{MetricReporter}} 
+{{Scheduled}} and {{MetricGroup}}. You can also take a look at other reporter 
implementations in Flink, and the metrics documentation 
https://ci.apache.org/projects/flink/flink-docs-release-1.4/monitoring/metrics.html.

> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Sumit Sarin
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (FLINK-4831) Implement a slf4j metric reporter

[Sumit Sarin (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-4831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135012#comment-16135012
 ] 

Sumit Sarin commented on FLINK-4831:


Hey Chesner,
After going through slf4j and log4j documentation and your past work in 
progress,
I believe that i do need to have an understanding of metrics in depth. So 
should i get acquainted with that or am i going in the wrong direction? I am 
currently unaware about counters, gauges, meters, histograms, etc.

> Implement a slf4j metric reporter
> -
>
> Key: FLINK-4831
> URL: https://issues.apache.org/jira/browse/FLINK-4831
> Project: Flink
>  Issue Type: Improvement
>  Components: Metrics
>Affects Versions: 1.1.2
>Reporter: Chesnay Schepler
>Assignee: Sumit Sarin
>Priority: Minor
>  Labels: easyfix, starter
>
> For debugging purpose it would be very useful to have a log4j metric 
> reporter. If you don't want to setup a metric backend you currently have to 
> rely on JMX, which a) works a bit differently than other reporters (for 
> example it doesn't extend AbstractReporter) and b) makes it a bit tricky to 
> analyze results as metrics are cleaned up once a job finishes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)