[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821668#comment-15821668 ] ASF GitHub Bot commented on FLINK-4396: --- GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/3116 [docs} [metrics] Update metrics documentation This PR makes several changes to the metrics documentation. Changes include * clearly document that reporter jars must be placed in the /lib folder, and document the /opt folder containing jars for flink reporters (FLINK-4396) * document that multiple reporters exist * extend the JMX documentation regarding domain and key-property list (FLINK-5451) * document metrics visualiuation in web-frontend * rework the system metrics section to be more readable, and add missing metrics (FLINK-5446) Here's a before/after of the system-metrics section: Before:  After:  You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink docs_metrics Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3116.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3116 commit 9437a073f988eda789cb992d05bfe83520ce72d7 Author: zentol Date: 2017-01-11T14:09:26Z [FLINK-4396] [docs] Document /opt reporter jars commit 395693c50a09a2575d9325e3fd99ca625856c4b6 Author: zentol Date: 2017-01-13T11:16:29Z [docs] Document reporter plurality commit f0a6abf30555d8c60cec3324f0f93c8d966df508 Author: zentol Date: 2017-01-13T11:17:20Z [FLINK-5451] Extend JMX reporter section commit 03ac8f602af8bcb30a59b29a3a7255e65e2bcd31 Author: zentol Date: 2017-01-13T11:18:01Z [docs] Document metrics visualization in web-frontend commit ca9c3bad269d3b1764ff5ef4686065fce5b14898 Author: zentol Date: 2017-01-13T11:18:34Z [FLINK-5446] [docs] Rework system-metrics section > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden >Assignee: Chesnay Schepler > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15818206#comment-15818206 ] Chesnay Schepler commented on FLINK-4396: - you are correct, it doesn't mention it. That's strange though, I'm pretty sure i added it at one point... > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816509#comment-15816509 ] Steven Ruppert commented on FLINK-4396: --- I just ran into this today. Reading: https://ci.apache.org/projects/flink/flink-docs-release-1.2/monitoring/metrics.html it doesn't say anywhere there you need to download extra libs. > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15715283#comment-15715283 ] Ufuk Celebi commented on FLINK-4396: Good idea > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432878#comment-15432878 ] Greg Hogan commented on FLINK-4396: --- This should be verified. I have also needed to copy additional jars to lib/ when configuring reporters. Also, have we considered providing a metrics reporters package on the download page? > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Fix For: 1.1.2 > > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432788#comment-15432788 ] Chesnay Schepler commented on FLINK-4396: - We don't require users to track down any dependencies; they should all be included in the respective reporter jars. > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Fix For: 1.1.2 > > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15432746#comment-15432746 ] Greg Hogan commented on FLINK-4396: --- I've relabeled this as an Improvement because we should not require users to track down Maven dependencies. If we don't want to include these jars in flink-dist then why not make flink-metrics a separate, user-installed jar with these dependencies included? > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Improvement > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Fix For: 1.1.2 > > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-4396) GraphiteReporter class not found at startup of jobmanager
[ https://issues.apache.org/jira/browse/FLINK-4396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421110#comment-15421110 ] Chesnay Schepler commented on FLINK-4396: - For clarification I would like to ask whether you really needed the metrics-dropwizard and DW metrics-graphite jars, as they should be included in the flink-metrics-graphite jar in 1.1. I've relabeled this issue as a Wish and not a bug for now as this is generally intended (and documented) behavior; any non-JMX reporter must be provided by the user by putting the jar in the /lib folder. > GraphiteReporter class not found at startup of jobmanager > - > > Key: FLINK-4396 > URL: https://issues.apache.org/jira/browse/FLINK-4396 > Project: Flink > Issue Type: Wish > Components: Build System, Metrics >Affects Versions: 1.1.1 > Environment: Windows and Unix >Reporter: RWenden > Fix For: 1.1.2 > > Original Estimate: 4h > Remaining Estimate: 4h > > For Flink 1.1.1 we configured Graphite metrics settings on the > flink-conf.yaml (for job manager (and taskmanager)). > We see the following error in the log: > 2016-08-15 14:20:34,167 ERROR org.apache.flink.runtime.metrics.MetricRegistry > - Could not instantiate metrics reportermy_reporter. Metrics > might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.graphite.GraphiteReporter > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.flink.runtime.metrics.MetricRegistry.(MetricRegistry.java:119) > We found out that this class is not packaged inside flink-dist_2.11-1.1.1.jar. > Long story short: we had to install/provide the following jars into the lib > folder to make Graphite metrics to work: > flink-metrics-graphite-1.1.1.jar > flink-metrics-dropwizard-1.1.1.jar > metrics-graphite-3.1.0.jar (from dropwizard) > We think these libraries (and the ones for Ganglia,StatsD,...) should be > included in flink-dist_2.11-1.1.1.jar, for these are needed at manager > startup time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
