[
https://issues.apache.org/jira/browse/FLINK-9616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517205#comment-16517205
]
ASF GitHub Bot commented on FLINK-9616:
---------------------------------------
GitHub user addisonj opened a pull request:
https://github.com/apache/flink/pull/6183
[FLINK-9616][metrics] Fix datadog to include shaded deps
## What is the purpose of the change
This fixes a broken build that wasn't properly including a shaded in the
jar it builds. This causes the instantiation of DatadogHttpReporter to fail and
with no easy way to fix since the dependencies exist on a shaded import path.
## Brief change log
- Changes
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
However, it can be validated by:
```
cd flink-metrics/flink-metrics-datadog
mvn package
jar tf target/flink-metrics-datadog-1.6-SNAPSHOT.jar
```
And then seeing the expected okhttp3 and okio dependencies being included
in the resulting jar.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): yes, but brings in
line with documented behavior here:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/monitoring/metrics.html#datadog-orgapacheflinkmetricsdatadogdatadoghttpreporter
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/instructure/flink datadog_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6183.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6183
----
commit 10fe56a9adbe6f35dc4b5fae0e7478e99028f5f7
Author: Addison Higham <ahigham@...>
Date: 2018-06-19T14:49:56Z
[FLINK-9616] Fix datadog to include shaded deps
flink-metrics-datadog wasn't properly included it's shaded dependencies
in the jar it builds.
Looking at other places where shaded dependecies are used, it seems like
the build wasn't working as intended.
----
> DatadogHttpReporter fails to be created due to missing shaded dependency
> ------------------------------------------------------------------------
>
> Key: FLINK-9616
> URL: https://issues.apache.org/jira/browse/FLINK-9616
> Project: Flink
> Issue Type: Bug
> Components: Metrics
> Affects Versions: 1.5.0
> Reporter: Addison Higham
> Priority: Major
>
> When using the DatadogHttpReporter, it fails to instantiate with the
> following exception:
> {code:java}
> 2018-06-19 06:01:19,640 INFO
> org.apache.flink.runtime.metrics.MetricRegistryImpl - Configuring dghttp with
> {apikey=<redacted>, tags=<redacted>,
> class=org.apache.flink.metrics.datadog.DatadogHttpReporter}.
> 2018-06-19 06:01:19,642 ERROR
> org.apache.flink.runtime.metrics.MetricRegistryImpl - Could not instantiate
> metrics reporter dghttp. Metrics might not be exposed/reported.
> java.lang.NoClassDefFoundError: org/apache/flink/shaded/okhttp3/MediaType
> at
> org.apache.flink.metrics.datadog.DatadogHttpClient.<clinit>(DatadogHttpClient.java:45)
> at
> org.apache.flink.metrics.datadog.DatadogHttpReporter.open(DatadogHttpReporter.java:105)
> at
> org.apache.flink.runtime.metrics.MetricRegistryImpl.<init>(MetricRegistryImpl.java:150)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.createMetricRegistry(ClusterEntrypoint.java:413)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:274)
> at
> org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint.initializeServices(MesosSessionClusterEntrypoint.java:92)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:225)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:189)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1889)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:188)
> at
> org.apache.flink.mesos.entrypoint.MesosSessionClusterEntrypoint.main(MesosSessionClusterEntrypoint.java:181)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.flink.shaded.okhttp3.MediaType
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 14 more
> {code}
> Looking at the pom.xml for `flink-metrics-datadog` it looks like that
> dependency is intended to be shaded and included in the jar, however, when we
> build the jar we see the following lines:
>
> {noformat}
> $ mvn package
> [INFO] Scanning for projects...
> [INFO]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Building flink-metrics-datadog 1.5.0
> [INFO]
> ------------------------------------------------------------------------
> ....
> [INFO] --- maven-shade-plugin:3.0.0:shade (shade-flink) @
> flink-metrics-datadog ---
> [INFO] Excluding com.squareup.okhttp3:okhttp:jar:3.7.0 from the shaded jar.
> [INFO] Excluding com.squareup.okio:okio:jar:1.12.0 from the shaded jar.
> [INFO] Including org.apache.flink:force-shading:jar:1.5.0 in the shaded jar.
> [INFO] Replacing original artifact with shaded artifact.
> {noformat}
> And inspecting the built jar:
> {noformat}
> $ jar tf flink-metrics-datadog-1.5.0.jar
> META-INF/
> META-INF/MANIFEST.MF
> org/
> org/apache/
> org/apache/flink/
> org/apache/flink/metrics/
> org/apache/flink/metrics/datadog/
> org/apache/flink/metrics/datadog/DatadogHttpClient$EmptyCallback.class
> org/apache/flink/metrics/datadog/DMetric.class
> org/apache/flink/metrics/datadog/DSeries.class
> org/apache/flink/metrics/datadog/DGauge.class
> org/apache/flink/metrics/datadog/DatadogHttpReporter.class
> org/apache/flink/metrics/datadog/DatadogHttpClient.class
> org/apache/flink/metrics/datadog/MetricType.class
> org/apache/flink/metrics/datadog/DatadogHttpReporter$DatadogHttpRequest.class
> org/apache/flink/metrics/datadog/DMeter.class
> org/apache/flink/metrics/datadog/DCounter.class
> META-INF/DEPENDENCIES
> META-INF/maven/
> META-INF/maven/org.apache.flink/
> META-INF/maven/org.apache.flink/flink-metrics-datadog/
> META-INF/maven/org.apache.flink/flink-metrics-datadog/pom.xml
> META-INF/maven/org.apache.flink/flink-metrics-datadog/pom.properties
> META-INF/NOTICE
> {noformat}
> We don't see the included dependencies
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
