damccorm opened a new issue, #20922:
URL: https://github.com/apache/beam/issues/20922
Spark runner in some environments (Spark 3 and Databricks) reports a warning
on Unknown metric type. The pipeline works well and the metrics appear correct
but the warning seems to be a problem of the runner.
>
> 2021-03-23T15:10:02,222] [INFO ] o.a.s.m.MetricsSystem - Metrics already
registered
java.lang.IllegalArgumentException: Unknown metric type: class
org.apache.beam.runners.spark.metrics.SparkBeamMetric
at
com.codahale.metrics.MetricRegistry.notifyListenerOfAddedMetric(MetricRegistry.java:532)
~[spark-3.0.1-yarn-shuffle.jar:?]
at com.codahale.metrics.MetricRegistry.addListener(MetricRegistry.java:363)
~[spark-3.0.1-yarn-shuffle.jar:?]
at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:93)
~[spark-3.0.1-yarn-shuffle.jar:?]
at
org.apache.spark.metrics.MetricsSystem.registerSource(MetricsSystem.scala:165)
[spark-core_2.12-3.0.1.jar:3.0.1]
at
org.apache.beam.runners.spark.SparkRunner.registerMetricsSource(SparkRunner.java:275)
[beam-runners-spark-2.27.0.jar:?]
at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:250)
[beam-runners-spark-2.27.0.jar:?]
at org.apache.beam.runners.spark.SparkRunner.run(SparkRunner.java:92)
[beam-runners-spark-2.27.0.jar:?]
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:322)
[beam-sdks-java-harness-2.27.0.jar:?]>
Imported from Jira
[BEAM-12032](https://issues.apache.org/jira/browse/BEAM-12032). Original Jira
may contain additional context.
Reported by: iemejia.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
