Micah Wylde created BEAM-6172:
---------------------------------
Summary: Flink metrics are not generated in standard format
Key: BEAM-6172
URL: https://issues.apache.org/jira/browse/BEAM-6172
Project: Beam
Issue Type: Improvement
Components: runner-flink
Affects Versions: 2.8.0
Reporter: Micah Wylde
The metrics that the flink runner exports do not follow the standard format
used by Flink, and doesn't respect Flink metric configuration options.
For example (with the default metrics configuration) beam produces a metric:
{code}
10-100-209-71.taskmanager.0f29b420b63fea58f6f321bc0cbf45f3.BeamApp-mwylde-1203224439-a7d8fdf6.group.0.__counter__group__org-apache-beam-runners-core-ReduceFnRunner__droppedDueToClosedWindow
{code}
whereas a native Flink metric looks like:
{code}
10-100-209-71.taskmanager.0f29b420b63fea58f6f321bc0cbf45f3.BeamApp-mwylde-1203224439-a7d8fdf6.Source-Custom-Source-7Kinesis-None-beam-env-docker-v1-0-ToKeyedWorkItem.0.numRecordsOut
{code}
In particular, Beam should respect the
[metric.scope.delimiter|https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/config.html#metrics-scope-delimiter]
configuration for separating components of a metric (currently it uses "__"),
and should not include the type of metric (counter, gauge, etc.).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
