[
https://issues.apache.org/jira/browse/BEAM-13074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julien Tournay updated BEAM-13074:
----------------------------------
Resolution: Not A Bug
Status: Resolved (was: Triage Needed)
Closing this issue bc the problem is not in Beam.
This is actually "normal" in the Flink runtime. At the end of the execution of
a task, Flink "unregister" user defined metrics. The metrics technically still
exist on the taskmanager and it is still possible to access them at the end of
the execution from the created accumulators but the reporters will just stop
reporting them. (See
[MetricsRegistryImpl.unregister|[https://github.com/apache/flink/blob/a390675e6f778387933d0fc53cca0e627bbd7c1a/flink-runtime/src/main/java/org/apache/flink/runtime/metrics/MetricRegistryImpl.java#L434-L436])]
For batch pipeline, it is likely that user defined metrics never get a chance
to be pushed to a TSD because the task gets closed and metrics unregistered
immediately after "finishBundle".
> Metrics are not reported by the Flink runner
> --------------------------------------------
>
> Key: BEAM-13074
> URL: https://issues.apache.org/jira/browse/BEAM-13074
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Affects Versions: 2.33.0
> Reporter: Julien Tournay
> Priority: P1
>
> When running a job using the Flink runner, metrics are not reported.
> See [https://github.com/jto/beam-flink-broken-metrics] for a minimal example.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
