[
https://issues.apache.org/jira/browse/BEAM-6829?focusedWorklogId=251792&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-251792
]
ASF GitHub Bot logged work on BEAM-6829:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/May/19 13:57
Start Date: 31/May/19 13:57
Worklog Time Spent: 10m
Work Description: tweise commented on issue #8585: [BEAM-6829] Use
transform name for metric namespace if none provided
URL: https://github.com/apache/beam/pull/8585#issuecomment-497718117
With this change, I still see the duplicate metrics warning:
```
[Source: Impulse -> [2]write/Write/WriteImpl/DoOnce/{FlatMap(<lambda at
core.py:2254>), Map(decode)} -> [1]write/Write/WriteImpl/InitializeWrite ->
(Map -> ToKeyedWorkItem, Map -> ToKeyedWorkItem, Map -> ToKeyedWorkItem) (1/1)]
WARN org.apache.flink.metrics.MetricGroup - Name collision: Group already
contains a Metric with the name 'beam.metric:element_count:v1'. Metric will not
be reported.[localhost, taskmanager, 0bd0adb5-1fba-49c7-811f-c8ae76f83bb8,
BeamApp-tweise-0531134309-499ce3fa,
[2]write/Write/WriteImpl/DoOnce/{FlatMap(<lambda at core.py:2254>),
Map(decode)}, 0]
```
But I also suspect that the proposed fix will have other side effects, since
it is changing the metric name (at least that's how it would appear in our
metric system).
The metrics as logged with the slf4j reporter before this change look like
this:
```
localhost.taskmanager.87150cf7-cc34-412c-9057-656edef0590c.BeamApp-tweise-0531054435-7c2c37e1.[1]write/Write/WriteImpl/FinalizeWrite.0.numRecordsIn:
1
localhost.taskmanager.87150cf7-cc34-412c-9057-656edef0590c.BeamApp-tweise-0531054435-7c2c37e1.[3]{read,
split, pair_with_one}.0.beam.metric:element_count:v1: 1
localhost.taskmanager.87150cf7-cc34-412c-9057-656edef0590c.BeamApp-tweise-0531054435-7c2c37e1.ToKeyedWorkItem.0.numRecordsOut:
0
localhost.taskmanager.87150cf7-cc34-412c-9057-656edef0590c.BeamApp-tweise-0531054435-7c2c37e1.[3]{read,
split,
pair_with_one}.0.ref_AppliedPTransform_read/Read/ReadSplits_16.metric:pardo_execution_time:finish_bundle_msecs:v1:
0
```
In our backend this translates to
`<prefix>.operator.beam-metric-element_count-v1.gauge.sum`
If we replace `beam` with `<transform_name>`, then we would effectively
change the metric name. Instead, shouldn't the transform name be part of the
scope?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 251792)
Time Spent: 50m (was: 40m)
> Duplicate metric warnings clutter log
> -------------------------------------
>
> Key: BEAM-6829
> URL: https://issues.apache.org/jira/browse/BEAM-6829
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.11.0
> Reporter: Thomas Weise
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Logs fill up quickly with these warnings:
> {code:java}
> WARN org.apache.flink.metrics.MetricGroup - Name collision: Group already
> contains a Metric with the name ...{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
