[
https://issues.apache.org/jira/browse/BEAM-6829?focusedWorklogId=325428&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-325428
]
ASF GitHub Bot logged work on BEAM-6829:
----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Oct/19 01:33
Start Date: 09/Oct/19 01:33
Worklog Time Spent: 10m
Work Description: tweise commented on pull request #8585: [BEAM-6829] Use
transform/pcollection name for metric namespace if none provided
URL: https://github.com/apache/beam/pull/8585#discussion_r332798020
##########
File path:
runners/core-java/src/main/java/org/apache/beam/runners/core/metrics/MonitoringInfoMetricName.java
##########
@@ -51,7 +51,14 @@ private MonitoringInfoMetricName(String urn, Map<String,
String> labels) {
@Override
public String getNamespace() {
if (labels.containsKey(MonitoringInfoConstants.Labels.NAMESPACE)) {
+ // User-generated metric
return labels.getOrDefault(MonitoringInfoConstants.Labels.NAMESPACE,
null);
+ } else if (labels.containsKey(MonitoringInfoConstants.Labels.PCOLLECTION))
{
+ // System-generated metric, prepend with a colon
+ return ":" +
labels.getOrDefault(MonitoringInfoConstants.Labels.PCOLLECTION, null);
+ } else if (labels.containsKey(MonitoringInfoConstants.Labels.PTRANSFORM)) {
+ // System-generated metric, prepend with a colon
+ return ":" +
labels.getOrDefault(MonitoringInfoConstants.Labels.PTRANSFORM, null);
Review comment:
Agreed, there should be no need for this prefix. I'm going to remove it. I
wouldn't be surprised though if we revisit the metric names as we test this
with a real world metric backend. The current names are quite verbose. Examples:
`36read/Read/Reshuffle/RemoveRandomKeys.None/beam:env:docker:v1:0:beam:metric:user_distribution
{NAMESPACE=__main__.WordExtractingDoFn,
PTRANSFORM=ref_AppliedPTransform_split_17, NAME=word_len_dist}:
DistributionResult{sum=114, count=29, min=0, max=11}`
(user metric created as `self.word_lengths_dist =
Metrics.distribution(self.__class__, 'word_len_dist')`
or
`17read/Read/Impulse.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_1}: DistributionResult{sum=13,
count=1, min=13, max=13},
6format.None/beam:env:docker:v1:0:beam:metric:sampled_byte_size:v1
{PCOLLECTION=ref_PCollection_PCollection_17}: DistributionResult{sum=395,
count=18, min=19, max=29}`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 325428)
Time Spent: 3h 20m (was: 3h 10m)
> Duplicate metric warnings clutter log
> -------------------------------------
>
> Key: BEAM-6829
> URL: https://issues.apache.org/jira/browse/BEAM-6829
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.11.0
> Reporter: Thomas Weise
> Assignee: Maximilian Michels
> Priority: Major
> Labels: portability
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> Logs fill up quickly with these warnings:
> {code:java}
> WARN org.apache.flink.metrics.MetricGroup - Name collision: Group already
> contains a Metric with the name ...{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
