[
https://issues.apache.org/jira/browse/FLINK-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222972#comment-16222972
]
Elias Levy commented on FLINK-7935:
-----------------------------------
Possibly. It depends on whether you could add multiple metrics or metric
groups that differ in their variables, but that could be formatted the same.
E.g. the TaskManagerJobMetricGroup creates and tracks distinct TaskMetricGroup
for each task in a the portion of a job the task manager is executing. The
metrics for each task are tracked separately, but I can format the scope so all
of them are reported with the same name ("taskmanager.job.task") but with
different variables/DD tags.
> Metrics with user supplied scope variables
> ------------------------------------------
>
> Key: FLINK-7935
> URL: https://issues.apache.org/jira/browse/FLINK-7935
> Project: Flink
> Issue Type: Improvement
> Affects Versions: 1.3.2
> Reporter: Elias Levy
>
> We use DataDog for metrics. DD and Flink differ somewhat in how they track
> metrics.
> Flink names and scopes metrics together, at least by default. E.g. by default
> the System scope for operator metrics is
> {{<host>.taskmanager.<tm_id>.<job_name>.<operator_name>.<subtask_index>}}.
> The scope variables become part of the metric's full name.
> In DD the metric would be named something generic, e.g.
> {{taskmanager.job.operator}}, and they would be distinguished by their tag
> values, e.g. {{tm_id=foo}}, {{job_name=var}}, {{operator_name=baz}}.
> Flink allows you to configure the format string for system scopes, so it is
> possible to set the operator scope format to {{taskmanager.job.operator}}.
> We do this for all scopes:
> {code}
> metrics.scope.jm: jobmanager
> metrics.scope.jm.job: jobmanager.job
> metrics.scope.tm: taskmanager
> metrics.scope.tm.job: taskmanager.job
> metrics.scope.task: taskmanager.job.task
> metrics.scope.operator: taskmanager.job.operator
> {code}
> This seems to work. The DataDog Flink metric's plugin submits all scope
> variables as tags, even if they are not used within the scope format. And it
> appears internally this does not lead to metrics conflicting with each other.
> We would like to extend this to user defined metrics, but you can define
> variables/scopes when adding a metric group or metric with the user API, so
> that in DD we have a single metric with a tag with many different values,
> rather than hundreds of metrics to just the one value we want to measure
> across different event types.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
