Re: Flink Metrics Naming

2021-06-01 Thread Chesnay Schepler
Some more background on MetricGroups: Internally there (mostly) 3 types of metric groups: On the one hand we have the ComponentMetricGroups (like TaskManagerMetricGroup) that describe a high-level Flink entity, which just add a constant expression to the logical scope(like taskmanager, task

Re: Flink Metrics Naming

2021-06-01 Thread Mason Chen
Upon further inspection, it seems like the user scope is not universal (i.e. comes through the connectors and not UDFs (like rich map function)), but the question still stands if the process makes sense. > On Jun 1, 2021, at 10:38 AM, Mason Chen wrote: > > Makes sense. We are primarily

Re: Flink Metrics Naming

2021-06-01 Thread Mason Chen
Makes sense. We are primarily concerned with removing the metric labels from the names as the user metrics get too long. i.e. the groups from `addGroup` are concatenated in the metric name. Do you think there would be any issues with removing the group information in the metric name and

Re: Flink Metrics Naming

2021-06-01 Thread Chesnay Schepler
The uniqueness of metrics and the naming of the Prometheus reporter are somewhat related but also somewhat orthogonal. Prometheus works similar to JMX in that the metric name (e.g., taskmanager.job.task.operator.numRecordsIn) is more or less a _class_ of metrics, with tags/labels allowing you

Re: Flink Metrics Naming

2021-06-01 Thread Till Rohrmann
Hi Mason, The idea is that a metric is not uniquely identified by its name alone but instead by its path. The groups in which it is defined specify this path (similar to directories). That's why it is valid to specify two metrics with the same name if they reside in different groups. I think

Flink Metrics Naming

2021-05-28 Thread Mason Chen
Can anyone give insight as to why Flink allows 2 metrics with the same “name”? For example, getRuntimeContext.addGroup(“group”, “group1”).counter(“myMetricName”); And getRuntimeContext.addGroup(“other_group”, “other_group1”).counter(“myMetricName”); Are totally valid. It seems that it has