Re: Kafka Metrics

2021-08-23 Thread Arvid Heise
(like most Flink-provided job/task/operator metrics): - "I don't want the KafkaConsumer metrics, except that exact once" (so also an include list?), or - "I only want Kafka metrics from the JM" (rules depending on the scope?), or - "I don't care about perSecond

Re: Kafka Metrics

2021-08-23 Thread Mason Chen
metric has a name equal to the specified metricName. Configurations are specified as follows `groupNameKey1:metricName1;groupNameKey2:metricName2`. Thus, I can deny list KafkaConsumer (group name key) and committed_offsets (metric name) which correspond to the legacy kafka metrics. Would Flink

Re: Kafka Metrics

2021-08-23 Thread Arvid Heise
Mason Chen wrote: > FYI, I'm referring to the legacy offsets metric gauges. > > On Thu, Aug 19, 2021 at 4:53 PM Mason Chen wrote: > >> Hi all, >> >> We have found that the per partition Kafka metrics contributes to a lot >> of metrics being indexed by our me

Re: Kafka Metrics

2021-08-19 Thread Mason Chen
FYI, I'm referring to the legacy offsets metric gauges. On Thu, Aug 19, 2021 at 4:53 PM Mason Chen wrote: > Hi all, > > We have found that the per partition Kafka metrics contributes to a lot of > metrics being indexed by our metrics system. > > We would still like to

Kafka Metrics

2021-08-19 Thread Mason Chen
Hi all, We have found that the per partition Kafka metrics contributes to a lot of metrics being indexed by our metrics system. We would still like to have the proxied kafka metrics from the kafka clients library. Is there a flag to only exclude Flink's additional Kafka metrics? Best, Mason