[
https://issues.apache.org/jira/browse/KAFKA-12469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17898691#comment-17898691
]
Apoorv Mittal commented on KAFKA-12469:
---------------------------------------
Hi [~junrao], I went through the threads you mentioned again and here is my
understanding.
A metric can be of type Yammer or not, in Kafka. However, kafka-clients cannot
have Yammer metrics.
Yammer metrics uses four fields: group, type, name and scope, leaving the tags
field. KafkaMetricsGroup class creates the scope field by filling from tags
field and replacing "." by "_".
The Mbean is created without any modification to tags name, no use of scope.
And JMX and Telemetry Reporter uses Mbean name to output metrics.
Problem:
For certain kafka-consumer topic metrics there happens replacement of "." to
"_" outside of KafkaMetricsGroup.
Solution:
The transformation should not be done for some, either for all or none. I would
say it should be for none outside KafkaMetricsGroup.
To support graphite, we should continue filling the scope field as we today.
This shall give uniform behaviour i.e. if reading by Graphite reporter then all
topic metrics will have "." replaced, else if reading by JMX or Telemetry
reporter then all will have topic as it is (i.e. maintain uniform behaviour for
tags and scope field across different metrics).
As far as I see [Graphite
documentation|https://graphite.readthedocs.io/en/latest/render_api.html#paths-and-wildcards],
I think they still use "." hierarchy model. But I am not sure how old this
documentation is. Coming to Ganglia, the jars are now based on dropwizard
version of metrics-core as com.yammer was last updated in 2012 and then moved
to dropwizard.
I do think we should be moving away from Yammer, which something is a different
discussion and I ll put my thoughts in a different KIP. But for this ticket I
think the above mentioned solution should be implemented unless I am missing
something.
> The topic names in the metrics do not retain their format when extracting
> through JMX.
> --------------------------------------------------------------------------------------
>
> Key: KAFKA-12469
> URL: https://issues.apache.org/jira/browse/KAFKA-12469
> Project: Kafka
> Issue Type: Bug
> Components: consumer, metrics
> Reporter: Rafał Chmielewski
> Assignee: Apoorv Mittal
> Priority: Major
> Attachments: JConsole - Kafka Client.png, Screenshot 2024-11-12 at
> 21.35.59.png, image-2024-11-12-21-32-55-180.png
>
>
> I have topic names that have a period in the name:
> product.order
> product.offering.price
>
> However, for the metrics issued by JMX by a program that is a consumer of
> Kafka messages, the dots are replaced with an underscore:
> kafka.consumer<type=consumer-fetch-manager-metrics,
> client-id=consumer-export-4, topic=product_offering_price,
> partition=1><>records-lead
>
> This creates a problem if I want to calculate the customer's lag in relation
> to the number of messages on Kafka.
>
> But for the producer, this problem doesn't occur:
> kafka.producer<type=producer-topic-metrics,
> client-id=bss.data.verification.pi_1,
> topic=product.offering.price><>record-send-total
>
> As a consumer I use Akka Alpakka. But I think it's using Apache library to
> connect to Kafka and report metrics via JMX.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
