[
https://issues.apache.org/jira/browse/KAFKA-10441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185897#comment-17185897
]
Pradyumna edited comment on KAFKA-10441 at 8/27/20, 2:56 PM:
-------------------------------------------------------------
Interestingly, the MBean is available for these metrics, and it is reporting
correct numbers, but it is not available via the configured metric.reporters
jmxterm output:
$>get -b
kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,topic=kptest1,type=consumer-fetch-manager-metrics
*
#mbean =
kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,topic=kptest1,type=consumer-fetch-manager-metrics:
bytes-consumed-total = 167.0;
bytes-consumed-rate = 0.0;
records-consumed-rate = 0.0;
fetch-size-max = NaN;
records-consumed-total = 1.0;
fetch-size-avg = NaN;
records-per-request-avg = NaN;
$>get -b
kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,partition=0,topic=kptest1,type=consumer-fetch-manager-metrics
*
#mbean =
kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,partition=0,topic=kptest1,type=consumer-fetch-manager-metrics:
records-lag-max = NaN;
records-lag = 0.0;
records-lag-avg = NaN;
records-lead-min = NaN;
records-lead-avg = NaN;
records-lead = 1.0;
$>
was (Author: pradyumna):
Interestingly, the MBean is available for these metrics, and it is reporting
correct numbers, but it is not available via the configured metric.reporters
jmxterm output:
$>get -b
kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,topic=kptest1,type=consumer-fetch-manager-metrics
*
#mbean =
kafka.consumer:client-id=connector-consumer-s3connector-kptest1-0,topic=kptest1,type=consumer-fetch-manager-metrics:
bytes-consumed-total = 167.0;
bytes-consumed-rate = 0.0;
records-consumed-rate = 0.0;
fetch-size-max = NaN;
records-consumed-total = 1.0;
fetch-size-avg = NaN;
records-per-request-avg = NaN;
$>
> Connect consumers are missing the fetch metrics
> -----------------------------------------------
>
> Key: KAFKA-10441
> URL: https://issues.apache.org/jira/browse/KAFKA-10441
> Project: Kafka
> Issue Type: Bug
> Components: consumer, KafkaConnect
> Affects Versions: 2.3.0
> Reporter: Pradyumna
> Priority: Minor
>
> Kafka connect workers create KafkaConsumers and consume from Kafka topics.
>
> However, unlike regular KafkaConsumers which have a good set of metrics in
> the group consumer-fetch-manager-metrics (such as records-lag,
> records-consumed-total etc), these KafkaConsumers created by Kafka Connect do
> not have any of these consumer metrics.
>
> I took a heap dump and saw that the "sensors" field in the metrics field of
> the KafkaConsumer objects created by Kafka Connect had only the connect
> metrics, and not the consumer fetch metrics.
>
> I tried briefly to find out where in the Kafka code is the metrics object
> being reset or the sensors field of the metrics object overridden by kafka
> connect, but I could not locate anything yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
