[
https://issues.apache.org/jira/browse/FLINK-29270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qingsheng Ren closed FLINK-29270.
---------------------------------
Resolution: Won't Fix
> Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE
> ----------------------------------------------------------------------------
>
> Key: FLINK-29270
> URL: https://issues.apache.org/jira/browse/FLINK-29270
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Runtime / Metrics
> Affects Versions: 1.14.4
> Reporter: Hilmi Al Fatih
> Priority: Minor
> Attachments: image-2022-09-13-11-41-51-759.png,
> image-2022-09-13-11-42-45-297.png, image-2022-09-13-11-43-38-226.png,
> image-2022-09-13-11-43-50-441.png
>
>
> I did a very simple kafka-to-kafka pipeline with flink 1.14.4 with checkpoint.
> I did 4 tests to confirm: for each old API
> (FlinkKafkaConsumer&FlinkKafkaProducer) and new API (KafkaSource&KafkaSink),
> I run AT_LEAST_ONCE and EXACLTY_ONCE. The only difference in the code is in
> the settings of Semantic part.
> However, the metrics shown by producer is not correct when doing EXACTLY_ONCE
> semantics in both APIs.
> * For FlinkKafkaProducer, the metrics seems to be restarted every checkpoint
> * For KafkaSink, the rate shows half of the actual produced rate.
> For at_least_once, they are all agree with the consuming rate.
> To understand the situation, I also check the incoming rate metrics for both
> source and sink topics. So here are the summary:
> h3. Old API
> h4. At Least Once
> !image-2022-09-13-11-41-51-759.png|width=616,height=190!
> h4. Exactly Once
> !image-2022-09-13-11-42-45-297.png|width=616,height=192!
> h3. New API (KafkaSource&KafkaSink)
> h4. At Least Once
> !image-2022-09-13-11-43-38-226.png|width=618,height=190!
> h4. Exactly Once
> !image-2022-09-13-11-43-50-441.png|width=620,height=189!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
