[
https://issues.apache.org/jira/browse/FLINK-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16089358#comment-16089358
]
ASF GitHub Bot commented on FLINK-6998:
---------------------------------------
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/4187#discussion_r127636725
--- Diff:
flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumerBase.java
---
@@ -505,6 +519,21 @@ public void run(SourceContext<T> sourceContext) throws
Exception {
throw new Exception("The partitions were not set for
the consumer");
}
+ // initialize commit metrics and default offset callback method
+ this.successfulCommits =
this.getRuntimeContext().getMetricGroup().counter("commitsSucceeded");
+ this.failedCommits =
this.getRuntimeContext().getMetricGroup().counter("commitsFailed");
+
+ this.offsetCommitCallback = new KafkaCommitCallback() {
+ @Override
+ public void onComplete(Exception exception) {
+ if (exception == null) {
+ successfulCommits.inc();
--- End diff --
I would also like to raise a thread-safety issue here.
Currently, since there's always only one pending offset commit in Kafka
09+, and Kafka08 commits in a blocking call, there will be no race condition in
incrementing the counters. However, changing these implementations in
subclasses (perhaps in the future) can easily introduce race conditions here.
At the very least, we probably should add a notice about thread-safety
contract in the Javadoc of `commitInternalOffsetsToKafka`.
> Kafka connector needs to expose metrics for failed/successful offset commits
> in the Kafka Consumer callback
> -----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-6998
> URL: https://issues.apache.org/jira/browse/FLINK-6998
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Reporter: Zhenzhong Xu
> Assignee: Zhenzhong Xu
>
> Propose to add "kafkaCommitsSucceeded" and "kafkaCommitsFailed" metrics in
> KafkaConsumerThread class.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
