lindong28 commented on a change in pull request #15161:
URL: https://github.com/apache/flink/pull/15161#discussion_r600205574
##########
File path:
flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/source/KafkaSourceBuilder.java
##########
@@ -435,6 +435,7 @@ private void parseAndSetRequiredProperties() {
true);
maybeOverride(
ConsumerConfig.GROUP_ID_CONFIG, "KafkaSource-" + new
Random().nextLong(), false);
+ maybeOverride(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false",
false);
Review comment:
Hmm... from what I can tell, old Kafka Source (i.e.
`FlinkKafkaConsumer`) actually allows `enable.auto.commit` to be set to `true`.
This is because the `FlinkKafkaConsumer`
[constructor](https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer.java#L210)
allows users to specify `enable.auto.commit` in `Properties` which is used to
instantiated Kafka consumer.
And it is mentioned in
[here](https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer.java#L251)
that we will overwrite the "enable.auto.commit" in the properties if our
offset commit mode is ON_CHECKPOINTS.
From what I learned from @becketqin, the use-case here is that, by enabling
auto offset commit (when the Flink job does not do checkpoint), we allow
third-party Kafka tools to track the progress of the Kafka consumer.
Does this make sense?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
