boyuanzz commented on a change in pull request #13710:
URL: https://github.com/apache/beam/pull/13710#discussion_r561206125
##########
File path:
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java
##########
@@ -1402,6 +1407,15 @@ public void populateDisplayData(DisplayData.Builder
builder) {
return toBuilder().setConsumerFactoryFn(consumerFactoryFn).build();
}
+ /**
+ * A custom {@link SerializableFunction} that determines whether the
{@link ReadFromKafkaDoFn}
+ * should stop reading from the given {@link TopicPartition}.
+ */
+ public ReadSourceDescriptors<K, V> withCheckStopReadingFn(
Review comment:
I added more related javadoc around `KafkaIO`.
Thinking about the commit transform again. I don't think additional changes
required here because:
* For stopped TopicPartition, we still want to commit the offset
* For deleted TopicPartition, we should get exceptions from Kafka Consumer
API, which has been handled in the Commit transform.
I'll merge this PR if all these look good to you : )
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
