scwhittle commented on code in PR #36935:
URL: https://github.com/apache/beam/pull/36935#discussion_r2576562216
##########
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java:
##########
@@ -108,6 +109,14 @@
*
* <h4>Splitting</h4>
*
+ * <p>Consumer group members must not consume from the same {@link
TopicPartition} simultaneously
+ * when {@code enable.auto.commit} is set. Doing so may arbitrarily overwrite
a consumer group's
+ * committed offset for a {@link TopicPartition}. Restriction trackers for a
{@link
+ * KafkaSourceDescriptor} are wrapped as {@link
UnsplittableRestrictionTracker<OffsetRange, Long>}
+ * and will only return a non-null {@link
org.apache.beam.sdk.transforms.splittabledofn.SplitResult}
+ * for a checkpoint. This ensures consistent behavior when {@code
enable.auto.commit} is set and
Review Comment:
nit: "ensures consistent" is too strong I think? There may still be
parallel scheduling on different VMs during scaling etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
