sjvanrossum commented on code in PR #36935:
URL: https://github.com/apache/beam/pull/36935#discussion_r2572134546
##########
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java:
##########
@@ -108,6 +109,12 @@
*
* <h4>Splitting</h4>
*
+ * <p>Consumer groups must not consume from the same {@link TopicPartition}
simultaneously. Doing so
+ * may arbitrarily overwrite a consumer group's committed offset for a {@link
TopicPartition}.
Review Comment:
I'd prefer that over `UnsplittableRestrictionTracker`, but we'd still need
to prevent splitting when `enable.auto.commit` is set. If runners split too
aggressively during processing then there's the risk of overwhelming a broker
with connections, but I'm not too concerned about that.
I've asked the release manager to weigh in on whether they want to
cherry-pick this change into 2.70.0 since it would unblock a user. If not, I'll
add those changes to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
