Abacn commented on code in PR #32170:
URL: https://github.com/apache/beam/pull/32170#discussion_r1722542827
##########
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaWriter.java:
##########
@@ -91,6 +93,19 @@ public void processElement(ProcessContext ctx,
MultiOutputReceiver receiver) thr
callback);
elementsWritten.inc();
+ if (!topicName.equals(reportedLineage)) {
Review Comment:
Yes it is possible multiple topics appears in a KafkaWriter. This check
reduces some overhead when there is single topic (e.g. set by spec), or there
was a GBK upstream that led elements in same topic grouped together and will be
processed together. This is the same pattern I used for PubSubIO sink Lineage
in #32037 (See diff of PubsubUnboundedSink.java)
> If we expect more than one are they guaranteed to be appear in together?
not necessary, depend on whether/how elements were grouped upstream, see
above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
