junaiddshaukat commented on code in PR #38987:
URL: https://github.com/apache/beam/pull/38987#discussion_r3435256853
##########
runners/kafka-streams/src/main/java/org/apache/beam/runners/kafka/streams/translation/ExecutableStageProcessor.java:
##########
@@ -87,22 +98,43 @@ class ExecutableStageProcessor
@Override
public void init(ProcessorContext<byte[], KStreamsPayload<?>> context) {
this.context = context;
- ExecutableStage executableStage =
ExecutableStage.fromPayload(stagePayload);
- this.stageContext =
KafkaStreamsExecutableStageContextFactory.getInstance().get(jobInfo);
- this.stageBundleFactory =
stageContext.getStageBundleFactory(executableStage);
+ // The SDK harness (stage context + bundle factory) is created lazily on
the first data
+ // element, so a stage that only forwards watermarks never spins one up.
This mirrors Spark's
+ // SparkExecutableStageFunction, which likewise does not build a bundle
factory when there are
+ // no inputs to process.
+ }
+
+ private StageBundleFactory ensureStageBundleFactory() {
+ StageBundleFactory factory = stageBundleFactory;
+ if (factory == null) {
+ ExecutableStage executableStage =
ExecutableStage.fromPayload(stagePayload);
+ ExecutableStageContext sc =
+ KafkaStreamsExecutableStageContextFactory.getInstance().get(jobInfo);
+ this.stageContext = sc;
+ factory = sc.getStageBundleFactory(executableStage);
+ this.stageBundleFactory = factory;
+ }
+ return factory;
}
@Override
public void process(Record<byte[], KStreamsPayload<?>> record) {
KStreamsPayload<?> payload = record.value();
if (payload.isWatermark()) {
Review Comment:
Safe to leave — record.value() can't be null here. An ExecutableStage only
ever receives non-null KStreamsPayloads forwarded in-JVM by Impulse or an
upstream stage; no source/repartition topic feeds a stage, so there's no
tombstone/null path. Will add a guard with the topic + serde work, where
deserialization makes null actually possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
