PatrickRen commented on a change in pull request #18790:
URL: https://github.com/apache/flink/pull/18790#discussion_r809646789
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/SourceOperator.java
##########
@@ -423,6 +422,16 @@ private DataInputStatus emitNextNotReading(DataOutput<OUT>
output) throws Except
}
}
+ private void initializeMainOutput(DataOutput<OUT> output) {
+ currentMainOutput = eventTimeLogic.createMainOutput(output,
this::onWatermarkEmitted);
+ initializeLatencyMarkerEmitter(output);
+ lastInvokedOutput = output;
+ outputPendingSplits.forEach(
+ split ->
currentMainOutput.createOutputForSplit(split.splitId()));
Review comment:
The output will be released only when the split is finished, so I think
the logic is OK here, just advancing the output creation to the moment of split
addition.
One concern in my mind is about adding an invalid splits to source reader.
For example a split whose starting offset is ahead of ending offset, is added
to the source reader, and the implementation of SplitReader just "swallows"
this invalid split instead of marking it as finished, there will be an output
leak.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
