kezhuw edited a comment on issue #2033: URL: https://github.com/apache/iceberg/issues/2033#issuecomment-766096422
@sunhaibotb @zhijiangW @rkhachatryan @pnowojski @kl0u @zentol @aljoscha I think it is a bug of Flink and was introduced in [FLINK-14228](https://issues.apache.org/jira/browse/FLINK-14228)(apache/flink#9854, apache/flink#10151). The changes for [FLINK-14228](https://issues.apache.org/jira/browse/FLINK-14228) relay on `StreamTask.afterInvoke` and `OperatorChain.closeOperators` will only be invoked after **end of input**. But that is not true after [FLIP-34: Terminate/Suspend Job with Savepoint ](https://cwiki.apache.org/confluence/x/JAglBg) long before. Task could enter a **finished** state after synchronous savepoint, that is an expected job suspension and stopping. I am also aware of [FLIP-147: Support Checkpoints After Tasks Finished](https://cwiki.apache.org/confluence/x/mw-ZCQ), I think maybe the three should make agreement on terminologies. Besides, I think FLIP-27 Source does not work with synchronous savepoint. @becketqin @StephanEwen I have pushed branch [synchronous-savepoint-conflict-with-bounded-end-input-case](https://github.com/kezhuw/flink/commits/synchronous-savepoint-conflict-with-bounded-end-input-case) in my local repository. Two test cases failed both: * `SavepointITCase.testStopSavepointWithBoundedInput` failed due to `BoundedOneInput.endInput` called. * `SavepointITCase.testStopSavepointWithFlip27Source` failed due to timeout. FLINK-14228: https://issues.apache.org/jira/browse/FLINK-14228 FLIP-34: https://cwiki.apache.org/confluence/x/JAglBg FLIP-147: https://cwiki.apache.org/confluence/x/mw-ZCQ `StreamTask.finishTask`: https://github.com/apache/flink/blob/d20867a58d8f9ba7520d611d5fab01e6f98253a5/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/StreamTask.java#L467 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
