[
https://issues.apache.org/jira/browse/FLINK-30238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697241#comment-17697241
]
Yun Gao commented on FLINK-30238:
---------------------------------
Hi [~fpaul] sorry for the long delay for it took me some time to do the double
confirmation:
# Currently endOfInput() should only be called on there is no new data or
stop-with-savepoint --drain. For these two cases, the job is logically
"terminated", namely they should not be restarted again:
https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/cli/#stopping-a-job-gracefully-creating-a-final-savepoint
# For normal savepoint or stop-with-savepoint without draining, the
endOfInput() will not be called, and the job is logically "suspended" and could
be restarted from the savepoint again.
# For this specific Kafka test case, I think it should not use
stopWithSavepoint with advancedTimestamp = true (which is in fact translated to
stop-with-savepont --drain finally), since it tries to create a savepoint that
the job could be restarted from.
Thus for the real-world case, if a job has "endOfInput()" called, it is
expected to be not restarted any more.
> Unified Sink committer does not clean up state on final savepoint
> -----------------------------------------------------------------
>
> Key: FLINK-30238
> URL: https://issues.apache.org/jira/browse/FLINK-30238
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Common
> Affects Versions: 1.17.0, 1.15.3, 1.16.1
> Reporter: Fabian Paul
> Priority: Critical
>
> During stop-with-savepoint the committer only commits the pending
> committables on notifyCheckpointComplete.
> This has several downsides.
> * Last committableSummary has checkpoint id LONG.MAX and is never cleared
> from the state leading to that stop-with-savepoint does not work when the
> pipeline recovers from a savepoint
> * While the committables are committed during stop-with-savepoint they are
> not forwarded to post-commit topology, potentially losing data and preventing
> to close open transactions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
