[
https://issues.apache.org/jira/browse/FLINK-21030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17278088#comment-17278088
]
Matthias commented on FLINK-21030:
----------------------------------
{quote}1. Taking a savepoint
2. Stopping the job gracefully (waiting that the notify checkpoint complete
messages are sent and the Tasks shut themselves down)
{quote}
We've been implementing two test cases to cover the use-case. Each of them
covers a failure in one of the stages [~trohrmann] described above. While doing
this, I ran into some timeout issue when the failure happened during phase 1
(savepoint creation). The failed savepoint does not leave the pending state
even though the snapshot creation failed. It looks like there's an
{{AsyncCheckpointRunnable}} (or something similar) missing that informs the
{{CheckpointCoordinator}} about the failed savepoint as part of the exception
handling in
[SubtaskCheckpointCoordinatorImpl.checkpointState|https://github.com/XComp/flink/blob/f421c6f72b91b7897ff2246bdf4f408e3609f6f0/flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/SubtaskCheckpointCoordinatorImpl.java#L305].
In the test, the checkpoint gets cleaned up during cluster shutdown. [~AHeise]
Shall I create a Jira issue for that or do I miss something here?
> Broken job restart for job with disjoint graph
> ----------------------------------------------
>
> Key: FLINK-21030
> URL: https://issues.apache.org/jira/browse/FLINK-21030
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.11.2
> Reporter: Theo Diefenthal
> Assignee: Matthias
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.11.4, 1.12.2, 1.13.0
>
>
> Building on top of bugs:
> https://issues.apache.org/jira/browse/FLINK-21028
> and https://issues.apache.org/jira/browse/FLINK-21029 :
> I tried to stop a Flink application on YARN via savepoint which didn't
> succeed due to a possible bug/racecondition in shutdown (Bug 21028). Due to
> some reason, Flink attempted to restart the pipeline after the failure in
> shutdown (21029). The bug here:
> As I mentioned: My jobgraph is disjoint and the pipelines are fully isolated.
> Lets say the original error occured in a single task of pipeline1. Flink then
> restarted the entire pipeline1, but pipeline2 was shutdown successfully and
> switched the state to FINISHED.
> My job thus was in kind of an invalid state after the attempt to stopping:
> One of two pipelines was running, the other was FINISHED. I guess this is
> kind of a bug in the restarting behavior that only all connected components
> of a graph are restarted, but the others aren't...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
