[
https://issues.apache.org/jira/browse/FLINK-13698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17333089#comment-17333089
]
Stephan Ewen edited comment on FLINK-13698 at 4/27/21, 10:06 AM:
-----------------------------------------------------------------
I think it is not this ticket that we need for the Scheduler stability
problems, or at least not what is described here.
We need to change the {{CompletedCheckpointStore}} to do asynchronous loading
of checkpoint metadata. So that the scheduler thread is not blocking on I/O
when it accesses the {{CompletedCheckpointStore}}.
That is an independent issue from this ticket, which deals mostly with the
checkpoint triggering and finalization.
was (Author: stephanewen):
I think it is not this ticket that we need, or at least not what is described
here.
We need to change the {{CompletedCheckpointStore}} to do asynchronous loading
of checkpoint metadata. That is an independent issue from this ticket.
> Rework threading model of CheckpointCoordinator
> -----------------------------------------------
>
> Key: FLINK-13698
> URL: https://issues.apache.org/jira/browse/FLINK-13698
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing
> Affects Versions: 1.10.0
> Reporter: Piotr Nowojski
> Priority: Critical
> Fix For: 1.14.0
>
>
> Currently {{CheckpointCoordinator}} and {{CheckpointFailureManager}} code is
> executed by multiple different threads (mostly {{ioExecutor}}, but not only).
> It's causing multiple concurrency issues, for example:
> https://issues.apache.org/jira/browse/FLINK-13497
> Proper fix would be to rethink threading model there. At first glance it
> doesn't seem that this code should be multi threaded, except of parts doing
> the actual IO operations, so it should be possible to run everything in one
> single ExecutionGraph's thread and just run asynchronously necessary IO
> operations with some feedback loop ("mailbox style").
> I would strongly recommend fixing this issue before adding new features in
> the \{{CheckpointCoordinator}} component.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
