[
https://issues.apache.org/jira/browse/FLINK-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950805#comment-16950805
]
Piotr Nowojski commented on FLINK-14344:
----------------------------------------
> I'm afraid there might be a risk of executing a blocking operation in main
> thread. So I try to execute it in IO thread, but there is another risk of
> deadlock.
Letting blocking user code executing in the IO thread might be also a dangerous
idea.
I think ideally I would prefer to make a contract that sync master hooks should
be non blocking executed in the main thread. Async hooks could also be executed
by the main thread and user should take care of spawning/re-using his own
thread to actually execute the async work (just as in {{AsyncWaitOperator}}).
Doing it in another way might be difficult, especially if we want to keep the
semantic, that the sync hooks are triggered and executed before any other
follow up actions (like triggering the actual checkpoint). If we execute sync
hook in any other thread beside the main thread, we would have to make sure
that none of the {{CheckpointCoordinator}} actions will be triggered until that
other thread finish its work.
> Snapshot master hook state asynchronously
> -----------------------------------------
>
> Key: FLINK-14344
> URL: https://issues.apache.org/jira/browse/FLINK-14344
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing
> Reporter: Biao Liu
> Assignee: Biao Liu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.10.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently we snapshot the master hook state synchronously. As a part of
> reworking threading model of {{CheckpointCoordinator}}, we have to make this
> non-blocking to satisfy the requirement of running in main thread.
> The behavior of snapshotting master hook state is similar to task state
> snapshotting. Master state snapshotting is taken before task state
> snapshotting. Because in master hook, there might be external system
> initialization which task state snapshotting might depend on.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
