[
https://issues.apache.org/jira/browse/FLINK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17499395#comment-17499395
]
Yang Wang commented on FLINK-26370:
-----------------------------------
[~kelemensanyi] You are right. The problem is calls to the flink clusters will
block operator thread.
I think you have given a thorough analysis of all the solution. The option #3
seems to be right direction to go. But I am afraid it will make the reconcile
too complicated. I am curious whether we could simply move out the blocking
calls(accessing the flink clusters via rest client) to a separate thread pool,
which periodically get the job status from flink cluster and cache them locally.
> Make Flink cluster communication asynchronous
> ---------------------------------------------
>
> Key: FLINK-26370
> URL: https://issues.apache.org/jira/browse/FLINK-26370
> Project: Flink
> Issue Type: Sub-task
> Components: Kubernetes Operator
> Reporter: Gyula Fora
> Assignee: Sandor Kelemen
> Priority: Major
>
> In the current architecture calls to the flink clusters (through the rest
> client) are made synchronously from the reconcile loop.
> These calls often take a long time due to various (compeltely normal) reasons:
> - Cluster is not ready -> long call + timeoutexception
> - Operation takes a long time -> cancel/savepoint operations are often
> expected to take seconds/minutes
> Both the observer and reconciler components make these calls.
> We should come up with a way to avoid making these sync calls from the main
> loop while still preserving the logic of the operator.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
