[
https://issues.apache.org/jira/browse/FLINK-28606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yunfeng Zhou updated FLINK-28606:
---------------------------------
Description:
This is a component of our solution to the consistency issue in the operator
coordinator mechanism. In this step, we would guarantee the consistency of all
communications in one direction, from OC to subtasks. This would need less
workload and should unblock the implementation of the CEP coordinator in
FLIP-200.
Roughly, we would need to implement the following process in this step.
# Let the OC finish processing all the incoming OperatorEvents before the
snapshot.
# Closes the gateway that sends operator events to its subtasks when the OC
completes the snapshot.
# Wait until all the outgoing OperatorEvents created before the snapshot are
sent and acked.
# Send checkpoint barriers to the Source operators.
# Open the corresponding gateway of a subtask when the OC learned that the
subtask has completed the checkpoint.
was:
This is a component of our solution to the consistency issue in the operator
coordinator mechanism. In this step, we would guarantee the consistency of all
communications in one direction, from OC to subtasks. This would need less
workload and should unblock the implementation of the CEP coordinator in
FLIP-200.
Roughly, we would need to implement the following process in this step.
#
Let the OC finish processing all the incoming OperatorEvents before the
snapshot.
#
Closes the gateway that sends operator events to its subtasks when the OC
completes snapshot.
#
Wait until all the outgoing OperatorEvents before the snapshot are sent and
acked.
#
Send checkpoint barriers to the Source operators.
#
Open the corresponding gateway of a subtask when the OC learned that the
subtask has completed the checkpoint.
> Preserve distributed consistency of OperatorEvents from OperatorCoordinator
> to subtasks
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-28606
> URL: https://issues.apache.org/jira/browse/FLINK-28606
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing
> Affects Versions: 1.14.3
> Reporter: Yunfeng Zhou
> Priority: Major
> Fix For: 1.16.0
>
>
> This is a component of our solution to the consistency issue in the operator
> coordinator mechanism. In this step, we would guarantee the consistency of
> all communications in one direction, from OC to subtasks. This would need
> less workload and should unblock the implementation of the CEP coordinator in
> FLIP-200.
> Roughly, we would need to implement the following process in this step.
> # Let the OC finish processing all the incoming OperatorEvents before the
> snapshot.
> # Closes the gateway that sends operator events to its subtasks when the OC
> completes the snapshot.
> # Wait until all the outgoing OperatorEvents created before the snapshot are
> sent and acked.
> # Send checkpoint barriers to the Source operators.
> # Open the corresponding gateway of a subtask when the OC learned that the
> subtask has completed the checkpoint.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
