[
https://issues.apache.org/jira/browse/HUDI-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yuzhaojing updated HUDI-1923:
-----------------------------
Description:
In flink, notifyCheckpointComplete not in checkpoint life cycle. If a
checkpoint is success and commit action in notifyCheckpointComplete is failed,
when we restore from the latest checkpoint, the element belong this instant
will be discard.
So, we should store commit state and restore it when flink restart.
was:if coordinator notifyCheckpointComplete funtion execute failed, when we
restore from the latest checkpoint, the element belong this instant will be
discard.
> Add state in StreamWriteFunction to restore
> -------------------------------------------
>
> Key: HUDI-1923
> URL: https://issues.apache.org/jira/browse/HUDI-1923
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Flink Integration
> Reporter: yuzhaojing
> Assignee: yuzhaojing
> Priority: Major
>
> In flink, notifyCheckpointComplete not in checkpoint life cycle. If a
> checkpoint is success and commit action in notifyCheckpointComplete is
> failed, when we restore from the latest checkpoint, the element belong this
> instant will be discard.
> So, we should store commit state and restore it when flink restart.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
