[
https://issues.apache.org/jira/browse/FLINK-25256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17505212#comment-17505212
]
Brian Zhou commented on FLINK-25256:
------------------------------------
Hi [~dwysakowicz] , I want to ask what the influence is of this issue, for
example, what kind of application or state backend settings is not able to
recover correctly.
We have internally tried to reproduce this issue for some basic applications.
We've found that when savepointing, all of the states are stored in the
metadata, which makes the job to recover correctly. We have tried both stateful
and stateless apps, both filesystem and rocksdb backend, and always seeing the
metadata file in the savepoint directory.
I understand from the code, it does look incorrect in checkpointing when
treating the externally induced source, but we need to get the influence to
help us make plans in our product releases.
> Savepoints do not work with ExternallyInducedSources
> ----------------------------------------------------
>
> Key: FLINK-25256
> URL: https://issues.apache.org/jira/browse/FLINK-25256
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Affects Versions: 1.14.0, 1.13.3, 1.12.7
> Reporter: Dawid Wysakowicz
> Assignee: Arvid Heise
> Priority: Major
>
> It is not possible to take a proper savepoint with
> {{ExternallyInducedSource}} or {{ExternallyInducedSourceReader}} (both legacy
> and FLIP-27 versions). The problem is that we're hardcoding
> {{CheckpointOptions}} in the {{triggerHook}}.
> The outcome of current state is that operators would try to take checkpoints
> in the checkpoint location whereas the {{CheckpointCoordinator}} will write
> metadata for those states in the savepoint location.
> Moreover the situation gets even weirder (I have not checked it entirely), if
> we have a mixture of {{ExternallyInducedSource(s)}} and regular sources. In
> such a case the location and format at which the state of a particular task
> is persisted depends on the order of barriers arrival. If a barrier from a
> regular source arrives last the task takes a savepoint, on the other hand if
> last barrier is from an externally induced source it will take a checkpoint.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
