[
https://issues.apache.org/jira/browse/FLINK-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336277#comment-16336277
]
Stephan Ewen commented on FLINK-8467:
-------------------------------------
A job that is restored from a savepoint is expected to use the savepoint's
Kafka offsets. The savepoint describes the point in time when the snapshot was
taken. Assume, for example, that you want to replay a job from a day ago in
order to fix some corrupt data or to debug an issue - in that case it is
important that the offsets from that point in time are used.
For upgrades, I would also think that this is the correct way. The savepoint
captures the state and the offsets as they belong together. Upgrading with
savepoint state and other offsets (like the committed Kafka offsets) means
breaking the exactly-once guarantees.
Can you explain more where you think this is broken?
> Restoring job that does not use checkpointing from savepoint breaks
> -------------------------------------------------------------------
>
> Key: FLINK-8467
> URL: https://issues.apache.org/jira/browse/FLINK-8467
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Reporter: Jelmer Kuperus
> Priority: Major
>
> When checkpointing is disabled, the Flink Kafka Consumer relies on the
> periodic offsets that are committed to the broker by the internal Kafka
> client. Such a job would, upon restart, continue from the committed offsets.
> However, in the situation that the job is restored from a savepoint, then
> the offsets within the savepoint supercede the broker-based offsets.
> Here's a simple project that demonstrates the problem :
> [https://github.com/jelmerk/flink-cancel-restart-job-without-checkpointing]
> And a link to the mailing list thread :
> [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Starting-a-job-that-does-not-use-checkpointing-from-a-savepoint-is-broken-td17874.html]
>
> If this is not something you want to address at least the upgrading guide
> ([https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/upgrading.html)]
> should caution against this
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
