[ 
https://issues.apache.org/jira/browse/FLINK-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572133#comment-15572133
 ] 

Cody Koeninger commented on FLINK-3037:
---------------------------------------

As a user, I want to be able to start low-value, high-volume streams at the 
latest offset, because it's not worth the time/money to start at the beginning.

I don't think silent data loss should be the default, but it should be 
configurable.

> Make the behavior of the Kafka consumer configurable if the offsets to 
> restore from are not available
> -----------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-3037
>                 URL: https://issues.apache.org/jira/browse/FLINK-3037
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kafka Connector
>            Reporter: Robert Metzger
>
> Currently, if the {{FlinkKafkaConsumer}} is restoring a checkpoint and the 
> offset is not available anymore in Kafka, its restoring according to 
> {{auto.offset.reset}}.
> This leads to inconsistent behavior (not exactly-once anymore) because the 
> operators will not receive data in sync with the checkpoint.
> With this pull request, I would like to make the behavior controllable, using 
> a flag. The simplest approach would be to let the consumer fail in that case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to