[
https://issues.apache.org/jira/browse/FLINK-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tzu-Li (Gordon) Tai reassigned FLINK-5704:
------------------------------------------
Assignee: Tzu-Li (Gordon) Tai
> Deprecate FlinkKafkaConsumer constructors in favor of improvements to
> decoupling from Kafka offset committing
> -------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-5704
> URL: https://issues.apache.org/jira/browse/FLINK-5704
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector, Streaming Connectors
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Priority: Blocker
> Fix For: 1.4.0
>
>
> With FLINK-3398 and FLINK-4280, the {{FlinkKafkaConsumer}} will be able to
> completely operate independently of committed offsets in Kafka.
> I.e.,
> (1) *Starting position*: when starting, the consumer can choose to not use
> any committed offsets in Kafka as the starting position
> (2) *Committing offsets back to Kafka*: the consumer can completely opt-out
> of committing offsets back to Kafka
> However, our current default behaviour for (1) is to respect committed
> offsets, and (2) is to always have offset committing. Users still have to
> call the respective setter configuration methods to change this.
> I think we should deprecate the current constructors in favor of new ones
> with default behaviours (1) start from the latest record, without respecting
> Kafka offsets, and (2) don't commit offsets.
> With this change, users explicitly call the config methods of FLINK-3398 and
> FLINK-4280 to *enable* respecting committed offsets for Kafka, instead of
> _disabling_ it. They would want to / need to enable it, only when perhaps to
> migrate from a non-Flink consuming application, or they wish to expose the
> internal checkpointed offsets to measure consumer lag using Kafka toolings.
> The main advantage for this change is that the API of {{FlinkKafkaConsumer}}
> can speak for itself that it does not depend on committed offsets in Kafka
> (this is a misconception that users frequently have), and that exactly-once
> depends solely on offsets checkpointed internally using Flink's checkpointing
> mechanics.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
