[
https://issues.apache.org/jira/browse/KAFKA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299148#comment-15299148
]
Gwen Shapira commented on KAFKA-3370:
-------------------------------------
actually, it is more complicated...
suppose that you have MirrorMaker - you want it to be earliest on start to
avoid missing data if MirrorMaker starts when topics already exist. But if it
accidentally gets ahead (or inconsistent) with HWM on broker, you don't want it
to reread and rereplicate the entire topic (true story) - in that case, we want
it to just move to latest.
Maybe we want to think through few scenarios (earliest if you are close to
beginning and latest if you are closer to end?) to see what makes sense.
> Add options to auto.offset.reset to reset offsets upon initialization only
> --------------------------------------------------------------------------
>
> Key: KAFKA-3370
> URL: https://issues.apache.org/jira/browse/KAFKA-3370
> Project: Kafka
> Issue Type: Bug
> Reporter: Guozhang Wang
> Assignee: Vahid Hashemian
> Fix For: 0.10.1.0
>
>
> Currently "auto.offset.reset" is applied in the following two cases:
> 1) upon starting the consumer for the first time (hence no committed offsets
> before);
> 2) upon fetching offsets out-of-range.
> For scenarios where case 2) needs to be avoid (i.e. people need to be
> notified upon offsets out-of-range rather than silently offset reset),
> "auto.offset.reset" need to be set to "none". However for case 1) setting
> "auto.offset.reset" to "none" will cause NoOffsetForPartitionException upon
> polling. And in this case, seekToBeginning/seekToEnd is mistakenly applied
> trying to set the offset at initialization, which are actually designed for
> during the life time of the consumer (in rebalance callback, for example).
> The fix proposal is to add two more options to "auto.offset.reset",
> "earliest-on-start", and "latest-on-start", whose semantics are "earliest"
> and "latest" for case 1) only, and "none" for case 2).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
