On Tue, Jul 21, 2015 at 2:38 AM, Stevo Slavić <ssla...@gmail.com> wrote:
> Hello Apache Kafka community, > > I find new consumer poll/seek javadoc a bit confusing. Just by reading docs > I'm not sure what the outcome will be, what is expected in following > scenario: > > - kafkaConsumer is instantiated with auto-commit off > - kafkaConsumer.subscribe(someTopic) > - kafkaConsumer.position is called for every TopicPartition HLC is actively > subscribed on > > and then when doing multiple poll calls in succession (without calling > commit), does seek have to be called in between poll calls to position HLC > to skip what was read in previous poll, or does HLC keep that state > (position after poll) in memory, so that next poll (without seek in between > two poll calls) will continue from where last poll stopped? > The position is tracked in-memory within the consumer, so as long as there isn't a consumer rebalance, consumption will just proceed with subsequent messages (i.e. the behavior I think most people would find intuitive). However, if a rebalance occurs (another consumer instance joins the group or some leave), then a partition may be assigned to an different consumer instance that has no idea about the current position and will restart based on the offset reset setting (because attempting to fetch the committed offset will fail since no offsets have been committed). -Ewen > Could be it's just me not understanding this from javadoc. If not, maybe > javadoc can be improved to make this (even) more obvious. > > Kind regards, > Stevo Slavic. > -- Thanks, Ewen
