>From the docs <https://kafka.apache.org/documentation/#impl_offsettracking> :
> The brokers periodically compact the offsets topic since it only needs to > maintain the most recent offset commit per partition Offsets are maintained per partition per consumer group, so it doesn't matter which member of a consumer group is reading a given partition - the offset will remain consistent. On Wed, May 30, 2018 at 9:23 AM, M. Manna <manme...@gmail.com> wrote: > Hello, > > I was trying to remember from docs (it's been a while) how the last > committed offsets work i.e. whether it's being tracked per consumer > group-basis or something else. This is specific to when auto.offset.reset > is set to "earliest"/"latest" and the last committed offset is determined. > > > . For example: > > 1) C0, C1, C2 are subscribed to topics t0 and t1 - each topic has 3 > partitions: > 2) Using range assignment - C0 [t0p0,t1p0] C1[t0p1,t1p1] and C2[t0p2,t1p2] > 3) Now C2 dies or leaves, or "Deactivates" - now rebalance occurs > 4) After Rebalance - C0[t0p0, t0p1, t1p0, t1p1] and C1[t0p2, t1p2] are the > new assignments > > Assuming that C0,C1,C2 are both part of *same *consumer group, and C2 > successfully committed to offset #6 (t0p2 and t1p2) before leaving, will #6 > be considered as the latest committed offset for that consumer group? In > other words, reassignment of that partition to any existing consumer under > that group will acknowlede #6 as the latest offset? > > If someone could quote the docs, it'll be appreciated. Meanwhile, I will > try to figure out something from the code if possible. > > Thanks, > -- Emmett Butler | Senior Software Engineer <http://www.parsely.com/?utm_source=Signature&utm_medium=emmett-butler&utm_campaign=Signature>
