The new consumers' group metadata is now stored in Kafka instead of zookeeper.
The group coordinator will wait for the JoinGroupRequest from all the consumers in the current generation before it moves on to ask group leader assign the partitions. After the consumers passes this barrier, supposedly the partition ownership has already been revoked. However, if there is a consumer failed to join the group within the joining window (max session timeout of all consumers), that consumer will miss the rebalance and got kicked out of the group without knowing about it. It is possible that this problematic consumer is still consuming from the partitions that have been reassigned to a new owner. In this case, there will be two consumer consuming from the same partition at the same time. But the old owner will not be able to commit offsets because its generation is old. Thanks, Jiangjie (Becket) Qin On Wed, Feb 24, 2016 at 4:58 AM, tuyang <tuyang19890...@126.com> wrote: > Hi, > after reviewed Kafka 0.9.0 code, I have a rebalance question: > coordinator will choose a client as leader to execute assign, > after transfer it to coordinator, when assignments are Propagate to each > member in consumer group,how can it sure one partition is only owned by a > consumer because of consumer’s execute orders.I can not find a owner node > in kafka 0.9 zk. > because the execute order of the two consumers, may be at a fixed time, A > consumer owned a partition at last generation, and B consumer owned this > partition at this partition, when this occurs, how it resolve? >
