Hi all - Just getting familiar with Kafka, and learning about consumer groups. Hoping someone can provide some context here.
As I understand it, consumers register with the broker and consume a topic. Multiple consumers can consume a single topic, as a consumer group. Each consumer actually gets a partition of messages, so there is no overlap -- a single consumer within a group will receive a message on its topic/partition. Consumer rebalancing is the process whereby members of a consumer group are added and/or dropped from the group, and partitions are sorted/reassigned to the current consumer group members. Some questions: * Is this accurate? What am I missing? * Operationally, is consumer "failover" basically service monitoring at the consumer process level? * How much coordination is required between producers and consumers around partitioning? (Automated, configuration, etc.) * How are topics monitored for SLA on throughput/load, i.e. spinning up consumers as needed for topic message spikes? Appreciate any further information and/or context anyone can share. cheers, Jeff
