Hi If you have two consumers in your consumer group, but only one partition in the topic, then only one consumer will do any work. It’s not the case that “those two nodes start competing for messages” — one node will read from the partition, the other will have nothing to do. So to scale up by adding more consumers in your consumer group, you’ll need more partitions in your topic.
Ian. > On Feb 24, 2017, at 9:30 AM, Jakub Stransky <stransky...@gmail.com> wrote: > > Hello everyone, > > I was reading/checking kafka documentation regarding point-2-point and > publish subscribe communications patterns in kafka and I am wondering how > to scale up consumer side in point to point scenario when consuming from > single kafka topic. > > Let say I have a single topic with single partition and I have one node > where the kafka consumer is running. If I want to scale up my service I add > another node - which has the same configuration as the first one (topic, > partition and consumer group id). Those two nodes start competing for > messages from kafka topic. > > What I am not sure in this scenario and is actually subject of my question > is "*Whether they do get each node unique messages or there is still > possibility that some messages will be consumed by both nodes etc*". > Because I can see scenarios that both nodes are started at the same time - > they gets the same topic offset from zookeeper and started consuming > messages from that offset. OR am I thinking in a wrong direction? > > Thanks > Jakub
