Hi Shri, I suggest you have a look at the protocol documentation and more specifically at this section: http://kafka.apache.org/protocol.html#protocol_partitioning
Regarding your question: The consumer does the same logic as the producer to get started. It sends a Metadata request to the bootstrap servers you provided as configuration. From there, it's able to find which brokers are the leaders for the partitions it wants to subscribe to. Then to get messages, it sends a Fetch request to the corresponding leader. If leadership changes, the previous leader will return an error indicating it is not the leader anymore (NOT_LEADER_FOR_PARTITION). To find the new leader, the Consumer will send a new Metadata request. On Thu, Aug 31, 2017 at 1:15 PM, Shrikant Patel <spa...@pdxinc.com> wrote: > Hi > > As I understand the producer in Kafka connect to broker list first to fetch > the metadata. Procedure uses that data to directly connect to leader for > partition that it's trying to publish to. > > From my understanding of Kafka protocol and following other threads, the > consumer so will be doing the same as producer. Meaning after the partition > is assigned to a consumer, the consumer will directly connect to leader to > read the message. > > Could provide some insight to that communication process (after partition is > assigned to consumer) between the broker and consumer. And if partition > leadership changes, how consumers become aware of that. > > Thanks in advance > Shri > > This e-mail and its contents (to include attachments) are the property of > National Health Systems, Inc., its subsidiaries and affiliates, including but > not limited to Rx.com Community Healthcare Network, Inc. and its > subsidiaries, and may contain confidential and proprietary or privileged > information. If you are not the intended recipient of this e-mail, you are > hereby notified that any unauthorized disclosure, copying, or distribution of > this e-mail or of its attachments, or the taking of any unauthorized action > based on information contained herein is strictly prohibited. Unauthorized > use of information contained herein may subject you to civil and criminal > prosecution and penalties. If you are not the intended recipient, please > immediately notify the sender by telephone at 800-433-5719 or return e-mail > and permanently delete the original e-mail.
