Hi Gagan, It is true that a consumer will only poll for messages from the partitions to which it is assigned, but any single consumer can be assigned to poll from multiple partitions. That will happen whenever the number of partitions is greater than the number of consumers.
In your example of 5 partitions, if you only had two consumers, then one consumer would poll from 3 partitions and the other would poll from 2. Similarly, if you have 5 consumers and 20 partitions, then each of the consumers will poll from 4 different partitions. It is worth pointing out, too, that if you have more consumers than you have partitions, some of your consumers will not receive any work at all because there are not enough partitions for each consumer to be assigned at least one. For example, if you have 5 partitions and 8 consumers, then three of your consumers will be effectively idle with no work (the other 5 consumers will be assigned a single partition each). Kind regards, -Al On 2019-04-23 11:31:30, Gagan Sabharwal spake thus: > Hi Alex, > > Thanks for getting back to me so quickly. > > What I conclude from your answer is that a consumer is only loyal to the > partition it is glued to. It cannot steal work from other partitions. Is > the understanding correct? > > Do we anticipate work stealing in any of the future releases ? > > Regards > Gagan > > On Tue 23 Apr, 2019, 10:44 AM Alex Woolford, <a...@woolford.io> wrote: > > > Hey Gagan, > > > > If there are more partitions than consumers in a group, some consumers are > > allocated multiple partitions. > > > > If there are no more messages for the partitions that have been allocated > > to a consumer, that consumer will continue to poll until messages show up > > in those partitions. It won't start consuming messages from partitions that > > have been assigned to another consumer. > > > > HTH, > > > > Alex > > > > On Mon, Apr 22, 2019 at 10:57 PM Gagan Sabharwal <gagan...@gmail.com> > > wrote: > > > > > Hi team, > > > > > > If a consumer group has 5 subscribers to a topic which has 5 partitions, > > do > > > the subscribers steal work if they are done with their own? > > > > > > Regards > > > Gagan > > > > > Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
