One consumer reads from one or more partitions. If you have 1 partition per topic, only one consumer in the consumer group will read from that partition, others will be in standby.
To process data in parallel in the consumer group, you need to increase the number of partitions per topic. For example, 2 partitions will allow 2 consumers to work in parallel, others will be standby. *Pavel Molchanov* Director of Software Development InfoDesk www.infodesk.com 1 Bridge Street | Suite 105 | Irvington | New York | 10551 <https://maps.google.com/?q=660+White+Plains+Road+%7C+Suite+300+%7C+Tarrytown+%7C+New+York+%7C+10591&entry=gmail&source=g>| Office: +1 (914) 332-5940 Change Privacy Settings <https://www.infodesk.com/unsubscription-options> | Contact Privacy Team <unsubscr...@infodesk.com> | Privacy Policy <https://www.infodesk.com/privacy-policy> This e-mail message may contain confidential or legally privileged information and is intended only for the use of the intended recipient(s). Any unauthorized disclosure, dissemination, distribution, copying or the taking of any action in reliance on the information herein is prohibited. On Thu, Jun 13, 2019 at 10:53 AM sunilv...@gmail.com <sunilv...@gmail.com> wrote: > I have 100 topics with one partition each. > I have 10 instance of consumer with the Same consumer group id and I'm > using subscribe using pattern. > I observe that all the topic partitions are assigned to the same consumer > and others are starving. It is not distributed. > I was expecting that rebalance would kick-in and the different topic > partitions would be assigned to different consumer instances. > Can anyone confirm if this is expected behavior. > > Thanks, > Sunil > >
