Thank you very much Christian. That's what I concluded too, I wanted just to double check.
Best regards, Ricardo Ferreira On Wed, Jan 28, 2015 at 4:44 PM, Christian Csar <christ...@csar.us> wrote: > Ricardo, > The parallelism of each logical consumer (consumer group) is the number > of partitions. So with four partitions it could make sense to have one > logical consumer (application) have two processes on different machines > each with two threads, or one process with four. While with two logical > consumers (two different applications) you would want each to have 4 > threads (4*2 = 8 threads total). > > There are also considerations depending on which consumer code you are > using (which I'm decidedly not someone with good information on) > > Christian > > On Wed, Jan 28, 2015 at 1:28 PM, Ricardo Ferreira < > jricardoferre...@gmail.com> wrote: > > > Hi experts, > > > > I'm newbie in the Kafka world, so excuse me for such basic question. > > > > I'm in the process of designing a client for Kafka, and after few hours > of > > study, I was told that to achieve a proper level of parallelism, it is a > > best practice having one thread for each partition of an topic. > > > > My question is that this rule-of-thumb also applies for multiple consumer > > applications. For instance: > > > > Considering a topic with 4 partitions, it is OK to have one consumer > > application with 4 threads, just like would be OK to have two consumer > > applications with 2 threads each. But what about having two consumer > > applications with 4 threads each? It would break any load-balancing made > by > > Kafka brokers? > > > > Anyway, I'd like to understand if the proper number of threads that > should > > match the number of partitions is per application or if there is some > other > > best practice. > > > > Thanks in advance, > > > > Ricardo Ferreira > > >
