Proper Relationship Between Partition and Threads

2015-01-28 Thread Ricardo Ferreira
Hi experts, I'm newbie in the Kafka world, so excuse me for such basic question. I'm in the process of designing a client for Kafka, and after few hours of study, I was told that to achieve a proper level of parallelism, it is a best practice having one thread for each partition of an topic. My

Re: Proper Relationship Between Partition and Threads

2015-01-28 Thread Ricardo Ferreira
Thank you very much Christian. That's what I concluded too, I wanted just to double check. Best regards, Ricardo Ferreira On Wed, Jan 28, 2015 at 4:44 PM, Christian Csar christ...@csar.us wrote: Ricardo, The parallelism of each logical consumer (consumer group) is the number of

Re: Proper Relationship Between Partition and Threads

2015-01-28 Thread Christian Csar
Ricardo, The parallelism of each logical consumer (consumer group) is the number of partitions. So with four partitions it could make sense to have one logical consumer (application) have two processes on different machines each with two threads, or one process with four. While with two logical