Re: Kafka Consumer Fairness when fetching events from different partitions.

2022-02-01 Thread Mazen Ezzeddine
research project ). Thank you. From: Edward Capriolo Sent: Monday, January 31, 2022 11:28 PM To: users@kafka.apache.org Subject: Re: Kafka Consumer Fairness when fetching events from different partitions. On Monday, January 31, 2022, Chad Preisler wrote: > He

Re: Kafka Consumer Fairness when fetching events from different partitions.

2022-01-31 Thread Edward Capriolo
On Monday, January 31, 2022, Chad Preisler wrote: > Hello, > > I got this from the JavaDocs for KafkaConsumer. > > * If a consumer is assigned multiple partitions to fetch data from, it > will try to consume from all of them at the same time, > * effectively giving these partitions the same pri

Re: Kafka Consumer Fairness when fetching events from different partitions.

2022-01-31 Thread Chad Preisler
Hello, I got this from the JavaDocs for KafkaConsumer. * If a consumer is assigned multiple partitions to fetch data from, it will try to consume from all of them at the same time, * effectively giving these partitions the same priority for consumption. However in some cases consumers may want

Kafka Consumer Fairness when fetching events from different partitions.

2022-01-22 Thread Mazen Ezzeddine
Dear all, Consider a kafka topic deployment with 3 partitions P1, P2, P3 with events/records lagging in the partitions equal to 100, 50, 75 for P1, P2, P3 respectively. And let’s suppose that num.poll.records (the maximum number of records that can be fetched from the broker ) is equal to 100.