Hi
I forgot to metion that I have multiple partitions and multiple consumer
processes.
But we can't process the messages in the same partition in parallel since they
might influence the processing of later records.
Does max.poll.records=1 always go to the remote server each time? What if I
increase fetch.min.bytes to say the expected size of 10 records. What will then
happen?
Best regards,
Mads
D. 07/05/2018 06.36 skrev "R Krishna" <krishna...@gmail.com>:
You can always add more partitions/consumer threads each fetching a few
more records than 1 but manually commit asynchronously one at a time, not
the best but better than doing max.poll.records=1 which fetches one record
from remote server at a time.
On Fri, May 4, 2018 at 4:19 AM, Mads Tandrup <
mads.tand...@schneider-electric.com> wrote:
> Hi
>
> What is the performance impact of setting `max.poll.records=1` as opposed
> to the default of 500?
>
> I have a Java application which process records one at a time. The
> processing time varies between messages, so we sometimes exceed the `
> max.poll.interval.ms`.
> While I could increase `max.poll.interval.ms` it would prevent me from
> detecting a livelock in the application quickly.
> There's no benefit of batching the records so I'm considering setting
> `max.poll.records=1`. We can define a sensible upper limit for the
> processing time of a single record.
>
> I've tried to look at the code and it seems that it fetches up to `
> fetch.max.bytes` and then keep it in-memory and returns records from the
> fetched data when `poll()` is called.
>
> So what is the performance impact of a low `max.poll.records`?
>
> Best regards,
> Mads
>
>
--
Radha Krishna, Proddaturi
253-234-5657
______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
______________________________________________________________________
