Re: [DISCUSS] KIP-281 ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User

[Alex Dunayevsky]

Hi Jason,

Thanks for your feedback and attention to detail (some credit to Kafka
0.8).
I've decreased the default polling loop timeout value to 10 seconds and
renamed parameter to --timeout. Updates reflected in PR and KIP.
Let's start the vote

Cheers,
Alex Dunayevsky

On Thu, Apr 5, 2018, 21:37 Alex Dunayevsky  wrote:

> Sure, updated the table under 'Public Interfaces' by adding the TimeUnit
> column.
> Thank you
>
> > In the table under 'Public Interfaces', please add a column with TimeUnit.
> > Looks good overall.
>
>
>
>
>
> ‌
>

Re: [DISCUSS] KIP-281 ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User

[Jason Gustafson]

Hey Alex,

Thanks for the KIP. Posting my reply which I accidentally added to another
KIP (before this KIP version was corrected):

I think the timeout controls the maximum allowed time that the consumer
will block for the next record. Maybe the meaning would be clearer with the
more concise name `--timeout`? That also fits with the old consumer which
overrides the `consumer.timeout.ms` property.

By the way, it seems like the default value was intentionally set low for
both the old and new consumers, but I'm not sure of the reason. We could
leave the default as it is if we want to be safe, but increasing it seems
ok to me. Perhaps we could start a little lower, though, say 10 seconds? In
any case, we should make it clear to the user that the timeout was reached.
It's surprising to see only the incomplete reported results following a
timeout.

Thanks,
Jason

On Thu, Apr 5, 2018 at 11:37 AM, Alex Dunayevsky 
wrote:

> Sure, updated the table under 'Public Interfaces' by adding the TimeUnit
> column.
> Thank you
>
> > In the table under 'Public Interfaces', please add a column with
> TimeUnit.
> > Looks good overall.
>
>
>
>
>
> ‌
>

Re: [DISCUSS] KIP-281 ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User

[Alex Dunayevsky]

Sure, updated the table under 'Public Interfaces' by adding the TimeUnit
column.
Thank you

> In the table under 'Public Interfaces', please add a column with TimeUnit.
> Looks good overall.





‌

Re: [DISCUSS] KIP-281 ConsumerPerformance: Increase Polling Loop Timeout and Make It Reachable by the End User

[Ted Yu]

In the table under 'Public Interfaces', please add a column with TimeUnit.

Looks good overall.

On Thu, Apr 5, 2018 at 7:14 AM, Alex Dunayevsky 
wrote:

> Hello friends,
>
> I would like to start a discussion thread for KIP-281:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 281%3A+ConsumerPerformance%3A+Increase+Polling+Loop+Timeout+
> and+Make+It+Reachable+by+the+End+User
>
> JIRA: KAFKA-6743: ConsumerPerformance fails to consume all messages on
> topics with large number of partitions
> 
>
> PR: https://github.com/apache/kafka/pull/4818
>
> Thank you,
> Alex Dunayevsky
>