[
https://issues.apache.org/jira/browse/KAFKA-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17329915#comment-17329915
]
Soumyajit Sahu edited comment on KAFKA-10619 at 4/23/21, 5:47 PM:
------------------------------------------------------------------
Hi [~d8tltanc] ,
I would like to raise an issue with making enable.idempotence=true by default.
Recently, we hit an issue where a producer app of ours was using
enable.idempotence=true, but was creating a producer object for each message.
We ended up in having thousands of producer ids for our log segment, and the
broker crashed with OutOfMemory exception while trying to take a snapshot.
I am not aware of any way to monitor this or check on such behavior. Hence,
setting this to true could break many clusters with a surprise.
It would be ideal to come up with a monitoring or controlling mechanism for
this before we set it to true by default.
was (Author: soumyajitsahu):
Hi [~d8tltanc] ,
I would like to raise an issue with making enable.idempotence=true by default.
Recently, we hit an issue where a producer app of ours was using
enable.idempotence=true, but was creating a producer object for each message.
We ended up in having thousands of producer ids for our log segment, and the
broker crashed with OutOfMemory exception while trying to take a snapshot.
I am not aware of any way to monitor this or check on such behavior. Hence,
setting this to true could break many clusters with a surprise.
It would be ideal to come up with a monitoring or controlling mechanism for
this before we set it to true by default.
> Producer will enable EOS by default
> -----------------------------------
>
> Key: KAFKA-10619
> URL: https://issues.apache.org/jira/browse/KAFKA-10619
> Project: Kafka
> Issue Type: Improvement
> Reporter: Cheng Tan
> Assignee: Cheng Tan
> Priority: Major
> Fix For: 3.0.0
>
>
> This is an after-work for KIP-185.
> In the producer config,
> # the default value of `acks` will change to `all`
> # `enable.idempotence` will change to `true`
> [An analysis of the impact of max.in.flight.requests.per.connection and acks
> on Producer
> performance|https://cwiki.apache.org/confluence/display/KAFKA/An+analysis+of+the+impact+of+max.in.flight.requests.per.connection+and+acks+on+Producer+performance]
> indicates that changing `acks` from `1` to `all` won't increase the latency
> and decrease the throughput in a significant way.
>
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
