[
https://issues.apache.org/jira/browse/KAFKA-14757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705689#comment-17705689
]
Philip Nee commented on KAFKA-14757:
------------------------------------
Hi [~sanand] - I'm not sure what's behavior of the
ConsumerAwareRebalanceListener, but the default listener invokes
onPartitionsRevoked upon invoking onPartitionsLost. See java doc "by default,
[{{onPartitionsLost(Collection)}}|https://kafka.apache.org/24/javadoc/org/apache/kafka/clients/consumer/ConsumerRebalanceListener.html#onPartitionsLost-java.util.Collection-]
will be calling
[{{onPartitionsRevoked(Collection)}}|https://kafka.apache.org/24/javadoc/org/apache/kafka/clients/consumer/ConsumerRebalanceListener.html#onPartitionsRevoked-java.util.Collection-]
directly"
> Kafka Cooperative Sticky Assignor results in significant duplicate consumption
> ------------------------------------------------------------------------------
>
> Key: KAFKA-14757
> URL: https://issues.apache.org/jira/browse/KAFKA-14757
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 3.1.1
> Environment: AWS MSK (broker) and Spring Kafka (2.8.7) for use in
> Spring Boot consumers.
> Reporter: Siddharth Anand
> Priority: Critical
> Attachments: image-2023-03-16-10-34-49-358.png,
> image-2023-03-16-10-39-18-510.png
>
>
> Details may be found within the linked document:
> [Kafka Cooperative Sticky Assignor Issue : Duplicate Consumption |
> [https://docs.google.com/document/d/1E7qAwGOpF8jo_YhF4NwUx9CXxUGJmT8OhHEqIg7-GfI/edit?usp=sharing]]
> In a nutshell, we noticed that the Cooperative Sticky Assignor resulted in
> significant duplicate message consumption. During last year's F1 Grand Prix
> events and World Cup soccer events, our company's Kafka-based platform
> received live-traffic. This live traffic, coupled with autoscaled consumers
> resulted in as much as 70% duplicate message consumption at the Kafka
> consumers.
> In December 2022, we ran a synthetic load test to confirm that duplicate
> message consumption occurs during consumer scale out/in and Kafka partition
> rebalancing when using the Cooperative Sticky Assignor. This issue does not
> occur when using the Range Assignor.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
