[
https://issues.apache.org/jira/browse/KAFKA-15178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicholas Telford updated KAFKA-15178:
-------------------------------------
Labels: easyfix patch-available (was: easyfix)
> Poor performance of ConsumerCoordinator with many TopicPartitions
> -----------------------------------------------------------------
>
> Key: KAFKA-15178
> URL: https://issues.apache.org/jira/browse/KAFKA-15178
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 3.5.0
> Reporter: Nicholas Telford
> Assignee: Nicholas Telford
> Priority: Minor
> Labels: easyfix, patch-available
> Attachments: pollPhase.png
>
>
> Doing some profiling of my Kafka Streams application, I noticed that the
> {{pollPhase}} suffers from a minor performance issue.
> See the pink tree on the left of the flame graph below.
> !pollPhase.png|width=1028,height=308!
> {{ConsumerCoordinator.poll}} calls {{{}rejoinNeededOrPending{}}}, which
> checks the current {{metadataSnapshot}} against the
> {{{}assignmentSnapshot{}}}. This comparison is a deep-equality check, and if
> there's a large number of topic-partitions being consumed by the application,
> then this comparison can perform poorly.
> I suspect this can be trivially addressed with a {{boolean}} flag that
> indicates when the {{metadataSnapshot}} has been updated (or is "dirty"), and
> actually needs to be checked, since most of the time it should be identical
> to {{{}assignmentSnapshot{}}}.
> I plan to raise a PR with this optimization to address this issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
