[
https://issues.apache.org/jira/browse/KAFKA-14668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-14668:
---------------------------------
Description:
While the KRaftMigrationDriver is in "dual write" mode, in addition to
propagating metadata back to ZooKeeper, it is sending controller RPCs to ZK
brokers. When a MetadataDelta is published, it will trigger the metadata
synchronization logic and send out redundant UMR even if there is no metadata
to change in the UMR. This doesn't affect correctness of the metadata in the
cluster, but will have a significant impact on performance for larger clusters.
We need to avoid sending RPCs to the ZK brokers when nothing relevant has
changed.
was:
While the KRaftMigrationDriver is in "dual write" mode, in addition to
propagating metadata back to ZooKeeper, it is sending controller RPCs to ZK
brokers. When a NoOpRecord is written, it will trigger the metadata
synchronization logic and send out redundant UMR even if no metadata has
changed. This doesn't affect correctness of the metadata in the cluster, but
will have a significant impact on performance for larger clusters.
We need to avoid sending RPCs to the ZK brokers when nothing has changed.
> Avoid sending unnecessary UMR in migration mode
> -----------------------------------------------
>
> Key: KAFKA-14668
> URL: https://issues.apache.org/jira/browse/KAFKA-14668
> Project: Kafka
> Issue Type: Bug
> Reporter: David Arthur
> Priority: Major
>
> While the KRaftMigrationDriver is in "dual write" mode, in addition to
> propagating metadata back to ZooKeeper, it is sending controller RPCs to ZK
> brokers. When a MetadataDelta is published, it will trigger the metadata
> synchronization logic and send out redundant UMR even if there is no metadata
> to change in the UMR. This doesn't affect correctness of the metadata in the
> cluster, but will have a significant impact on performance for larger
> clusters.
>
> We need to avoid sending RPCs to the ZK brokers when nothing relevant has
> changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
