[
https://issues.apache.org/jira/browse/KAFKA-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583098#comment-16583098
]
ASF GitHub Bot commented on KAFKA-7299:
---------------------------------------
junrao closed pull request #5515: KAFKA-7299: Batch LeaderAndIsr requests for
AutoLeaderRebalance
URL: https://github.com/apache/kafka/pull/5515
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core/src/main/scala/kafka/controller/KafkaController.scala
b/core/src/main/scala/kafka/controller/KafkaController.scala
index 645080f7641..e397e80db66 100644
--- a/core/src/main/scala/kafka/controller/KafkaController.scala
+++ b/core/src/main/scala/kafka/controller/KafkaController.scala
@@ -990,16 +990,13 @@ class KafkaController(val config: KafkaConfig, zkClient:
KafkaZkClient, time: Ti
// check ratio and if greater than desired ratio, trigger a rebalance
for the topic partitions
// that need to be on this broker
if (imbalanceRatio > (config.leaderImbalancePerBrokerPercentage.toDouble
/ 100)) {
- topicsNotInPreferredReplica.keys.foreach { topicPartition =>
- // do this check only if the broker is live and there are no
partitions being reassigned currently
- // and preferred replica election is not in progress
- if (controllerContext.isReplicaOnline(leaderBroker, topicPartition)
&&
- controllerContext.partitionsBeingReassigned.isEmpty &&
-
!topicDeletionManager.isTopicQueuedUpForDeletion(topicPartition.topic) &&
- controllerContext.allTopics.contains(topicPartition.topic)) {
- onPreferredReplicaElection(Set(topicPartition),
isTriggeredByAutoRebalance = true)
- }
- }
+ // do this check only if the broker is live and there are no
partitions being reassigned currently
+ // and preferred replica election is not in progress
+ val candidatePartitions = topicsNotInPreferredReplica.keys.filter(tp
=> controllerContext.isReplicaOnline(leaderBroker, tp) &&
+ controllerContext.partitionsBeingReassigned.isEmpty &&
+ !topicDeletionManager.isTopicQueuedUpForDeletion(tp.topic) &&
+ controllerContext.allTopics.contains(tp.topic))
+ onPreferredReplicaElection(candidatePartitions.toSet,
isTriggeredByAutoRebalance = true)
}
}
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> batch LeaderAndIsr requests during auto preferred leader election
> -----------------------------------------------------------------
>
> Key: KAFKA-7299
> URL: https://issues.apache.org/jira/browse/KAFKA-7299
> Project: Kafka
> Issue Type: Sub-task
> Components: core
> Affects Versions: 2.0.0
> Reporter: Jun Rao
> Assignee: huxihx
> Priority: Major
>
> Currently, in KafkaController.checkAndTriggerAutoLeaderRebalance(), we call
> onPreferredReplicaElection() one partition at a time. This means that the
> controller will be sending LeaderAndIsrRequest one partition at a time. It
> would be more efficient to call onPreferredReplicaElection() for a batch of
> partitions to reduce the number of LeaderAndIsrRequests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
