[
https://issues.apache.org/jira/browse/KAFKA-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Lin updated KAFKA-4415:
----------------------------
Description:
As of current implementation, when controller receives
ControlledShutdownRequest, it will 1) for every broker in the cluster, for
every partition on the broker which wants to shutdown, create an instance of
PartitionStateInfo and add it to
ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every
broker, for every follower partitions on the broker which wants to shutdown,
send one MetadataUpdateRequst to that broker.
In order to shutdown a broker, the controller will need to instantiate
O(partitionNum * brokerNum) PartitionStateInfo. This is not efficient. The
broker should only need to instantiate O(partitionNum) PartitionStateInfo.
Micro-benchmark results show that this optimization can reduce the time of
processing ControlledShutdownRequest by 30%.
was:
As of current implementation, when controller receives
ControlledShutdownRequest, it will 1) for every broker in the cluster, for
every partition on the broker which wants to shutdown, create an instance of
PartitionStateInfo and add it to
ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every
broker, for every follower partitions on the broker which wants to shutdown,
send one MetadataUpdateRequst to that broker.
In order to shutdown a broker, the controller will need to instantiate
O(partitionNum * brokerNum) PartitionStateInfo and send O(partitionNum *
brokerNum) partitionStateInfo. This is not efficient. The broker should only
need to instantiate O(partitionNum) PartitionStateInfo and send O(brokerNum)
MetadataUpdateRequest.
Micro-benchmark results show that this optimization can reduce the time of
processing ControlledShutdownRequest by 30%.
> Reduce time to create and send MetadataUpdateRequest
> ----------------------------------------------------
>
> Key: KAFKA-4415
> URL: https://issues.apache.org/jira/browse/KAFKA-4415
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.1.0
> Reporter: Dong Lin
> Assignee: Dong Lin
> Fix For: 0.10.2.0
>
>
> As of current implementation, when controller receives
> ControlledShutdownRequest, it will 1) for every broker in the cluster, for
> every partition on the broker which wants to shutdown, create an instance of
> PartitionStateInfo and add it to
> ControllerBrokerRequestBatch.,updateMetadataRequestMap; and 2) for every
> broker, for every follower partitions on the broker which wants to shutdown,
> send one MetadataUpdateRequst to that broker.
> In order to shutdown a broker, the controller will need to instantiate
> O(partitionNum * brokerNum) PartitionStateInfo. This is not efficient. The
> broker should only need to instantiate O(partitionNum) PartitionStateInfo.
> Micro-benchmark results show that this optimization can reduce the time of
> processing ControlledShutdownRequest by 30%.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
