GitHub user becketqin opened a pull request:
https://github.com/apache/kafka/pull/1149
KAFKA-3436: Speed up controlled shutdown.
This patch does the followings:
1. Batched LeaderAndIsrRequest and UpdateMetadataRequest during controlled
shutdown.
2. Added async read and write method to an extending ZkClient. Used the
async zk operation for LeaderAndIsr read and update. The async method can be
used in other places as well (e.g. preferred leader election, replica
reassignment, controller bootstrap, etc), but those are out of the scope of
this ticket.
Conducted some rolling boucne test, a controlled shutdown involving 2500
partitions takes around 3 seconds now. Previously it can takes more than 30
seconds.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/becketqin/kafka KAFKA-3436
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1149.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1149
----
commit c2d22821c6c3ad7aa45090def6b984719209f5af
Author: Jiangjie Qin <[email protected]>
Date: 2016-03-27T21:29:30Z
KAFKA-3436: Speed up controlled shutdown
commit 7e7cf3fb1fc4a44d7af4ea935b38bf2e90e6cadd
Author: Jiangjie Qin <[email protected]>
Date: 2016-03-28T00:47:22Z
Remove pre-sent StopReplicaRequests and split state transition into
multiple groups.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
