Hello Tidy Bot, Alexey Serbin, Yuqi Du, Yingchun Lai, Kudu Jenkins, Wang Xixu,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20310
to look at the new patch set (#11).
Change subject: KUDU-3497 optimize leader rebalancer algorithm
......................................................................
KUDU-3497 optimize leader rebalancer algorithm
Leader rebalancing might not work well for now, especially for
the tables with smaller number of hash partitions.
For instance, for a table, consisting of 9 tablets, RF = 3, in a3-tservers
cluster.
Its leaders distribution is as follow:
Tablet server A : 4
Tablet server B : 4
Tablet server C : 1
According to the algorithm for now, there will not be any rebalance
operation scheduled.
Therefore, try to find a better algorithm to make it always find
the best leader distribution.
The formula is:
expected leader num = (tablets sum) % (tablets server num) = 0 ?
(tablets sum) / (tablets server num) :
ceil((tablets sum) / (tablets server num))
A tserver whose leader num is more than the expected value needs
to transfer the leaderships.
So the leader skew will never be more than 1.
Change-Id: I0f1fe796fd98da2d8764da793b7e254319e6348a
---
M src/kudu/master/auto_leader_rebalancer-test.cc
M src/kudu/master/auto_leader_rebalancer.cc
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/mini-cluster/external_mini_cluster.h
M src/kudu/mini-cluster/internal_mini_cluster.cc
M src/kudu/mini-cluster/internal_mini_cluster.h
M src/kudu/mini-cluster/mini_cluster.h
7 files changed, 313 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/20310/11
--
To view, visit http://gerrit.cloudera.org:8080/20310
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0f1fe796fd98da2d8764da793b7e254319e6348a
Gerrit-Change-Number: 20310
Gerrit-PatchSet: 11
Gerrit-Owner: Song Jiacheng <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Song Jiacheng <[email protected]>
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Wang Xixu <[email protected]>
Gerrit-Reviewer: Yingchun Lai <[email protected]>
Gerrit-Reviewer: Yuqi Du <[email protected]>
