Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20310 )
Change subject: KUDU-3497 optimize leader rebalancer algorithm ...................................................................... KUDU-3497 optimize leader rebalancer algorithm Leader rebalancing might not work well for now, especially for the tables with smaller number of hash partitions. For instance, for a table, consisting of 9 tablets, RF = 3, in a3-tservers cluster. Its leaders distribution is as follow: Tablet server A : 4 Tablet server B : 4 Tablet server C : 1 According to the algorithm for now, there will not be any rebalance operation scheduled. Therefore, try to find a better algorithm to make it always find the best leader distribution. The formula is: expected leader num = (tablets sum) % (tablets server num) = 0 ? (tablets sum) / (tablets server num) : ceil((tablets sum) / (tablets server num)) A tserver whose leader num is more than the expected value needs to transfer the leaderships. So the leader skew will never be more than 1. Change-Id: I0f1fe796fd98da2d8764da793b7e254319e6348a Reviewed-on: http://gerrit.cloudera.org:8080/20310 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin <[email protected]> --- M src/kudu/master/auto_leader_rebalancer-test.cc M src/kudu/master/auto_leader_rebalancer.cc M src/kudu/mini-cluster/external_mini_cluster.cc M src/kudu/mini-cluster/external_mini_cluster.h M src/kudu/mini-cluster/internal_mini_cluster.cc M src/kudu/mini-cluster/internal_mini_cluster.h M src/kudu/mini-cluster/mini_cluster.h 7 files changed, 314 insertions(+), 10 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/20310 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0f1fe796fd98da2d8764da793b7e254319e6348a Gerrit-Change-Number: 20310 Gerrit-PatchSet: 18 Gerrit-Owner: Song Jiacheng <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Song Jiacheng <[email protected]> Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Wang Xixu <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Reviewer: Yuqi Du <[email protected]>
