Hello Tidy Bot, Alexey Serbin, Yingchun Lai, Yifan Zhang, Kudu Jenkins, Andrew
Wong, Abhishek Chennaka,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18454
to look at the new patch set (#28).
Change subject: [master] KUDU-3390 support auto rebalance tablet leaders across
TServers
......................................................................
[master] KUDU-3390 support auto rebalance tablet leaders across TServers
The number of leader replicas per tablet server can become imbalanced
over time, which lead to load skew on some nodes. This patch adding and
auto leader rebalance task to avoid leader replicas skew.
Two reasons of load skew:
- The main reason. Scan Requests has two modes: LeaderOnly(default) and
CLOSEST_REPLICA.
For more accurate results, users will choose the LeaderOnly(default) mode.
Mostly, the scan load is positive correlation with leader numbers.
- The other reason. Write requests, leaders receive write requests and
followers receive
UpdateConsensus request(raft called AppendEntries), the flow of processing is a
little
different, which is hidden variables, maybe cause imbalanced load. Leader
rebalance will
make leader and followers balanced and eliminate hidden variables and make
service more stable.
An experiment present leader rebalance's effect(more detail infomations at
KUDU-3390):
|leader ratio| scan cost | cpu usage | io
before|40: 0: 0 | 811.586 s | 47%, 18%, 19% | 102MB/s ioutil:55%, 8KB/s
2%, 64KB/s 3%
after |13: 14: 13 | 611.012 s | 39%, 45%, 35% | 53MB/s ioutil:31%, 80MB/s
18%, 45MB/s 24%
Change-Id: Ibfb60d8759a93b6a19238637c27df4f6b1cac918
---
M src/kudu/master/CMakeLists.txt
A src/kudu/master/auto_leader_rebalancer-test.cc
A src/kudu/master/auto_leader_rebalancer.cc
A src/kudu/master/auto_leader_rebalancer.h
M src/kudu/master/auto_rebalancer-test.cc
M src/kudu/master/auto_rebalancer.cc
M src/kudu/master/auto_rebalancer.h
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
9 files changed, 804 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/54/18454/28
--
To view, visit http://gerrit.cloudera.org:8080/18454
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ibfb60d8759a93b6a19238637c27df4f6b1cac918
Gerrit-Change-Number: 18454
Gerrit-PatchSet: 28
Gerrit-Owner: Yuqi Du <[email protected]>
Gerrit-Reviewer: Abhishek Chennaka <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yifan Zhang <[email protected]>
Gerrit-Reviewer: Yingchun Lai <[email protected]>
Gerrit-Reviewer: Yuqi Du <[email protected]>
