Hello Tidy Bot, Kudu Jenkins, Andrew Wong,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18294
to look at the new patch set (#5).
Change subject: [tools] range rebalancing for 'kudu cluster rebalance'
......................................................................
[tools] range rebalancing for 'kudu cluster rebalance'
This patch adds range rebalancing functionality into the
'kudu cluster rebalance' CLI tool. The implementation is rather an
MVP: the range rebalancing can now be performed only for a single
table per run. As far as I can see, there is a room for improvement
since it's possible to perform range-aware replica movements even
during standard whole cluster rebalancing.
Below is two snapshots of distribution of the range-specific tablet
replicas in a cluster. Those are produced by running the tool with
extra --report_only --output_replica_distribution_details flags
before and after range rebalancing for a single table:
kudu cluster rebalance \
--enable_range_rebalancing \
--tables=default.loadgen_auto_6800f4ec4e164b2b8e42db7b5044df09 \
127.0.0.1:8765
before:
========================================================================
Table: abb2bbf8b4ff4bc0989bc82c78d4ae2b
Range start key: ''
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 8
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 0
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 0
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 0
Range start key: 'ff80000000000001fff4'
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 0
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 8
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 0
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 0
Range start key: 'ff80000000000003ffe8'
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 0
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 0
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 8
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 0
Range start key: 'ff80000000000005ffdc'
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 0
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 0
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 0
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 8
after:
========================================================================
Table: abb2bbf8b4ff4bc0989bc82c78d4ae2b
Range start key: ''
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2
Range start key: 'ff80000000000001fff4'
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2
Range start key: 'ff80000000000003ffe8'
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2
Range start key: 'ff80000000000005ffdc'
UUID | Server address | Replica Count
----------------------------------+----------------+---------------
15a8d0fef42c4da2bd5d9e1c5a2de301 | 127.0.0.1:9870 | 2
3243029e0db04680a2653c6acc048813 | 127.0.0.1:9876 | 2
324ef10666b14ab9bb61e775fa351ad6 | 127.0.0.1:9872 | 2
a5c6f822f5cc4645bbb4a14874e311d4 | 127.0.0.1:9874 | 2
Change-Id: I7d2e19266e993f5e2ae13ba18d323c83db30eac1
---
M src/kudu/master/auto_rebalancer.cc
M src/kudu/rebalance/rebalance-test.cc
M src/kudu/rebalance/rebalance_algo-test.cc
M src/kudu/rebalance/rebalance_algo.cc
M src/kudu/rebalance/rebalance_algo.h
M src/kudu/rebalance/rebalancer.cc
M src/kudu/rebalance/rebalancer.h
M src/kudu/tools/rebalancer_tool-test.cc
M src/kudu/tools/rebalancer_tool.cc
M src/kudu/tools/tool_action_cluster.cc
10 files changed, 1,195 insertions(+), 329 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/18294/5
--
To view, visit http://gerrit.cloudera.org:8080/18294
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7d2e19266e993f5e2ae13ba18d323c83db30eac1
Gerrit-Change-Number: 18294
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
