Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14177 )
Change subject: KUDU-2780: create thread for auto-rebalancing ...................................................................... KUDU-2780: create thread for auto-rebalancing The auto-rebalancer thread is a background task of the master's catalog manager. Each loop iteration, the thread collects information on tablet servers, tables, and tablets, in order to determine the best rebalancing replica moves for the current state of the cluster. The maximum number of moves per tserver, per iteration, is controlled by a flag. Currently, each batch of moves scheduled per iteration is executed synchronously. The thread waits for the moves to complete, then sleeps for a time interval, which is controlled by a flag, before resuming. If the cluster has placement policy violations, the thread will prioritize scheduling and executing replica moves to reinstate the policy. Otherwise, the thread will perform inter-location (cross-location), then intra-location (by table, then by tserver) rebalancing. By default, auto-rebalancing is disabled. This can be changed by a flag in the catalog manager. The rebalancer tool should be run first, to fully rebalance the cluster, before enabling auto-rebalancing. Change-Id: Ifca25d1063c07047cf2123e6792b3c7395be20e4 Reviewed-on: http://gerrit.cloudera.org:8080/14177 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin <[email protected]> Reviewed-by: Andrew Wong <[email protected]> --- M src/kudu/master/CMakeLists.txt A src/kudu/master/auto_rebalancer-test.cc A src/kudu/master/auto_rebalancer.cc A src/kudu/master/auto_rebalancer.h M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master_runner.cc M src/kudu/master/ts_descriptor.h M src/kudu/master/ts_manager.cc M src/kudu/master/ts_manager.h M src/kudu/rebalance/rebalancer.cc M src/kudu/rebalance/rebalancer.h M src/kudu/tools/rebalancer_tool.cc M src/kudu/tools/tool_replica_util.cc M src/kudu/tools/tool_replica_util.h 15 files changed, 1,755 insertions(+), 107 deletions(-) Approvals: Kudu Jenkins: Verified Alexey Serbin: Looks good to me, approved Andrew Wong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/14177 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifca25d1063c07047cf2123e6792b3c7395be20e4 Gerrit-Change-Number: 14177 Gerrit-PatchSet: 27 Gerrit-Owner: Hannah Nguyen <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Hannah Nguyen <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241)
