Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20195 )
Change subject: KUDU-3476: Make replica placement range and table aware ...................................................................... KUDU-3476: Make replica placement range and table aware Previously, the replica selection policy randomly selected two tablet servers and placed the replica on the tserver with less replicas. This could lead to hotspotting if placing replicas from the same range on the same set of tservers since the policy doesn't discriminate by range. With this patch, the replica selection policy now ranks the available tservers by range and table load and places the replica accordingly. It prioritizes replicas by range first, replicas by table are used as a tiebreaker, then total replicas is used as the final tiebreaker. The range and table load is determined by the existing number of replicas before the placement begins and the number of pending replicas placed on the tserver while placing replicas. The flag --enable_range_replica_placement on the master side controls whether or not this new policy is used. For this feature to work, both the range start key and the table id of the table the range belongs to must be defined. This is because multiple tables could have the same range defined by the same range start key, so to differentiate the ranges, the table id is required. The link to the design doc is here: https://docs.google.com/document/d/1r-p0GW8lj2iLA3VGvZWAem09ykCmR5jEe8npUhJ07G8/edit?usp=sharing Change-Id: I9caeb8d5547e946bfeb152a99e1ec034c3fa0a0f Reviewed-on: http://gerrit.cloudera.org:8080/19931 Tested-by: Alexey Serbin <[email protected]> Reviewed-by: Alexey Serbin <[email protected]> (cherry picked from commit 10fdaf6a93a4bd3289d162b5f8351c4f0f5928c8) Reviewed-on: http://gerrit.cloudera.org:8080/20195 Reviewed-by: Yingchun Lai <[email protected]> --- M src/kudu/integration-tests/create-table-itest.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/master.proto M src/kudu/master/master_runner.cc M src/kudu/master/master_service.cc M src/kudu/master/placement_policy-test.cc M src/kudu/master/placement_policy.cc M src/kudu/master/placement_policy.h M src/kudu/master/ts_descriptor.cc M src/kudu/master/ts_descriptor.h M src/kudu/tools/rebalancer_tool-test.cc M src/kudu/tserver/heartbeater.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h 14 files changed, 1,048 insertions(+), 103 deletions(-) Approvals: Yingchun Lai: Looks good to me, approved Alexey Serbin: Verified -- To view, visit http://gerrit.cloudera.org:8080/20195 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: branch-1.17.x Gerrit-MessageType: merged Gerrit-Change-Id: I9caeb8d5547e946bfeb152a99e1ec034c3fa0a0f Gerrit-Change-Number: 20195 Gerrit-PatchSet: 4 Gerrit-Owner: Mahesh Reddy <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy <[email protected]> Gerrit-Reviewer: Yingchun Lai <[email protected]>
