Marton Greber has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24945 )
Change subject: KUDU-3802 add cluster_leader_skew gauge ...................................................................... KUDU-3802 add cluster_leader_skew gauge The counters on the auto-leader-rebalancer report activity: how many rounds ran, how many moves were scheduled. They don't report whether leadership is currently spread unevenly across the cluster, so an operator can't tell from them whether rebalancing is needed at all. This changelist adds a cluster_leader_skew gauge on the master, next to the existing cluster_replica_skew one. It reports the difference in Raft leader count between the tablet server hosting the most leaders and the one hosting the fewest. Tablet servers already track their own leader count in the num_raft_leaders gauge, so the count only needs to be carried over to the master. TSHeartbeatRequestPB gains an optional num_raft_leaders field, and the master stores the last reported value in the TSDescriptor of the corresponding tablet server. Both the heartbeat field and the master-side value are optional, so a tablet server running a version that predates the field is left out of the calculation rather than counted as hosting no leaders. Counting it as zero would make it look like the least loaded server in the cluster, and the gauge would read as high as the maximum leader count for as long as a rolling upgrade takes. The new master-test scenario forges heartbeats to check the value of the gauge, including the case of a tablet server that reports no count. The new registration-test scenario checks that the count a real tablet server reports actually reaches the master. Change-Id: Iec82f841934f4e310509ee1b7156040b3bba5489 Reviewed-on: http://gerrit.cloudera.org:8080/24945 Reviewed-by: Alexey Serbin <[email protected]> Reviewed-by: Marton Greber <[email protected]> Tested-by: Marton Greber <[email protected]> --- M src/kudu/integration-tests/registration-test.cc M src/kudu/master/master-test.cc M src/kudu/master/master.proto M src/kudu/master/master_service.cc M src/kudu/master/ts_descriptor.h M src/kudu/master/ts_manager.cc M src/kudu/master/ts_manager.h M src/kudu/tserver/heartbeater.cc 8 files changed, 203 insertions(+), 0 deletions(-) Approvals: Alexey Serbin: Looks good to me, approved Marton Greber: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24945 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iec82f841934f4e310509ee1b7156040b3bba5489 Gerrit-Change-Number: 24945 Gerrit-PatchSet: 4 Gerrit-Owner: Gabriella Lotz <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Gabriella Lotz <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Marton Greber <[email protected]>
