[
https://issues.apache.org/jira/browse/KUDU-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18123876#comment-18123876
]
ASF subversion and git services commented on KUDU-3802:
-------------------------------------------------------
Commit 92d89b116407553399dbd3509fa6690679499cdf in kudu's branch
refs/heads/master from Gabriella Lotz
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=92d89b116 ]
KUDU-3802 add cluster_leader_skew gauge
The counters on the auto-leader-rebalancer report activity: how many
rounds ran, how many moves were scheduled. They don't report whether
leadership is currently spread unevenly across the cluster, so an
operator can't tell from them whether rebalancing is needed at all.
This changelist adds a cluster_leader_skew gauge on the master, next
to the existing cluster_replica_skew one. It reports the difference in
Raft leader count between the tablet server hosting the most leaders
and the one hosting the fewest.
Tablet servers already track their own leader count in the
num_raft_leaders gauge, so the count only needs to be carried over to
the master. TSHeartbeatRequestPB gains an optional num_raft_leaders
field, and the master stores the last reported value in the
TSDescriptor of the corresponding tablet server.
Both the heartbeat field and the master-side value are optional, so a
tablet server running a version that predates the field is left out of
the calculation rather than counted as hosting no leaders. Counting it
as zero would make it look like the least loaded server in the
cluster, and the gauge would read as high as the maximum leader count
for as long as a rolling upgrade takes.
The new master-test scenario forges heartbeats to check the value of
the gauge, including the case of a tablet server that reports no
count. The new registration-test scenario checks that the count a real
tablet server reports actually reaches the master.
Change-Id: Iec82f841934f4e310509ee1b7156040b3bba5489
Reviewed-on: http://gerrit.cloudera.org:8080/24945
Reviewed-by: Alexey Serbin <[email protected]>
Reviewed-by: Marton Greber <[email protected]>
Tested-by: Marton Greber <[email protected]>
> Add cluster leader-skew gauge
> -----------------------------
>
> Key: KUDU-3802
> URL: https://issues.apache.org/jira/browse/KUDU-3802
> Project: Kudu
> Issue Type: Sub-task
> Reporter: Gabriella Lotz
> Assignee: Gabriella Lotz
> Priority: Major
>
> The counters on the auto-rebalancer and the ones added under KUDU-3791 for
> the auto-leader-rebalancer report activity: how many moves were scheduled,
> how many rounds completed. They don't report whether the cluster is currently
> skewed.
> The replica side already has this: cluster_replica_skew at ts_manager.cc:55
> is a function gauge that returns the difference between the tserver with the
> most replicas and the tserver with the fewest.
> Add a matching cluster_leader_skew gauge: max minus min leader count across
> live tservers. The leader rebalancer already computes this per-round in
> RunLeaderRebalancer, so the gauge can share that state or recompute on scrape.
> This gives operators a "does the cluster need rebalancing right now" signal
> to sit alongside the "is the rebalancer doing anything" signals from the
> counters.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
