Lokesh Khurana created PHOENIX-7973:
---------------------------------------
Summary: HA client can adopt a stale (lower-version)
ClusterRoleRecord when one RegionServer endpoint lags the peer
Key: PHOENIX-7973
URL: https://issues.apache.org/jira/browse/PHOENIX-7973
Project: Phoenix
Issue Type: Sub-task
Reporter: Lokesh Khurana
Assignee: Lokesh Khurana
Component: Phoenix HA / Consistent Failover
—HighAvailabilityGroup.getClusterRoleRecordFromEndpoint()
Problem: When fetching the ClusterRoleRecord (CRR) from the RegionServer
endpoints, the client queries cluster 1 first and, as long as that record has
no UNKNOWN role, returns it immediately without consulting cluster 2. CRR
version propagation across RegionServers is not synchronized, so during startup
or an in-flight admin/failover transition one endpoint can momentarily serve a
lower admin version than its peer. In that window the client adopts the stale,
lower-version CRR and silently reverts to an older cluster-role view. The
refresh path guards only with ClusterRoleRecord.equals() (which ignores
version) and never calls the existing isNewerThan() helper, so nothing detects
the downgrade.
Fix: Always fetch the CRR from both cluster endpoints and reconcile: prefer a
record without an UNKNOWN role (a known-role record is usable; an UNKNOWN one
is not), and within the same category prefer the higher admin version. This
guarantees the client never adopts a CRR older than one a peer already
advertises, at startup or during transitions. The reconciliation is factored
into a package-private static helper for unit testing (mirroring
shouldCountFailover). Adds a second endpoint RPC on the CRR refresh path only —
CRR fetches occur on connect/refresh (cached), not per-query, so no meaningful
perf impact.
Impact: Prevents transient split-brain / stale-role windows on the Phoenix HA
client. No API or wire-format change; client-side only.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
