Hi all, This email describes recent work to improve the high availability of the authority (permission) module in IoTDB, ensuring correct behavior during ConfigNode failover and network partition scenarios. (pr: https://github.com/apache/iotdb/pull/18425) **Background** The authority module manages user/role permissions and caches them on each DataNode for performance. Permission changes (grant, revoke, create/delete user, etc.) are written through the ConfigNode Raft consensus layer, then propagated to DataNodes to invalidate their local caches. The previous implementation used heartbeat-based cache expiration and synchronous, sequential RPC calls for cache invalidation. **Problem** Under network partition between ConfigNode and DataNode, the heartbeat-based mechanism could leave DataNodes with stale permission caches for an unbounded period. Additionally, the synchronous one-by-one RPC approach made cache invalidation fragile: if one DataNode was unreachable, the propagation could stall or silently skip nodes. **Solution** The change has two main parts: 1. **Lease-based fencing replaces heartbeat-based expiration** (`ClusterAuthorityFetcher`). Instead of checking a heartbeat timestamp and guessing cache freshness, every permission check now verifies the DataNode's metadata lease. If the lease is fenced (expired), the check fails closed and the caller retries — forcing a fresh cache pull from ConfigNode once connectivity is restored. This is the same fencing mechanism already used by the schema engine. 2. **Async broadcast replaces synchronous sequential RPC** (`AuthOperationProcedure`). After committing an auth plan to consensus, the ConfigNode broadcasts the cache invalidation request to all online DataNodes in parallel via `ClusterCachePropagator`, the same utility used for schema cache propagation. If any DataNode fails to respond, the procedure logs a warning and marks itself failed with a clear message suggesting manual cache clearance. The `refreshToken()` / `checkCacheAvailable()` path on the DataNode side and its corresponding heartbeat call in `DataNodeInternalRPCServiceImpl` have been removed. The `IAuthorityFetcher` interface no longer exposes `refreshToken()`. **Scope** This change only affects the cache invalidation path of the authority module. The consensus write path (auth plan submission) is unchanged. The semantics of cache invalidation on DataNodes are now fail-closed rather than eventually-consistent — safer under partition, at the cost of temporarily rejecting permission checks on a fenced DataNode until the lease recovers. Feedback welcome. --- **中文版：** 各位好， 这封邮件介绍近期对 IoTDB 权限模块高可用性的改进，确保在 ConfigNode 故障转移和网络分区场景下权限行为的正确性。 **背景** 权限模块管理用户/角色的权限信息，并在各 DataNode 上缓存以提高查询性能。权限变更操作（授权、撤销、创建/删除用户等）通过 ConfigNode 的 Raft 共识层写入，然后传播到 DataNode 使其本地缓存失效。原有实现依赖心跳机制判断缓存是否过期，并采用同步、逐一的 RPC 调用进行缓存失效通知。 **问题** 在 HA 事件（ConfigNode 领导者变更、ConfigNode 与 DataNode 网络分区）下，基于心跳的过期机制可能导致 DataNode 在不确定的时间内持有过期权限。同时，同步逐一的 RPC 方式使缓存失效传播较为脆弱：某个 DataNode 不可达时，传播可能停滞或静默跳过。 **解决方案** 本改动包含两个主要部分： 1. **基于租约的 fencing 替代心跳过期机制**（`ClusterAuthorityFetcher`）。每次权限检查不再是查看心跳时间戳来猜测缓存新鲜度，而是验证 DataNode 的元数据租约。若租约被 fencing（过期），权限检查以 fail-closed 方式失败并触发重试，迫使调用方在连接恢复后从 ConfigNode 重新拉取缓存。这套 fencing 机制与 Schema 引擎已使用的机制一致。 2. **异步广播替代同步串行 RPC**（`AuthOperationProcedure`）。权限计划写入共识层后，ConfigNode 通过 `ClusterCachePropagator` 向所有在线 DataNode 并行广播缓存失效请求（与 Schema 缓存传播使用相同工具）。若有 DataNode 未响应，过程记录 warning 日志并以明确信息标记失败，提示管理员可手动清理权限缓存。 DataNode 侧的 `refreshToken()` / `checkCacheAvailable()` 路径及 `DataNodeInternalRPCServiceImpl` 中对应的心跳调用已移除。`IAuthorityFetcher` 接口不再暴露 `refreshToken()`。 **影响范围** 此变更仅影响权限模块的缓存失效路径，共识写入路径（权限计划提交）未做修改。DataNode 上缓存失效的语义从"最终一致"转变为 fail-closed——在网络分区下更安全，代价是在被 fencing 的 DataNode 上暂时拒绝权限检查，直至租约恢复。
