harshkha3417 opened a new pull request, #7812:
URL: https://github.com/apache/shenyu/pull/7812
Fixes #7808.
### Motivation
When ShenYu Admin runs in cluster mode and the current master fails over,
the gateway's websocket connection to the old master drops or gets demoted, but
the gateway remains stale until `WebsocketSyncDataService`'s background timer
triggers `masterCheck()`.
This issue was caused by four cooperating bottlenecks:
1. **Hard-coded 60s Check Interval**: `WebsocketSyncDataService` scheduled
`masterCheck()` with a fixed 60-second delay on `AbstractRoundTask`, which was
not configurable via `WebsocketConfig` / application properties
(`shenyu.sync.websocket.master-check-interval`).
2. **Delayed Re-seeding on Client Pruning**: In `masterCheck()`,
`CollectionUtils.isEmpty(clients)` was only evaluated at the start of the
method. When the dead/demoted master client was pruned by the while loop,
`clients` became empty and remained empty for another entire round (up to 60s)
before re-seeding clients from `websocketConfig.getUrls()`.
3. **Passive Socket Closure Without Master Re-election Trigger**: When the
websocket connection to the master admin dropped unexpectedly (`onClose`),
`ShenyuWebsocketClient` only attempted to reconnect to its own dead URL via
internal exponential backoff without notifying `WebsocketSyncDataService` that
the cluster master connection had been lost.
4. **Lack of Running Mode Probing on Open Connections**: If an Admin
instance lost master status without severing the TCP/websocket connection, the
client remained permanently marked with `isConnectedToMaster == true` because
`RUNNING_MODE` was only queried once on `onOpen`.
### Modifications
1. **Configurable Master Check Interval**:
- Added `masterCheckInterval` (defaults to 60 seconds) with getter,
setter, `equals`, `hashCode`, and `toString` to `WebsocketConfig`.
- In `WebsocketSyncDataService`, used
`websocketConfig.getMasterCheckInterval()` when initializing the periodic round
task.
2. **Immediate Re-seeding in the Same Pass**:
- In `WebsocketSyncDataService.masterCheck()`, re-evaluated
`CollectionUtils.isEmpty(clients)` after the client pruning loop. If all
clients were removed, `clients` is immediately re-seeded from
`websocketConfig.getUrls()`.
3. **Master Lost Reactive Notification**:
- Added constructor overload to `ShenyuWebsocketClient` accepting
`Consumer<ShenyuWebsocketClient> onMasterLostCallback`.
- In `onClose()`, if `wasMaster && !this.manuallyClosed.get()`,
`onMasterLostCallback` is invoked.
- In `onMessage()` for `RUNNING_MODE`, if cluster mode reports `wasMaster
&& !isMaster && !this.manuallyClosed.get()`, `onMasterLostCallback` is invoked.
- In `WebsocketSyncDataService`, `onMasterLost` schedules a debounced
one-shot `masterCheck()` (after a 200ms delay) so the gateway immediately
converges to the newly elected master without waiting for the full periodic
interval.
4. **Periodic Running Mode Verification**:
- In `ShenyuWebsocketClient.healthCheck()`, if the connection is open and
the running mode is not standalone, `DataEventTypeEnum.RUNNING_MODE.name()` is
sent alongside heartbeats to promptly catch leadership demotions.
5. **Testing**:
- Updated `WebsocketConfigTest` for getter/setter, equals, hashCode, and
toString.
- Added tests in `WebsocketSyncDataServiceTest` verifying configurable
check interval, same-pass re-seeding when all clients are removed, and master
loss notification triggering rediscovery.
- Added tests in `ShenyuWebsocketClientTest` covering `onClose`
notifications, `manuallyClosed` suppression, `RUNNING_MODE` demotion
notification, and cluster `healthCheck` probing.
### Verification
- `shenyu-sync-data-websocket` unit tests passed (127/127)
- `shenyu-spring-boot-starter-sync-data-websocket` unit tests passed (4/4)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
