villebro opened a new issue, #45493: URL: https://github.com/apache/superset/issues/45493
### Bug description A browser tab that sits idle loses its realtime socket and never gets it back on its own. It retries about every 30s, each attempt rejected with "JWT not present", until the user happens to make an HTTP request. Meanwhile list views stop updating, and after the third failed attempt every failed retry fails all pending charts with "Realtime connection unavailable". Observed on two tabs against a 7.0.0rc2 instance, from Envoy access logs and websocket server logs: - 00:33:15: last request that minted the cookie (`GET /api/v1/me/`, the client keepalive). The cookie `max_age` is 900s, so it expires at 00:48:15. - 00:42:37: both sockets reconnect after the websocket server pod restarted. Their keepalive is now due at 00:51:37. - 00:48:33: the server terminates both sockets at its next expiry check. - 00:48:37 to 01:48:38: 236 upgrades rejected with "JWT not present" (Envoy logs them as `503 UC`, because the server destroys the socket without a response). - 01:48:44: the user loads `/welcome/`, the response mints a cookie, and the next retry succeeds. **Root cause** - The credential is an `httponly` cookie (`superset-ws-token`, 15-minute JWT) that the client can't read. It is only re-minted as a side effect of other HTTP requests (`_should_remint`, `register_ws_channel_cookie` in `superset/websocket/channel.py`). - The client's keepalive fires at `0.6 x lifetime` after the socket opened, not after the cookie was minted (`superset-frontend/src/middleware/realtime.ts`). A socket that reconnects more than about 6 minutes into the cookie's life schedules its refresh after the cookie has expired. - The reconnect path never refreshes the cookie, so once it has expired every retry fails identically. - The server terminates sockets at token expiry and rejects unauthenticated upgrades by destroying the TCP socket with no HTTP status, so the client can't tell an auth failure from a network failure. - Triggers in normal operation: any websocket server restart (rollouts), a Redis blip, a network blip, laptop sleep. The cookie also forces the websocket server to be on the same browser-visible host as the web server. **How to reproduce** 1. Set `WEBSOCKET_JWT_EXPIRATION_SECONDS = 60` for quick runs. 2. Open a dashboard and make no further requests. 3. Restart the websocket server at about 70% of the token lifetime. 4. The socket never recovers, and every retry is rejected with "JWT not present". **Expected behavior** The socket recovers and stays connected past the original expiry. The credential is refreshed on the open socket without a reconnect, and an auth failure is distinguishable from a network failure. ### Superset version master / latest-dev ### Python version 3.11 ### Node version Not applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
