GitHub user dosubot[bot] added a comment to the discussion: How the WebSocket server interacts with Redis ?
<!-- Answer --> The superset-websocket server interacts with Redis by reading async event data from Redis Streams and forwarding it to browser clients over WebSocket. Events are published to both a global stream and user-specific streams by the Superset Flask app. The WebSocket server does not use Redis consumer groups; instead, each instance reads the full stream so all clients get all relevant events ([source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/README.md#L19-L115)). Query parallelism and backpressure are managed by configuring how many messages are read from Redis in parallel (`redisStreamReadCount`) and the blocking time for reads (`redisStreamReadBlockMs`). TTL and key clearing for Redis Streams are handled by limiting the number of events in each stream (`GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT` and `GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE`), not by Redis key expiration ([source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/async_events/async_query_manager.py#L66-L308)). Stale data and inactive connections are cleaned up by periodic garbage collection (`gcChannelsIntervalMs`) and ping/pong health checks (`pingSocketsIntervalMs`, `socketResponseTimeoutMs`) ([source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/src/index.ts#L23-L463)). Payloads sent over WebSocket are JSON-encoded event data. There is no explicit payload size limit enforced in the current implementation. The protocol is a custom JSON-over-WebSocket scheme, so reusing it for direct front-end subscriptions to Redis would require implementing the same JWT-based authentication and channel management as the superset-websocket server ([source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/README.md#L19-L115)). Authorization uses JWT tokens stored in cookies (`jwtCookieName`), which must match between the Flask app and the WebSocket server. The JWT contains a channel ID for routing events. Subscriptions/channels are managed in-memory by tracking sockets per channel; all sockets for a channel receive all events for that channel ([source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/src/index.ts#L23-L463)). Security settings include JWT verification (with configurable secret and algorithms), optional Redis TLS/SSL, and Content Security Policy (CSP) configuration to restrict WebSocket origins. The WebSocket server must run on the same hostname as the Flask app for cookies to be shared for authentication ([source](https://github.com/apache/superset/discussions/34703)). Let me know if you need config examples or details on a specific part of the setup. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/ca3bc12f-4b1d-46dd-8d67-e6cb043ede18?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=The%20superset-websocket%20server%20interacts%20with%20Redis%20by%20reading%20async%20event%20data%20from%20Redis%20Streams%20and%20forwarding%20it%20to%20browser%20clients%20over%20WebSocket.%20Events%20are%20published%20to%20both%20a%20global%20stream%20and%20user-specific%20streams%20by%20the%20Superset%20Flask%20app.%20The%20WebSocket%20server%20does%20not%20use%20Redis%20consumer%20groups%3B%20instead%2C%20each%20instance%20reads%20the%20full%20stream%20so%20all%20clients%20get%20all%20relevant%20events%C2%A0%28%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/README.md%23L19-L115%29%29.%0A%0AQuery%20parallelism%20and%20backpressure%20are%20managed%20by%20configuring%20how%20many% 20messages%20are%20read%20from%20Redis%20in%20parallel%20%28%60redisStreamReadCount%60%29%20and%20the%20blocking%20time%20for%20reads%20%28%60redisStreamReadBlockMs%60%29.%20TTL%20and%20key%20clearing%20for%20Redis%20Streams%20are%20handled%20by%20limiting%20the%20number%20of%20events%20in%20each%20stream%20%28%60GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT%60%20and%20%60GLOBAL_ASYNC_QUERIES_REDIS_STREAM_LIMIT_FIREHOSE%60%29%2C%20not%20by%20Redis%20key%20expiration%C2%A0%28%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/async_events/async_query_manager.py%23L66-L308%29%29.%20Stale%20data%20and%20inactive%20connections%20are%20cleaned%20up%20by%20periodic%20garbage%20collection%20%28%60gcChannelsIntervalMs%60%29%20and%20ping/pong%20health%20checks%20%28%60pingSocketsIntervalMs%60%2C%20%60socketResponseTimeoutMs%60%29%C2%A0%28%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset- websocket/src/index.ts%23L23-L463%29%29.%0A%0APayloads%20sent%20over%20WebSocket%20are%20JSON-encoded%20event%20data.%20There%20is%20no%20explicit%20payload%20size%20limit%20enforced%20in%20the%20current%20implementation.%20The%20protocol%20is%20a%20custom%20JSON-over-WebSocket%20scheme%2C%20so%20reusing%20it%20for%20direct%20front-end%20subscriptions%20to%20Redis%20would%20require%20implementing%20the%20same%20JWT-based%20authentication%20and%20channel%20management%20as%20the%20superset-websocket%20server%C2%A0%28%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/README.md%23L19-L115%29%29.%0A%0AAuthorization%20uses%20JWT%20tokens%20stored%20in%20cookies%20%28%60jwtCookieName%60%29%2C%20which%20must%20match%20between%20the%20Flask%20app%20and%20the%20WebSocket%20server.%20The%20JWT%20contains%20a%20channel%20ID%20for%20routing%20events.%20Subscriptions/channels%20are%20managed%20in-memory%20by%20tracking%20sockets%20 per%20channel%3B%20all%20sockets%20for%20a%20channel%20receive%20all%20events%20for%20that%20channel%C2%A0%28%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset-websocket/src/index.ts%23L23-L463%29%29.%0A%0ASecurity%20settings%20include%20JWT%20verification%20%28with%20configurable%20secret%20and%20algorithms%29%2C%20optional%20Redis%20TLS/SSL%2C%20and%20Content%20Security%20Policy%20%28CSP%29%20configuration%20to%20restrict%20WebSocket%20origins.%20The%20WebSocket%20server%20must%20run%20on%20the%20same%20hostname%20as%20the%20Flask%20app%20for%20cookies%20to%20be%20shared%20for%20authentication%C2%A0%28%5Bsource%5D%28https%3A//github.com/apache/superset/discussions/34703%29%29.%20%0A%0ALet%20me%20know%20if%20you%20need%20config%20examples%20or%20details%20on%20a%20specific%20part%20of%20the%20setup.) [](https://go.dosu.dev/discord -bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/36383) GitHub link: https://github.com/apache/superset/discussions/36383#discussioncomment-15138664 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
