GitHub user dosubot[bot] added a comment to the discussion: How the WebSocket
server interacts with Redis ?
<!-- Answer -->
Here’s how you can configure superset-websocket and Superset for async event
messaging, and implement a custom message format with fields like request_id,
type, axis, level, offset, limit, payload, metrics_meta, and error:
**Configuration Examples**
In `superset_config.py` (Superset backend):
```python
GLOBAL_ASYNC_QUERIES = True
GLOBAL_ASYNC_QUERIES_TRANSPORT = "ws"
GLOBAL_ASYNC_QUERIES_WEBSOCKET_URL = "ws://localhost:8080/"
GLOBAL_ASYNC_QUERIES_REDIS_CONFIG = {
"host": "127.0.0.1",
"port": 6379,
"db": 2,
"ssl": False
}
GLOBAL_ASYNC_QUERIES_JWT_SECRET = "your-secret-key"
GLOBAL_ASYNC_QUERIES_JWT_COOKIE_NAME = "async-token"
GLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX = "async-events-"
```
In `superset-websocket/config.json`:
```json
{
"port": 8080,
"redis": {
"port": 6379,
"host": "127.0.0.1",
"db": 2,
"ssl": false
},
"redisStreamPrefix": "async-events-",
"jwtSecret": "your-secret-key",
"jwtCookieName": "async-token"
}
```
Make sure the JWT secret, cookie name, Redis details, and stream prefix match
in both configs. The hostname must match how you access Superset in your
browser for cookies to work. Both the Flask server and superset-websocket
server must be running simultaneously for async features to work
[source](https://github.com/apache/superset/issues/36069).
**Custom Message Format Implementation**
Superset’s async event messages are built in the backend (see
`async_query_manager.py`). The event data dict is constructed using
`build_job_metadata` and updated via `update_job`. You can add custom fields
(like request_id, type, axis, etc.) by passing them as keyword arguments to
`update_job`. These fields will be included in the JSON payload written to
Redis and forwarded by superset-websocket to clients
[source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/async_events/async_query_manager.py#L53-L308).
**Example: Custom Event Payload**
Here’s what a custom event payload might look like:
```json
{
"id": "1615426152415-0", // Redis stream ID
"channel_id": "abc123",
"job_id": "job-uuid",
"user_id": "1",
"request_id": "req-uuid",
"type": "query",
"axis": "x",
"level": "detail",
"offset": 0,
"limit": 100,
"payload": { "data": [...] },
"metrics_meta": { "duration_ms": 123 },
"error": null,
"status": "done",
"result_url": "/api/v1/chart/data/job-uuid"
}
```
To implement this, update your async query Celery tasks (e.g., in
`async_queries.py`) to pass these fields to `async_query_manager.update_job`.
For example:
```python
async_query_manager.update_job(
job_metadata,
async_query_manager.STATUS_DONE,
request_id="req-uuid",
type="query",
axis="x",
level="detail",
offset=0,
limit=100,
payload=payload,
metrics_meta={"duration_ms": 123},
error=None,
result_url=result_url,
)
```
Any fields you add here will be included in the event data sent to Redis and
delivered to the frontend via WebSocket
[source](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/tasks/async_queries.py#L82-L166).
**Best Practices & Caveats**
- There’s no explicit config option for message format in superset-websocket;
you control the payload in the backend code.
- Make sure any consumers (frontend, other services) expect and handle your
custom fields.
- The error field is typically a list of error dicts, populated if exceptions
occur during async query execution.
- There’s no enforced payload size limit in superset-websocket, but you can set
limits in the backend (e.g., `SQLLAB_PAYLOAD_MAX_MB` for SQL Lab)
[source](https://github.com/apache/superset/pull/30522).
Let me know if you want code snippets for a specific async query type or more
details on frontend handling!
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
---
<sup>How did I do?
[Good](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=great_response)
|
[Irrelevant](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=irrelevant_answer)
|
[Incorrect](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=incorrect_sources)
|
[Verbose](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=too_verbose)
|
[Hallucination](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=hallucination)
| [Report
🐛](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=bug_report)
|
[Other](https://app.dosu.dev/response-feedback/d81218be-f731-4857-8fd4-b69492a5d8d4?feedback_type=other)</sup>
[](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=Here%E2%80%99s%20how%20you%20can%20configure%20superset-websocket%20and%20Superset%20for%20async%20event%20messaging%2C%20and%20implement%20a%20custom%20message%20format%20with%20fields%20like%20request_id%2C%20type%2C%20axis%2C%20level%2C%20offset%2C%20limit%2C%20payload%2C%20metrics_meta%2C%20and%20error%3A%0A%0A%2A%2AConfiguration%20Examples%2A%2A%0A%0AIn%20%60superset_config.py%60%20%28Superset%20backend%29%3A%0A%0A%60%60%60python%0AGLOBAL_ASYNC_QUERIES%20%3D%20True%0AGLOBAL_ASYNC_QUERIES_TRANSPORT%20%3D%20%22ws%22%0AGLOBAL_ASYNC_QUERIES_WEBSOCKET_URL%20%3D%20%22ws%3A//localhost%3A8080/%22%0AGLOBAL_ASYNC_QUERIES_REDIS_CONFIG%20%3D%20%7B%0A%20%20%20%20%22host%22%3A%20%22127.0.0.1%22%2C%0A%20%20%20%20%22port%22%3A%206379%2C%0A%20%20%20%20%22db%22%3A
%202%2C%0A%20%20%20%20%22ssl%22%3A%20False%0A%7D%0AGLOBAL_ASYNC_QUERIES_JWT_SECRET%20%3D%20%22your-secret-key%22%0AGLOBAL_ASYNC_QUERIES_JWT_COOKIE_NAME%20%3D%20%22async-token%22%0AGLOBAL_ASYNC_QUERIES_REDIS_STREAM_PREFIX%20%3D%20%22async-events-%22%0A%60%60%60%0AIn%20%60superset-websocket/config.json%60%3A%0A%0A%60%60%60json%0A%7B%0A%20%20%22port%22%3A%208080%2C%0A%20%20%22redis%22%3A%20%7B%0A%20%20%20%20%22port%22%3A%206379%2C%0A%20%20%20%20%22host%22%3A%20%22127.0.0.1%22%2C%0A%20%20%20%20%22db%22%3A%202%2C%0A%20%20%20%20%22ssl%22%3A%20false%0A%20%20%7D%2C%0A%20%20%22redisStreamPrefix%22%3A%20%22async-events-%22%2C%0A%20%20%22jwtSecret%22%3A%20%22your-secret-key%22%2C%0A%20%20%22jwtCookieName%22%3A%20%22async-token%22%0A%7D%0A%60%60%60%0AMake%20sure%20the%20JWT%20secret%2C%20cookie%20name%2C%20Redis%20details%2C%20and%20stream%20prefix%20match%20in%20both%20configs.%20The%20hostname%20must%20match%20how%20you%20access%20Superset%20in%20your%20browser%20for%20cookies%20to%20work.%20
Both%20the%20Flask%20server%20and%20superset-websocket%20server%20must%20be%20running%20simultaneously%20for%20async%20features%20to%20work%20%5Bsource%5D%28https%3A//github.com/apache/superset/issues/36069%29.%0A%0A%2A%2ACustom%20Message%20Format%20Implementation%2A%2A%0A%0ASuperset%E2%80%99s%20async%20event%20messages%20are%20built%20in%20the%20backend%20%28see%20%60async_query_manager.py%60%29.%20The%20event%20data%20dict%20is%20constructed%20using%20%60build_job_metadata%60%20and%20updated%20via%20%60update_job%60.%20You%20can%20add%20custom%20fields%20%28like%20request_id%2C%20type%2C%20axis%2C%20etc.%29%20by%20passing%20them%20as%20keyword%20arguments%20to%20%60update_job%60.%20These%20fields%20will%20be%20included%20in%20the%20JSON%20payload%20written%20to%20Redis%20and%20forwarded%20by%20superset-websocket%20to%20clients%20%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/async_events/async_query_manager.py%23L53-L308%
29.%0A%0A%2A%2AExample%3A%20Custom%20Event%20Payload%2A%2A%0A%0AHere%E2%80%99s%20what%20a%20custom%20event%20payload%20might%20look%20like%3A%0A%0A%60%60%60json%0A%7B%0A%20%20%22id%22%3A%20%221615426152415-0%22%2C%20%20%20%20%20%20%20%20%20%20%20//%20Redis%20stream%20ID%0A%20%20%22channel_id%22%3A%20%22abc123%22%2C%0A%20%20%22job_id%22%3A%20%22job-uuid%22%2C%0A%20%20%22user_id%22%3A%20%221%22%2C%0A%20%20%22request_id%22%3A%20%22req-uuid%22%2C%0A%20%20%22type%22%3A%20%22query%22%2C%0A%20%20%22axis%22%3A%20%22x%22%2C%0A%20%20%22level%22%3A%20%22detail%22%2C%0A%20%20%22offset%22%3A%200%2C%0A%20%20%22limit%22%3A%20100%2C%0A%20%20%22payload%22%3A%20%7B%20%22data%22%3A%20%5B...%5D%20%7D%2C%0A%20%20%22metrics_meta%22%3A%20%7B%20%22duration_ms%22%3A%20123%20%7D%2C%0A%20%20%22error%22%3A%20null%2C%0A%20%20%22status%22%3A%20%22done%22%2C%0A%20%20%22result_url%22%3A%20%22/api/v1/chart/data/job-uuid%22%0A%7D%0A%60%60%60%0A%0ATo%20implement%20this%2C%20update%20your%20async%20query%20Celery%20ta
sks%20%28e.g.%2C%20in%20%60async_queries.py%60%29%20to%20pass%20these%20fields%20to%20%60async_query_manager.update_job%60.%20For%20example%3A%0A%0A%60%60%60python%0Aasync_query_manager.update_job%28%0A%20%20%20%20job_metadata%2C%0A%20%20%20%20async_query_manager.STATUS_DONE%2C%0A%20%20%20%20request_id%3D%22req-uuid%22%2C%0A%20%20%20%20type%3D%22query%22%2C%0A%20%20%20%20axis%3D%22x%22%2C%0A%20%20%20%20level%3D%22detail%22%2C%0A%20%20%20%20offset%3D0%2C%0A%20%20%20%20limit%3D100%2C%0A%20%20%20%20payload%3Dpayload%2C%0A%20%20%20%20metrics_meta%3D%7B%22duration_ms%22%3A%20123%7D%2C%0A%20%20%20%20error%3DNone%2C%0A%20%20%20%20result_url%3Dresult_url%2C%0A%29%0A%60%60%60%0AAny%20fields%20you%20add%20here%20will%20be%20included%20in%20the%20event%20data%20sent%20to%20Redis%20and%20delivered%20to%20the%20frontend%20via%20WebSocket%20%5Bsource%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/tasks/async_queries.py%23L82-L166%29.%0A%0A%2A%2ABe
st%20Practices%20%26%20Caveats%2A%2A%0A%0A-%20There%E2%80%99s%20no%20explicit%20config%20option%20for%20message%20format%20in%20superset-websocket%3B%20you%20control%20the%20payload%20in%20the%20backend%20code.%0A-%20Make%20sure%20any%20consumers%20%28frontend%2C%20other%20services%29%20expect%20and%20handle%20your%20custom%20fields.%0A-%20The%20error%20field%20is%20typically%20a%20list%20of%20error%20dicts%2C%20populated%20if%20exceptions%20occur%20during%20async%20query%20execution.%0A-%20There%E2%80%99s%20no%20enforced%20payload%20size%20limit%20in%20superset-websocket%2C%20but%20you%20can%20set%20limits%20in%20the%20backend%20%28e.g.%2C%20%60SQLLAB_PAYLOAD_MAX_MB%60%20for%20SQL%20Lab%29%20%5Bsource%5D%28https%3A//github.com/apache/superset/pull/30522%29.%0A%0ALet%20me%20know%20if%20you%20want%20code%20snippets%20for%20a%20specific%20async%20query%20type%20or%20more%20details%20on%20frontend%20handling%21) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/36383)
GitHub link:
https://github.com/apache/superset/discussions/36383#discussioncomment-15146911
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
