villebro commented on issue #29515: URL: https://github.com/apache/superset/issues/29515#issuecomment-2226811598
A quick summary from the discussion on this SIP in the Town Hall meeting on 2024-07-12: - The issue of query deduplication, and inability to manage running queries, is confirmed to be a major issue for deployments running at scale. By being able to move query execution from the webservers into async workers we stand to gain a lot in terms of infra efficiency, performance and general usability. - On the topic of polling vs WebSockets (WS), there was hesitation to decommission the existing WS server, as WS will likely be very useful for future features, too. For this reason it was decided to attempt to simplify the architecture by removing query context keys in favor of the normal cache keys, and in general making the codebase more approachable, while still retaining the WS server and support for it in the main backend. The SIP will shortly be updated to reflect these changes. Furthermore, a weekly meeting has been setup for coordinating the effort to redesign the WS implementation to bring the core features and changes proposed in this SIP, most notably query deduplication, management UI for queued/running queries, and architectural simplification. The meeting will take place on Mondays at noon PST (all interested are welcome to join!) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
