justinpark opened a new issue, #21385: URL: https://github.com/apache/superset/issues/21385
## [SIP-88] Proposal for async sqlLab persistence ### Motivation At Airbnb we have a performance (lag) issue using `SQLLAB_BACKEND_PERSISTENCE` feature. Since `SQLLAB_BACKEND_PERSISTENCE` designed in a blocking way (as shown in the following), user can be stuck while `tab_stat` table in busy state. (even unable to run a query while the `tabstateview` endpoint timed out) https://github.com/apache/superset/blob/8c2719b11cebe451c5ba68193cf8dd51e4dce8e3/superset-frontend/src/SqlLab/actions/sqlLab.js#L614-L625 In order to improve this lag issue, we'd like to have a non-blocking mechanism for this db syncing workflow. i.e. like g-doc auto saving. moreover we'd like to design a hybrid solution (i.e. keep using redux-localstorage and save same snaphot in remote) to keep the advantage (offline saving) of using local storage. ### Proposed Change The proposed solution is simple. We would like to keep using redux-localstorage as well as the current localStorage mechanism (as SQLLAB_BACKEND_PERSISTENCE turned OFF). Like QueryAutoSync does, we would like to build a component that observes the changes of queryEditor and then periodically sync the updated queryEditor snapshot from redux state using existing `/tabstateview/` api. For the payload wise, we'd like to use `extra_json` in `tab_stat` table to store the snapshot(by json format) of queryEditor as stored in localStorage (rather than mapping each value in a proper table column like sql, label, query_limit). As existing persistence feature does, we will design similar hydration flow in frontend to initialize the sqlLab state from the backend. With this json blob it will simply convert to json and then merge with the existing local state. ### New or Changed Public Interfaces We will introduce new `SQLLAB_BACKEND_PERSISTENCE_ASYNC` FLAG and need to pass the `tab_state_ids` value including `extra_json` in this case. ### New dependencies I do not for-see any new dependencies. ### Migration Plan and Compatibility I do not for-see any migration required. ### Rejected Alternatives None. to: @etr2460 @ktmud @john-bodley -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
