GitHub user fowlerp397-tech added a comment to the discussion: Slow saving changes on exist dataset
Superset can be slow with virtual datasets due to complex queries, large data volumes, lack of caching, or resource constraints. Try optimizing queries, enabling caching, and checking backend performance for better speed. On Tue, Mar 31, 2026, 8:49 AM dosubot[bot] ***@***.***> wrote: > This is a well-documented pain point. When you save a virtual dataset, > Superset runs a *two-stage synchronous pipeline* [1] > <https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/datasets/api.py#L429-L431> > : > > 1. *UpdateDatasetCommand* — saves your metadata changes to the > Superset DB (with several validation queries for column/metric uniqueness, > existence, etc.) > 2. *RefreshDatasetCommand* — *executes your dataset's SQL query* (with > LIMIT > 1) against the underlying data warehouse to sync column metadata [2] > > <https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/commands/dataset/refresh.py#L44-L48> > > That second step is the main bottleneck: the HTTP request *blocks* until > your warehouse responds. If you're connected to something like Databricks, > Snowflake, or BigQuery with cold-start or query compilation overhead, even > a trivial save can take a long time [3] > <https://github.com/apache/superset/issues/38012#issuecomment-3908177644>. > There's currently no async offloading for this step. > > A few other factors that can compound the slowness: > > - *Large number of columns/metrics* — the update path loads the full > column and metric collections into memory for diffing before persisting > changes [4] > > <https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/daos/dataset.py#L247> > . > - *Frontend freeze with 100+ metrics* — if the editor itself is > slow/frozen, PR #37555 <https://github.com/apache/superset/pull/37555> > added pagination to the dataset editor tables to fix a 6+ minute browser > hang [5] <https://github.com/apache/superset/pull/37555>. Make sure > you're on a version that includes this fix. > > *What you can try:* > > - *Increase gunicorn workers* so that one blocked save doesn't starve > other requests. > - *Check your underlying database performance* — the save speed is > directly tied to how fast that warehouse can execute a SELECT ... > LIMIT 1 version of your dataset SQL. > - *Upgrade Superset* if you're on an older version — several > performance fixes have landed (consolidated double PUT calls in #33523 > <https://github.com/apache/superset/pull/33523>, N+1 fix in #37899 > <https://github.com/apache/superset/pull/37899>, editor pagination in > #37555 <https://github.com/apache/superset/pull/37555>). > > *To reply, just mention @dosu > <https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=brand-link>.* > ------------------------------ > > How did I do? Good > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=great_response&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-great_response> > | Irrelevant > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=irrelevant_answer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-irrelevant_answer> > | Incorrect > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=incorrect_sources&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-incorrect_sources> > | Verbose > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=too_verbose&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-too_verbose> > | Hallucination > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=hallucination&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-hallucination> > | Report 🐛 > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=bug_report&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-bug_report> > | Other > <https://app.dosu.dev/response-feedback/d08910e7-8175-445c-9f86-ecdb9046bddd?feedback_type=other&utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=feedback-other> > > [image: Chat with Dosu] > <https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=chat-badge> > [image: Open in Cursor] > <https://cursor.com/link/prompt?text=This%20is%20a%20well-documented%20pain%20point.%20When%20you%20save%20a%20virtual%20dataset%2C%20Superset%20runs%20a%20%2A%2Atwo-stage%20synchronous%20pipeline%2A%2A%20%5B%5B1%5D%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/datasets/api.py%23L429-L431%29%3A%0A%0A1.%20%2A%2A%60UpdateDatasetCommand%60%2A%2A%20%E2%80%94%20saves%20your%20metadata%20changes%20to%20the%20Superset%20DB%20%28with%20several%20validation%20queries%20for%20column/metric%20uniqueness%2C%20existence%2C%20etc.%29%0A2.%20%2A%2A%60RefreshDatasetCommand%60%2A%2A%20%E2%80%94%20%2A%2Aexecutes%20your%20dataset%27s%20SQL%20query%2A%2A%20%28with%20%60LIMIT%201%60%29%20against%20the%20underlying%20data%20warehouse%20to%20sync%20column%20metadata%20%5B%5B2%5D%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/commands/dataset/refresh.py%23L44-L48%29%0A%0AThat%20second%20step%20is%20the%20m ain%20bottleneck%3A%20the%20HTTP%20request%20%2A%2Ablocks%2A%2A%20until%20your%20warehouse%20responds.%20If%20you%27re%20connected%20to%20something%20like%20Databricks%2C%20Snowflake%2C%20or%20BigQuery%20with%20cold-start%20or%20query%20compilation%20overhead%2C%20even%20a%20trivial%20save%20can%20take%20a%20long%20time%20%5B%5B3%5D%5D%28https%3A//github.com/apache/superset/issues/38012%23issuecomment-3908177644%29.%20There%27s%20currently%20no%20async%20offloading%20for%20this%20step.%0A%0AA%20few%20other%20factors%20that%20can%20compound%20the%20slowness%3A%0A%0A-%20%2A%2ALarge%20number%20of%20columns/metrics%2A%2A%20%E2%80%94%20the%20update%20path%20loads%20the%20full%20column%20and%20metric%20collections%20into%20memory%20for%20diffing%20before%20persisting%20changes%20%5B%5B4%5D%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/daos/dataset.py%23L247%29.%0A-%20%2A%2AFrontend%20freeze%20with%20100%2B%20metrics%2A%2A%20%E2%80%94%20if %20the%20editor%20itself%20is%20slow/frozen%2C%20%5BPR%20%2337555%5D%28https%3A//github.com/apache/superset/pull/37555%29%20added%20pagination%20to%20the%20dataset%20editor%20tables%20to%20fix%20a%206%2B%20minute%20browser%20hang%20%5B%5B5%5D%5D%28https%3A//github.com/apache/superset/pull/37555%29.%20Make%20sure%20you%27re%20on%20a%20version%20that%20includes%20this%20fix.%0A%0A%2A%2AWhat%20you%20can%20try%3A%2A%2A%0A%0A-%20%2A%2AIncrease%20gunicorn%20workers%2A%2A%20so%20that%20one%20blocked%20save%20doesn%27t%20starve%20other%20requests.%0A-%20%2A%2ACheck%20your%20underlying%20database%20performance%2A%2A%20%E2%80%94%20the%20save%20speed%20is%20directly%20tied%20to%20how%20fast%20that%20warehouse%20can%20execute%20a%20%60SELECT%20...%20LIMIT%201%60%20version%20of%20your%20dataset%20SQL.%0A-%20%2A%2AUpgrade%20Superset%2A%2A%20if%20you%27re%20on%20an%20older%20version%20%E2%80%94%20several%20performance%20fixes%20have%20landed%20%28consolidated%20double%20PUT%20calls%20in%20%5B%2333 523%5D%28https%3A//github.com/apache/superset/pull/33523%29%2C%20N%2B1%20fix%20in%20%5B%2337899%5D%28https%3A//github.com/apache/superset/pull/37899%29%2C%20editor%20pagination%20in%20%5B%2337555%5D%28https%3A//github.com/apache/superset/pull/37555%29%29.> > [image: Join Discord] > <https://go.dosu.dev/discord-bot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-assistant&utm_content=join-discord> > [image: Share on X] > <https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/38975> > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/superset/discussions/38975?email_source=notifications&email_token=B72HASPGXYR3CMSYKMRDFOT4TPEIJA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCNRTHE2TINZZUZZGKYLTN5XKU43VMJZWG4TJMJSWJJLFOZSW45FMMZXW65DFOJPWG3DJMNVQ#discussioncomment-16395479>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/B72HASN4WVUY7IPD5Y6CMRL4TPEIJAVCNFSM6AAAAACXHYRLAWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTMMZZGU2DOOI> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > GitHub link: https://github.com/apache/superset/discussions/38975#discussioncomment-16398050 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
