GitHub user dosubot[bot] added a comment to the discussion: Superset db upgrade is not triggering upgrade
<!-- Answer --> There’s no official benchmark for upgrading Superset with 800 charts and 16,000 dashboards, but migration time can be significant at this scale. Superset migrations often process every dashboard and chart individually in Python, transforming JSON metadata and committing changes for each record. This means the time required grows with the number of dashboards and charts, and can range from several minutes to hours depending on your database performance and hardware resources. Some migration scripts even commit after each dashboard, which adds overhead for large datasets[^1][^2][^3]. To estimate timing more precisely, you can use the benchmark_migration.py script in a test environment with similar data volume—it measures migration time for different entity counts[^1]. While your query on pg_stat_activity is a good way to monitor active migrations, you can also check the migration logs for progress output (many scripts print which dashboard or chart is being processed). For large upgrades, always back up your metadata database and, if possible, run the upgrade during a maintenance window to minimize impact. [^1]: [benchmark_migration.py script](https://github.com/apache/superset/blob/master/scripts/benchmark_migration.py) [^2]: [Example dashboard migration script](https://github.com/apache/superset/blob/master/superset/migrations/versions/2018-07-22_11-59_bebcf3fed1fe_convert_dashboard_v1_positions.py) [^3]: [Another dashboard migration example](https://github.com/apache/superset/blob/master/superset/migrations/versions/2018-02-13_08-07_e866bd2d4976_smaller_grid.py) <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/46ef1be1-0d55-4df8-b71e-5b73ca3df799?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=There%E2%80%99s%20no%20official%20benchmark%20for%20upgrading%20Superset%20with%20800%20charts%20and%2016%2C000%20dashboards%2C%20but%20migration%20time%20can%20be%20significant%20at%20this%20scale.%20Superset%20migrations%20often%20process%20every%20dashboard%20and%20chart%20individually%20in%20Python%2C%20transforming%20JSON%20metadata%20and%20committing%20changes%20for%20each%20record.%20This%20means%20the%20time%20required%20grows%20with%20the%20number%20of%20dashboards%20and%20charts%2C%20and%20can%20range%20from%20several%20minutes%20to%20hours%20depending%20on%20your%20database%20performance%20and%20hardware%20resources.%20Some%20migration%20scripts%20even%20commit%20after%20each%20dashboard%2C%20which%20adds%20overhead%20for%20large%20datasets %5B%5E1%5D%5B%5E2%5D%5B%5E3%5D.%0A%0ATo%20estimate%20timing%20more%20precisely%2C%20you%20can%20use%20the%20benchmark_migration.py%20script%20in%20a%20test%20environment%20with%20similar%20data%20volume%E2%80%94it%20measures%20migration%20time%20for%20different%20entity%20counts%5B%5E1%5D.%20While%20your%20query%20on%20pg_stat_activity%20is%20a%20good%20way%20to%20monitor%20active%20migrations%2C%20you%20can%20also%20check%20the%20migration%20logs%20for%20progress%20output%20%28many%20scripts%20print%20which%20dashboard%20or%20chart%20is%20being%20processed%29.%0A%0AFor%20large%20upgrades%2C%20always%20back%20up%20your%20metadata%20database%20and%2C%20if%20possible%2C%20run%20the%20upgrade%20during%20a%20maintenance%20window%20to%20minimize%20impact.%0A%0A%5B%5E1%5D%3A%20%5Bbenchmark_migration.py%20script%5D%28https%3A//github.com/apache/superset/blob/master/scripts/benchmark_migration.py%29%0A%5B%5E2%5D%3A%20%5BExample%20dashboard%20migration%20script%5D%28https%3A//github.com/apac he/superset/blob/master/superset/migrations/versions/2018-07-22_11-59_bebcf3fed1fe_convert_dashboard_v1_positions.py%29%0A%5B%5E3%5D%3A%20%5BAnother%20dashboard%20migration%20example%5D%28https%3A//github.com/apache/superset/blob/master/superset/migrations/versions/2018-02-13_08-07_e866bd2d4976_smaller_grid.py%29) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/discussions/37261) GitHub link: https://github.com/apache/superset/discussions/37261#discussioncomment-15544876 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
