michael-s-molina opened a new issue, #29801: URL: https://github.com/apache/superset/issues/29801
### Bug description https://github.com/apache/superset/pull/28394 introduced the `upgrade_catalog_perms` which makes a full scan of the `query` table which can contains millions of rows in some organizations. There are two main problems: 1 - We query for query_id, query.database_id for all rows to later set the catalog for each row instead of just submitting an UPDATE statement directly. 2 - This is function is invoked multiple time from different migrations which executes the full scan of the query table multiple times. Ideally, we would try to merge these migrations given how critical the query table is. ### How to reproduce the bug Use `superset db upgrade` to run any of the following migrations which use `upgrade_catalog_perms`: ``` 2024-05-01_10-52_58d051681a3b_add_catalog_perm_to_tables.py 2024-05-08_19-33_4081be5b6b74_enable_catalog_in_databricks.py 2024-05-09_18-44_87ffc36f9842_enable_catalog_in_bigquery_presto_trino_.py ``` ### Screenshots/recordings _No response_ ### Superset version master / latest-dev ### Python version 3.9 ### Node version 16 ### Browser Chrome ### Additional context _No response_ ### Checklist - [X] I have searched Superset docs and Slack and didn't find a solution to my problem. - [X] I have searched the GitHub issue tracker and didn't find a similar bug report. - [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
