villebro opened a new pull request, #20073: URL: https://github.com/apache/superset/pull/20073
### SUMMARY When running the migration `a9422eeaae74_new_dataset_models_take_2.py` from #19421 I ran into a problem where some old database metadata caused the migration to fail. This adds a check for the truthiness of `drivername` similar to what's done a few rows above https://github.com/apache/superset/blob/1c3c3ae2ea1541b3719f303dd211e159adee3792/superset/migrations/versions/a9422eeaae74_new_dataset_models_take_2.py#L577 to ensure that the the migration completes even if the metadata corrupted. ### BEFORE ``` Traceback (most recent call last): File "/Users/ville/src/superset/venv/bin/superset", line 33, in <module> sys.exit(load_entry_point('apache-superset', 'console_scripts', 'superset')()) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 1128, in __call__ return self.main(*args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/flask/cli.py", line 601, in main return super().main(*args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 1659, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/flask/cli.py", line 445, in decorator return __ctx.invoke(f, *args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/click/core.py", line 754, in invoke return __callback(*args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/flask_migrate/cli.py", line 149, in upgrade _upgrade(directory, revision, sql, tag, x_arg) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 98, in wrapped f(*args, **kwargs) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 185, in upgrade command.upgrade(config, revision, sql=sql, tag=tag) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/alembic/command.py", line 294, in upgrade script.run_env() File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/alembic/script/base.py", line 490, in run_env util.load_python_file(self.dir, "env.py") File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file module = load_module_py(module_id, path) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py spec.loader.exec_module(module) File "<frozen importlib._bootstrap_external>", line 843, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/Users/ville/src/superset/superset/extensions/../migrations/env.py", line 126, in <module> run_migrations_online() File "/Users/ville/src/superset/superset/extensions/../migrations/env.py", line 118, in run_migrations_online context.run_migrations() File "<string>", line 8, in run_migrations File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/alembic/runtime/environment.py", line 813, in run_migrations self.get_context().run_migrations(**kw) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/alembic/runtime/migration.py", line 561, in run_migrations step.migration_fn(**kw) File "/Users/ville/src/superset/superset/migrations/versions/a9422eeaae74_new_dataset_models_take_2.py", line 885, in upgrade postprocess_datasets(session) File "/Users/ville/src/superset/superset/migrations/versions/a9422eeaae74_new_dataset_models_take_2.py", line 603, in postprocess_datasets expression, get_dialect_name(drivername), show_warning=False File "/Users/ville/src/superset/superset/utils/memoized.py", line 50, in __call__ value = self.func(*args, **kwargs) File "/Users/ville/src/superset/superset/connectors/sqla/utils.py", line 185, in get_dialect_name return SqlaURL(drivername).get_dialect().name File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 189, in get_dialect entrypoint = self._get_entrypoint() File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/sqlalchemy/engine/url.py", line 172, in _get_entrypoint cls = registry.load(name) File "/Users/ville/src/superset/venv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 277, in load raise exc.NoSuchModuleError( sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects: ``` ### AFTER ``` INFO [alembic.runtime.migration] Running upgrade ad07e4fdbaba -> a9422eeaae74, new_dataset_models_take_2 >> Copy 35 physical tables to sl_tables... >> Copy 91 SqlaTable to sl_datasets... Copy dataset owners... Link physical datasets with tables... >> Copy 1,033 table columns to sl_columns... Link all columns to sl_datasets... >> Copy 101 metrics to sl_columns... Link metric columns to datasets... >> Run postprocessing on 1,134 columns [Column 1 to 1,134] 1,011 may be updated Assign table column relations... >> Run postprocessing on 91 datasets Process dataset 1~91... >> Assign new UUIDs to tables... Adding uuids for `sl_tables`... Done. Assigned 34 uuids in 0.012s. >> Drop intermediate columns... INFO [alembic.runtime.migration] Running upgrade a9422eeaae74 -> cbe71abde154, fix report schedule and execution log Revision ID: 6f139c533bea Revises: cbe71abde154 Create Date: 2021-05-27 16:10:59.567684 ``` ### TESTING INSTRUCTIONS <!--- Required! What steps can be taken to manually verify the changes? --> ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
