FromTheCosmos opened a new issue #17059:
URL: https://github.com/apache/superset/issues/17059
When upgrading from 0.39 superset on Ubuntu 18.04 and upgrading to Superset
1.3.0 on Ubuntu 20.04, there is a problem with data being moved from one
superset instance to another.
#### How to reproduce the bug
1. Copy `superset.db` from Ubuntu 18.04 system
2. Move a copy of `superset.db` to the Ubuntu 20.04 instance with superset
1.3.0 in the `etc/superset/.superset/superset.db` location
3. Activate the virtual environment and run `superset db upgrade` and
`superset init`
4. The `superset db upgrade` finishes without error, however the `superset
init` command does not complete due to decryption key error.
```
$ . /etc/superset/bin/activate
(superset) $ superset db upgrade
logging was configured successfully
2021-10-11 18:34:46,763:INFO:superset.utils.logging_configurator:logging was
configured successfully
2021-10-11 18:34:46,829:INFO:root:Configured event logger of type <class
'superset.utils.log.DBEventLogger'>
/etc/superset/lib/python3.8/site-packages/flask_caching/__init__.py:201:
UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively
disabled.
warnings.warn(
No PIL installation found
2021-10-11 18:34:48,917:INFO:superset.utils.screenshots:No PIL installation
found
WARNI [alembic.env] SQLite Database support for metadata databases will
be removed in a future version of Superset.
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
(superset) $ superset init
logging was configured successfully
2021-10-11 18:35:56,758:INFO:superset.utils.logging_configurator:logging was
configured successfully
2021-10-11 18:35:57,082:INFO:root:Configured event logger of type <class
'superset.utils.log.DBEventLogger'>
/etc/superset/lib/python3.8/site-packages/flask_caching/__init__.py:201:
UserWarning: Flask-Caching: CACHE_TYPE is set to null, caching is effectively
disabled.
warnings.warn(
No PIL installation found
2021-10-11 18:35:58,187:INFO:superset.utils.screenshots:No PIL installation
found
Syncing role definition
2021-10-11 18:36:00,085:INFO:superset.security.manager:Syncing role
definition
Syncing Admin perms
2021-10-11 18:36:00,096:INFO:superset.security.manager:Syncing Admin perms
Syncing Alpha perms
2021-10-11 18:36:00,290:INFO:superset.security.manager:Syncing Alpha perms
Syncing Gamma perms
2021-10-11 18:36:00,372:INFO:superset.security.manager:Syncing Gamma perms
Syncing granter perms
2021-10-11 18:36:00,453:INFO:superset.security.manager:Syncing granter perms
Syncing sql_lab perms
2021-10-11 18:36:00,534:INFO:superset.security.manager:Syncing sql_lab perms
Fetching a set of all perms to lookup which ones are missing
2021-10-11 18:36:00,612:INFO:superset.security.manager:Fetching a set of all
perms to lookup which ones are missing
Creating missing datasource permissions.
2021-10-11 18:36:00,686:INFO:superset.security.manager:Creating missing
datasource permissions.
Traceback (most recent call last):
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py",
line 126, in decrypt
decrypted = decrypted.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 1:
invalid continuation byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/etc/superset/bin/superset", line 8, in <module>
sys.exit(superset())
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 829,
in __call__
return self.main(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/flask/cli.py", line 586,
in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 782,
in main
rv = self.invoke(ctx)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 1259,
in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 1066,
in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 610,
in invoke
return callback(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/decorators.py", line
21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/flask/cli.py", line 426,
in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 610,
in invoke
return callback(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/decorators.py", line
21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/flask/cli.py", line 426,
in decorator
return __ctx.invoke(f, *args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/click/core.py", line 610,
in invoke
return callback(*args, **kwargs)
File "/etc/superset/lib/python3.8/site-packages/superset/cli.py", line 90,
in init
security_manager.sync_role_definitions()
File
"/etc/superset/lib/python3.8/site-packages/superset/security/manager.py", line
704, in sync_role_definitions
self.create_missing_perms()
File
"/etc/superset/lib/python3.8/site-packages/superset/security/manager.py", line
648, in create_missing_perms
merge_pv("datasource_access", datasource.get_perm())
File
"/etc/superset/lib/python3.8/site-packages/superset/connectors/sqla/models.py",
line 621, in get_perm
return f"[{self.database}].[{self.table_name}](id:{self.id})"
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line
294, in __get__
return self.impl.get(instance_state(instance), dict_)
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line
730, in get
value = self.callable_(state, passive)
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line
759, in _load_for_state
return self._emit_lazyload(
File "<string>", line 1, in <lambda>
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line
847, in _emit_lazyload
q(session)
File "/etc/superset/lib/python3.8/site-packages/sqlalchemy/ext/baked.py",
line 615, in _load_on_pk_identity
result = list(bq.for_session(self.session).params(**params))
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line
100, in instances
cursor.close()
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py",
line 68, in __exit__
compat.raise_(
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line
182, in raise_
raise exception
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80,
in instances
rows = [proc(row) for row in fetch]
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 80,
in <listcomp>
rows = [proc(row) for row in fetch]
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line
579, in _instance
_populate_full(
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line
725, in _populate_full
dict_[key] = getter(row)
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy/sql/type_api.py", line
1278, in process
return process_value(impl_processor(value), dialect)
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py",
line 469, in process_result_value
value = super().process_result_value(value=value, dialect=dialect)
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py",
line 414, in process_result_value
decrypted_value = self.engine.decrypt(value)
File
"/etc/superset/lib/python3.8/site-packages/sqlalchemy_utils/types/encrypted/encrypted_type.py",
line 128, in decrypt
raise ValueError('Invalid decryption key')
ValueError: Invalid decryption key
```
### Expected results
I expect superset to be able to upgrade and use `superset.db` for the init
without it prompting error.
### Actual results
Invalid decryption key error appears.
### Environment
- superset version: `1.3.0`
- python version: `Python 3.8.10`
- ubuntu 20.04
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included
it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version
of superset.
- [x] I have checked the issue tracker for the same issue and I haven't
found one similar. (There are similar ones, but no solution)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
