Mottimo opened a new issue, #24247:
URL: https://github.com/apache/airflow/issues/24247
### Apache Airflow version
2.3.1
### What happened
The execution of of the command "airflow db upgrade" to upgrade the version
from 2.2.5 to 2.3.0 or 2.3.1 fails with the following error:
(airflow) [airflow@rabbit01 airflow]$ export
LD_LIBRARY_PATH=/opt/python-bit/lib:$LD_LIBRARY_PATH
(airflow) [airflow@rabbit01 airflow]$ airflow db upgrade
DB: mysql+mysqldb://airflow:***@localdb.xyste.it:3306/airflow
Performing upgrade with database
mysql+mysqldb://airflow:***@localdb.localdomain.it:3306/airflow
[2022-06-06 10:18:48,523] {db.py:1445} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl MySQLImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 45ba3f1493b9 ->
849da589634d, Prefix DAG permissions.
Traceback (most recent call last):
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1783, in _cursor_execute
self.dialect.do_execute(cursor, statement, parameters, context)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 716, in do_execute
cursor.execute(statement, parameters)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/cursors.py",
line 206, in execute
res = self._query(query)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/cursors.py",
line 319, in _query
db.query(q)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/connections.py",
line 254, in query
_mysql.connection.query(self, query)
**MySQLdb._exceptions.ProgrammingError: (1146, "Table
'airflow.ab_permission_id_seq' doesn't exist")**
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1608, in _execute_context
context = constructor(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 998, in _init_compiled
self._process_executesingle_defaults()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1866, in _process_executesingle_defaults
val = self.get_insert_default(c)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1811, in get_insert_default
return self._exec_default(column, column.default, column.type)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1672, in _exec_default
return self.fire_sequence(default, type_)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py",
line 1401, in fire_sequence
return self._execute_scalar(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1269, in _execute_scalar
conn._cursor_execute(self.cursor, stmt, parameters, context=self)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1785, in _cursor_execute
self._handle_dbapi_exception(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1929, in _handle_dbapi_exception
util.raise_(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1783, in _cursor_execute
self.dialect.do_execute(cursor, statement, parameters, context)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 716, in do_execute
cursor.execute(statement, parameters)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/cursors.py",
line 206, in execute
res = self._query(query)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/cursors.py",
line 319, in _query
db.query(q)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/connections.py",
line 254, in query
_mysql.connection.query(self, query)
**sqlalchemy.exc.ProgrammingError: (MySQLdb._exceptions.ProgrammingError)
(1146, "Table 'airflow.ab_permission_id_seq' doesn't exist")**
[SQL: select nextval(ab_permission_id_seq)]
(Background on this error at: http://sqlalche.me/e/14/f405)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/x01/airflow/environments/airflow/bin/airflow", line 8, in
<module>
sys.exit(main())
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/__main__.py",
line 38, in main
args.func(args)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/cli/cli_parser.py",
line 51, in command
return func(*args, **kwargs)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/utils/cli.py",
line 99, in wrapper
return f(*args, **kwargs)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/cli/commands/db_command.py",
line 82, in upgradedb
db.upgradedb(to_revision=to_revision, from_revision=from_revision,
show_sql_only=args.show_sql_only)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/utils/session.py",
line 77, in wrapper
return func(*args, session=session, **kwargs)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/utils/db.py",
line 1446, in upgradedb
command.upgrade(config, revision=to_revision or 'heads')
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/alembic/command.py",
line 320, in upgrade
script.run_env()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/alembic/script/base.py",
line 563, in run_env
util.load_python_file(self.dir, "env.py")
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/alembic/util/pyfiles.py",
line 92, in load_python_file
module = load_module_py(module_id, path)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/alembic/util/pyfiles.py",
line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/migrations/env.py",
line 107, in <module>
run_migrations_online()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/migrations/env.py",
line 101, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/alembic/runtime/environment.py",
line 851, in run_migrations
self.get_context().run_migrations(**kw)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/alembic/runtime/migration.py",
line 620, in run_migrations
step.migration_fn(**kw)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/migrations/versions/0073_2_0_0_prefix_dag_permissions.py",
line 231, in upgrade
migrate_to_new_dag_permissions(db)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/migrations/versions/0073_2_0_0_prefix_dag_permissions.py",
line 161, in migrate_to_new_dag_permissions
can_read_action = get_or_create_action(db.session, 'can_read')
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/airflow/migrations/versions/0073_2_0_0_prefix_dag_permissions.py",
line 109, in get_or_create_action
session.commit()
File "<string>", line 2, in commit
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/session.py",
line 1423, in commit
self._transaction.commit(_to_root=self.future)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/session.py",
line 829, in commit
self._prepare_impl()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/session.py",
line 808, in _prepare_impl
self.session.flush()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/session.py",
line 3255, in flush
self._flush(objects)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/session.py",
line 3395, in _flush
transaction.rollback(_capture_exception=True)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py",
line 70, in __exit__
compat.raise_(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/session.py",
line 3355, in _flush
flush_context.execute()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py",
line 453, in execute
rec.execute(self)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/unitofwork.py",
line 627, in execute
util.preloaded.orm_persistence.save_obj(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py",
line 242, in save_obj
_emit_insert_statements(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/orm/persistence.py",
line 1219, in _emit_insert_statements
result = connection._execute_20(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1520, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/sql/elements.py",
line 313, in _execute_on_connection
return connection._execute_clauseelement(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1389, in _execute_clauseelement
ret = self._execute_context(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1614, in _execute_context
self._handle_dbapi_exception(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1929, in _handle_dbapi_exception
util.raise_(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1608, in _execute_context
context = constructor(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 998, in _init_compiled
self._process_executesingle_defaults()
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1866, in _process_executesingle_defaults
val = self.get_insert_default(c)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1811, in get_insert_default
return self._exec_default(column, column.default, column.type)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1672, in _exec_default
return self.fire_sequence(default, type_)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/dialects/mysql/base.py",
line 1401, in fire_sequence
return self._execute_scalar(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 1269, in _execute_scalar
conn._cursor_execute(self.cursor, stmt, parameters, context=self)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1785, in _cursor_execute
self._handle_dbapi_exception(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1929, in _handle_dbapi_exception
util.raise_(
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/base.py",
line 1783, in _cursor_execute
self.dialect.do_execute(cursor, statement, parameters, context)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/sqlalchemy/engine/default.py",
line 716, in do_execute
cursor.execute(statement, parameters)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/cursors.py",
line 206, in execute
res = self._query(query)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/cursors.py",
line 319, in _query
db.query(q)
File
"/app/x01/airflow/environments/airflow/lib/python3.8/site-packages/MySQLdb/connections.py",
line 254, in query
_mysql.connection.query(self, query)
**sqlalchemy.exc.StatementError: (sqlalchemy.exc.ProgrammingError)
(MySQLdb._exceptions.ProgrammingError) (1146, "Table
'airflow.ab_permission_id_seq' doesn't exist")**
It seems that the upgrade looks for a table that is not present and not
created by the upgrade itself. The same error is returned when the command
"airflow db init" is executed on a clean environment.
### What you think should happen instead
The db upgrade should create first the missing table and any other related
(and missing) object.
### How to reproduce
Just running the command using a MariaDB (or Mysql maybe) database.
### Operating System
CentOS Linux release 7.9.2009
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==3.4.0
apache-airflow-providers-celery==2.1.4
apache-airflow-providers-elasticsearch==3.0.3
apache-airflow-providers-ftp==2.1.2
apache-airflow-providers-http==2.1.2
apache-airflow-providers-imap==2.2.3
apache-airflow-providers-jdbc==2.1.3
apache-airflow-providers-jenkins==2.1.0
apache-airflow-providers-jira==2.0.5
apache-airflow-providers-mysql==2.2.3
apache-airflow-providers-odbc==2.0.4
apache-airflow-providers-oracle==2.2.3
apache-airflow-providers-postgres==4.1.0
apache-airflow-providers-redis==2.0.4
apache-airflow-providers-samba==3.0.4
apache-airflow-providers-sftp==2.6.0
apache-airflow-providers-sqlite==2.1.3
apache-airflow-providers-ssh==2.4.4
### Deployment
Virtualenv installation
### Deployment details
Virtualenv based on Python 3.8.7
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
