jscheffl opened a new issue, #49487: URL: https://github.com/apache/airflow/issues/49487
### Apache Airflow version
3.0.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
I am attempting to upgrade our 2.10.5 to 3.0.0rc4 and migration fails with:
```
[2025-04-20T21:55:25.758+0000] {migration.py:618} INFO - Running upgrade
d03e4a635aa3 -> 2b47dc6bc8df, add dag versioning.
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 1910, in _execute_context
self.dialect.do_execute(
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py",
line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.NotNullViolation: column "dag_id" of relation "dag_code"
contains null values
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line
55, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py",
line 48, in command
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line
111, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py",
line 55, in wrapped_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/db_command.py",
line 197, in migratedb
run_db_migrate_command(args, db.upgradedb, _REVISION_HEADS_MAP)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/db_command.py",
line 125, in run_db_migrate_command
command(
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py",
line 101, in wrapper
return func(*args, session=session, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/db.py", line
1142, in upgradedb
command.upgrade(config, revision=to_revision or "heads")
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/command.py", line
408, in upgrade
script.run_env()
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/script/base.py",
line 586, in run_env
util.load_python_file(self.dir, "env.py")
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/util/pyfiles.py",
line 95, in load_python_file
module = load_module_py(module_id, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/util/pyfiles.py",
line 113, in load_module_py
spec.loader.exec_module(module) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in
_call_with_frames_removed
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/migrations/env.py",
line 138, in <module>
run_migrations_online()
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/migrations/env.py",
line 132, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/runtime/environment.py",
line 946, in run_migrations
self.get_context().run_migrations(**kw)
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/runtime/migration.py",
line 623, in run_migrations
step.migration_fn(**kw)
File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py",
line 238, in upgrade
with op.batch_alter_table("dag_code") as batch_op:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/contextlib.py", line 144, in __exit__
next(self.gen)
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/operations/base.py",
line 397, in batch_alter_table
impl.flush()
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/operations/batch.py",
line 115, in flush
fn(*arg, **kw)
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/ddl/postgresql.py",
line 188, in alter_column
super().alter_column(
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/ddl/impl.py", line
280, in alter_column
self._exec(
File
"/home/airflow/.local/lib/python3.12/site-packages/alembic/ddl/impl.py", line
246, in _exec
return conn.execute(construct, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/future/engine.py",
line 286, in execute
return self._execute_20(
^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 1710, in _execute_20
return meth(self, args_10style, kwargs_10style, execution_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line
80, in _execute_on_connection
return connection._execute_ddl(
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 1477, in _execute_ddl
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 1953, in _execute_context
self._handle_dbapi_exception(
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 2134, in _handle_dbapi_exception
util.raise_(
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/util/compat.py",
line 211, in raise_
raise exception
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py",
line 1910, in _execute_context
self.dialect.do_execute(
File
"/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py",
line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.errors.NotNullViolation) column
"dag_id" of relation "dag_code" contains null values
[SQL: ALTER TABLE dag_code ALTER COLUMN dag_id SET NOT NULL]
(Background on this error at: https://sqlalche.me/e/14/gkpj)
```
### What you think should happen instead?
Migration should not fail or report an error what the problem is.
### How to reproduce
Happens in migration env... unfortunately no diagnostic information printed
to understand where it fails.
### Operating System
Kubernetes
### Versions of Apache Airflow Providers
not relevant
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Helm chart 1.16.0 on K8s
Deployment based on extended Docker image from 3.0.0rc4, Python 3.12
### Anything else?
Deployment was on Airflow 2.10.5 before.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
