ibardarov-fms opened a new issue, #28458:
URL: https://github.com/apache/airflow/issues/28458
### Apache Airflow version
2.5.0
### What happened
I am upgrading from 2.4.1 to 2.50.
When doing the airflow db upgrade I got
```
just airflow-db-upgrade
./airflow.sh db upgrade
[+] Running 2/0
⠿ Container airflow-redis-1 Running
0.0s
⠿ Container airflow-airflow-init-1 Created
0.0s
[+] Running 2/2
⠿ Container airflow-redis-1 Healthy
1.2s
⠿ Container airflow-airflow-init-1 Started
1.6s
DB:
postgresql+psycopg2://airflow_us_east_1:***@analytics-postgres-14-4.cluster-chemdf9ueboq.us-east-1.rds.amazonaws.com:5432/airflow_us_east_1
Performing upgrade with database
postgresql+psycopg2://airflow_us_east_1:***@analytics-postgres-14-4.cluster-chemdf9ueboq.us-east-1.rds.amazonaws.com:5432/airflow_us_east_1
[2022-12-19 09:22:53,278] {migration.py:204} INFO - Context impl
PostgresqlImpl.
[2022-12-19 09:22:53,279] {migration.py:211} INFO - Will assume
transactional DDL.
[2022-12-19 09:22:53,403] {db.py:1570} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
INFO [alembic.runtime.migration] Running upgrade ee8d93fcc81e ->
1986afd32c1b, Add DagRunNote and TaskInstanceNote
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py",
line 1901, in _execute_context
cursor, statement, parameters, context
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py",
line 736, in do_execute
cursor.execute(statement, parameters)
psycopg2.errors.InvalidForeignKey: there is no unique constraint matching
given keys for referenced table "task_instance"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/__main__.py", line
39, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/cli_parser.py",
line 52, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/cli.py", line
108, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/cli/commands/db_command.py",
line 88, in upgradedb
reserialize_dags=args.reserialize_dags,
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/session.py",
line 75, in wrapper
return func(*args, session=session, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/utils/db.py", line
1577, in upgradedb
command.upgrade(config, revision=to_revision or "heads")
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/command.py", line
322, in upgrade
script.run_env()
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/script/base.py", line
569, in run_env
util.load_python_file(self.dir, "env.py")
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/util/pyfiles.py",
line 94, in load_python_file
module = load_module_py(module_id, path)
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/util/pyfiles.py",
line 110, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in
_call_with_frames_removed
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/migrations/env.py",
line 117, in <module>
run_migrations_online()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/migrations/env.py",
line 111, in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/runtime/environment.py",
line 853, in run_migrations
self.get_context().run_migrations(**kw)
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/runtime/migration.py",
line 623, in run_migrations
step.migration_fn(**kw)
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/migrations/versions/0121_2_5_0_add_dagrunnote_and_taskinstancenote.py",
line 87, in upgrade
sa.ForeignKeyConstraint(("user_id",), ["ab_user.id"],
name="task_instance_note_user_fkey"),
File "<string>", line 8, in create_table
File "<string>", line 3, in create_table
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/operations/ops.py",
line 1254, in create_table
return operations.invoke(op)
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/operations/base.py",
line 399, in invoke
return fn(self, operation)
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/operations/toimpl.py",
line 114, in create_table
operations.impl.create_table(table)
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/ddl/impl.py", line
354, in create_table
self._exec(schema.CreateTable(table))
File
"/home/airflow/.local/lib/python3.7/site-packages/alembic/ddl/impl.py", line
195, in _exec
return conn.execute(construct, multiparams)
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py",
line 1380, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/sql/ddl.py", line
81, in _execute_on_connection
self, multiparams, params, execution_options
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py",
line 1478, in _execute_ddl
compiled,
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py",
line 1944, in _execute_context
e, statement, parameters, cursor, context
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py",
line 2125, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/util/compat.py",
line 210, in raise_
raise exception
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/base.py",
line 1901, in _execute_context
cursor, statement, parameters, context
File
"/home/airflow/.local/lib/python3.7/site-packages/sqlalchemy/engine/default.py",
line 736, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.InvalidForeignKey) there
is no unique constraint matching given keys for referenced table "task_instance"
[SQL:
CREATE TABLE task_instance_note (
user_id INTEGER,
task_id VARCHAR(250) NOT NULL,
dag_id VARCHAR(250) NOT NULL,
run_id VARCHAR(250) NOT NULL,
map_index INTEGER NOT NULL,
content VARCHAR(1000),
created_at TIMESTAMP WITH TIME ZONE NOT NULL,
updated_at TIMESTAMP WITH TIME ZONE NOT NULL,
CONSTRAINT task_instance_note_pkey PRIMARY KEY (task_id, dag_id,
run_id, map_index),
CONSTRAINT task_instance_note_ti_fkey FOREIGN KEY(dag_id, task_id,
run_id, map_index) REFERENCES task_instance (dag_id, task_id, run_id,
map_index) ON DELETE CASCADE,
CONSTRAINT task_instance_note_user_fkey FOREIGN KEY(user_id)
REFERENCES ab_user (id)
)
]
(Background on this error at: https://sqlalche.me/e/14/f405)
error: Recipe `airflow-db-upgrade` failed on line 11 with exit code 1
```
### What you think should happen instead
I would expect the task_instance table to have all the keys created for me
by previous migrations.
### How to reproduce
I don't know - maybe try to upgrade from 2.4.1 to 2.50 using postgres as a
storage.
### Operating System
Linux
### Versions of Apache Airflow Providers
not related.
### Deployment
Docker-Compose
### Deployment details
The postgres is serverless "Aurora PostgreSQL"
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
