vatsrahul1001 opened a new issue, #63423:
URL: https://github.com/apache/airflow/issues/63423
### Apache Airflow version
main (development)
### If "Other Airflow 3 version" selected, which one?
_No response_
### What happened?
The downgrade() in `0096_3_2_0_remove_team_id.py` re-creates the FK
dag_bundle_team_team_id_fkey but never re-creates the index
idx_dag_bundle_team_team_id on dag_bundle_team. That index was originally
created by 0083_3_1_0_add_teams.py.
So after a downgrade, the DB is missing that index. When you run airflow db
migrate again, migration 0096 upgrade tries to drop it and blows up
```
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/python/bin/airflow", line 10, in <module>
sys.exit(main())
File "/opt/airflow/airflow-core/src/airflow/__main__.py", line 55, in main
args.func(args)
File "/opt/airflow/airflow-core/src/airflow/cli/cli_config.py", line 49,
in command
return func(*args, **kwargs)
File "/opt/airflow/airflow-core/src/airflow/utils/cli.py", line 113, in
wrapper
return f(*args, **kwargs)
File
"/opt/airflow/airflow-core/src/airflow/utils/providers_configuration_loader.py",
line 54, in wrapped_function
return func(*args, **kwargs)
File "/opt/airflow/airflow-core/src/airflow/cli/commands/db_command.py",
line 206, in migratedb
run_db_migrate_command(args, db.upgradedb, _REVISION_HEADS_MAP)
File "/opt/airflow/airflow-core/src/airflow/cli/commands/db_command.py",
line 134, in run_db_migrate_command
command(
File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 100,
in wrapper
return func(*args, session=session, **kwargs) # type: ignore[arg-type]
File "/opt/airflow/airflow-core/src/airflow/utils/db.py", line 1246, in
upgradedb
_run_upgradedb(config, to_revision, session)
File "/opt/airflow/airflow-core/src/airflow/utils/db.py", line 1169, in
_run_upgradedb
command.upgrade(config, revision=to_revision or "heads")
File "/usr/python/lib/python3.10/site-packages/alembic/command.py", line
483, in upgrade
script.run_env()
File "/usr/python/lib/python3.10/site-packages/alembic/script/base.py",
line 545, in run_env
util.load_python_file(self.dir, "env.py")
File "/usr/python/lib/python3.10/site-packages/alembic/util/pyfiles.py",
line 116, in load_python_file
module = load_module_py(module_id, path)
File "/usr/python/lib/python3.10/site-packages/alembic/util/pyfiles.py",
line 136, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in
_call_with_frames_removed
File "/opt/airflow/airflow-core/src/airflow/migrations/env.py", line 138,
in <module>
run_migrations_online()
File "/opt/airflow/airflow-core/src/airflow/migrations/env.py", line 132,
in run_migrations_online
context.run_migrations()
File "<string>", line 8, in run_migrations
File
"/usr/python/lib/python3.10/site-packages/alembic/runtime/environment.py", line
969, in run_migrations
self.get_context().run_migrations(**kw)
File
"/usr/python/lib/python3.10/site-packages/alembic/runtime/migration.py", line
626, in run_migrations
step.migration_fn(**kw)
File
"/opt/airflow/airflow-core/src/airflow/migrations/versions/0096_3_2_0_remove_team_id.py",
line 45, in upgrade
with op.batch_alter_table("dag_bundle_team") as batch_op:
File "/usr/python/lib/python3.10/contextlib.py", line 142, in __exit__
next(self.gen)
File
"/usr/python/lib/python3.10/site-packages/alembic/operations/base.py", line
408, in batch_alter_table
impl.flush()
File
"/usr/python/lib/python3.10/site-packages/alembic/operations/batch.py", line
115, in flush
fn(*arg, **kw)
File "/usr/python/lib/python3.10/site-packages/alembic/ddl/impl.py", line
483, in drop_index
self._exec(schema.DropIndex(index, **kw))
File "/usr/python/lib/python3.10/site-packages/alembic/ddl/impl.py", line
256, in _exec
return conn.execute(construct, params)
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 1419, in execute
return meth(
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/sql/ddl.py",
line 187, in _execute_on_connection
return connection._execute_ddl(
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 1530, in _execute_ddl
ret = self._execute_context(
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 1846, in _execute_context
return self._exec_single_context(
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 1986, in _exec_single_context
self._handle_dbapi_exception(
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 2363, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py",
line 1967, in _exec_single_context
self.dialect.do_execute(
File
"/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line
952, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedObject) index
"idx_dag_bundle_team_team_id" does not exist
[SQL:
DROP INDEX idx_dag_bundle_team_team_id]
(Background on this error at: https://sqlalche.me/e/20/f405)
[Breeze:3.10.20] root@d784b48fa376:/opt/airflow$
```
### What you think should happen instead?
_No response_
### How to reproduce
**Steps to Reproduce:**
1. `breeze start-airflow --executor CeleryExecutor --backend postgres
--load-example-dags --use-airflow-version 3.1.8`
2. Run some DAGs.
3. Execute stop_airflow in tmux session
4. `breeze start-airflow --executor CeleryExecutor --backend postgres
--load-example-dags`
5. Migration 3.1.8 → 3.2.0 succeeds.
6. Run airflow db downgrade -n 3.1.8 in tmux session and drograde succeeds
7. Try running airflow db migrate and it will fail
### Operating System
Mac
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
