HsiuChuanHsu commented on issue #56192: URL: https://github.com/apache/airflow/issues/56192#issuecomment-3393068953
Hi @hket-ianchu, I was able to reproduce the issue you provided. Below are the reproduction results: **Create testing data in `dag_version` & `task_instance` table.** - `dag_version`: created_at | 2025-09-01 <img width="477" height="158" alt="Image" src="https://github.com/user-attachments/assets/f0395a14-58f1-4e08-942d-478a8e58f8de"; /> - `task_instance-1`: start_date | 2025-09-06 <img width="434" height="666" alt="Image" src="https://github.com/user-attachments/assets/be676f8e-35d7-43ea-a6c4-84725c88903d"; /> - `task_instance-2`: start_date | 2025-10-01 <img width="435" height="652" alt="Image" src="https://github.com/user-attachments/assets/934d4091-2c35-4549-b37f-0a1c3a11162e"; /> --- **Use `airflow db clean` to clean up the records** ```bash airflow db clean --clean-before-timestamp "2025-09-21T10:30" --skip-archive --yes ``` **Result** ```bash airflow db clean --clean-before-timestamp "2025-09-21T10:30" --skip-archive --yes /opt/airflow/airflow-core/src/airflow/dag_processing/dagbag.py:40 DeprecationWarning: airflow.exceptions.AirflowDagCycleException is deprecated. Use airflow.sdk.exceptions.AirflowDagCycleException instead. 2025-10-11T08:33:45.238607Z [warning ] Table celery_tasksetmeta not found. Skipping. [airflow.utils.db_cleanup] loc=db_cleanup.py:555 2025-10-11T08:33:45.238729Z [warning ] Table celery_taskmeta not found. Skipping. [airflow.utils.db_cleanup] loc=db_cleanup.py:555 Checking table task_instance_history Found 0 rows meeting deletion criteria. Checking table xcom Found 0 rows meeting deletion criteria. Checking table task_instance Found 1 rows meeting deletion criteria. Performing Delete... Moving data to table _airflow_deleted__task_instance__20251011083345 Finished Performing Delete Checking table deadline Found 0 rows meeting deletion criteria. Checking table dag_run Found 1 rows meeting deletion criteria. Performing Delete... Moving data to table _airflow_deleted__dag_run__20251011083345 Finished Performing Delete Checking table dag_version Found 1 rows meeting deletion criteria. Performing Delete... Moving data to table _airflow_deleted__dag_version__20251011083345 Traceback (most recent call last): File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context self.dialect.do_execute( File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute cursor.execute(statement, parameters) psycopg2.errors.ForeignKeyViolation: update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance" DETAIL: Key (id)=(018c5e5e-7a8b-7000-8000-000000000001) is still referenced from table "task_instance". The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/python/bin/airflow", line 10, in <module> sys.exit(main()) ^^^^^^ File "/opt/airflow/airflow-core/src/airflow/__main__.py", line 55, in main args.func(args) File "/opt/airflow/airflow-core/src/airflow/cli/cli_config.py", line 49, in command return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/airflow/airflow-core/src/airflow/utils/cli.py", line 115, in wrapper return f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/opt/airflow/airflow-core/src/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/airflow/airflow-core/src/airflow/cli/commands/db_command.py", line 297, in cleanup_tables run_cleanup( File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 100, in wrapper return func(*args, session=session, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 544, in run_cleanup _cleanup_table( File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 371, in _cleanup_table _do_delete( File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 261, in _do_delete raise e File "/opt/airflow/airflow-core/src/airflow/utils/db_cleanup.py", line 257, in _do_delete session.execute(delete) File "/usr/python/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1717, in execute result = conn._execute_20(statement, params or {}, execution_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1710, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/python/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection return connection._execute_clauseelement( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1577, in _execute_clauseelement ret = self._execute_context( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1953, in _execute_context self._handle_dbapi_exception( File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2134, in _handle_dbapi_exception util.raise_( File "/usr/python/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context self.dialect.do_execute( File "/usr/python/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute cursor.execute(statement, parameters) sqlalchemy.exc.IntegrityError: (psycopg2.errors.ForeignKeyViolation) update or delete on table "dag_version" violates foreign key constraint "task_instance_dag_version_id_fkey" on table "task_instance" DETAIL: Key (id)=(018c5e5e-7a8b-7000-8000-000000000001) is still referenced from table "task_instance". [SQL: DELETE FROM dag_version USING _airflow_deleted__dag_version__20251011083345 WHERE dag_version.id = _airflow_deleted__dag_version__20251011083345.id] (Background on this error at: https://sqlalche.me/e/14/gkpj) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
