sshari94 opened a new pull request, #75370: URL: https://github.com/apache/airflow/pull/75370
Before: `airflow db clean --skip-archive` still copied every selected row into an `_airflow_deleted__*` table with `CREATE TABLE ... AS SELECT`, deleted the originals, and then dropped the copy. On large tables that copy is the slow, timeout-prone step, and it is wasted work when the rows are not being kept. After: with `--skip-archive`, each batch is a single `DELETE ... WHERE <pk> IN (SELECT <pk> FROM (<cleanup query>))`. No archive table is created, not even a temporary one. This matches what the docs already say (`usage-cli.rst`: "`--skip-archive` deletes rows directly without the intermediate archive table"). How: `_do_delete` hands the `skip_archive` case to a new `_delete_without_archive`. It uses the same cleanup query, so the recency filter, keep-last, Dag filters, `skip_if_referenced` and `--batch-size` all behave as before. Because the `skip_if_referenced` guard runs inside the same DELETE statement, there is no gap between selecting and deleting rows for the race guard to cover. The extra derived table lets MySQL delete from a table that the subquery also reads. The archive path is unchanged, except that the drop-the-archive cleanup it only needed for `skip_archive` is gone. Tests: `test__skip_archive` now records every SQL statement and asserts that none touches an archive table, with and without `--batch-size`. The mock tests that covered dropping the temporary archive now cover rollback and commit on both paths. The race test for `skip_if_referenced` now runs on the archive%2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
