This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v3-0-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 77e671514dfb5e83087bcd897824b8ec99a484b4 Author: Daniel Standish <[email protected]> AuthorDate: Mon Apr 21 11:38:14 2025 -0700 Remove dag_code records with no serdag (#49478) It is assumed by the migration process that there will not be any dag_code records that don't have a serdag record. It seems this only happens when there are two files with the same dag id. --------- Co-authored-by: Tzu-ping Chung <[email protected]> (cherry picked from commit 42024188b9553dd50072e717af754cb95c901972) --- airflow-core/docs/img/airflow_erd.sha256 | 2 +- airflow-core/docs/installation/upgrading_to_airflow3.rst | 5 +++++ .../src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py | 3 +++ 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/airflow-core/docs/img/airflow_erd.sha256 b/airflow-core/docs/img/airflow_erd.sha256 index 33fe09fd943..5ec0d8e84d1 100644 --- a/airflow-core/docs/img/airflow_erd.sha256 +++ b/airflow-core/docs/img/airflow_erd.sha256 @@ -1 +1 @@ -bc93e7288a7a8355b15dc721accaf80260f370f3afa0d478248f9fe4692a1f1d \ No newline at end of file +2a3ce828be7c3800cb61de4c081fcf46f1f94e31ac732f831e65a8a9581b5bb0 \ No newline at end of file diff --git a/airflow-core/docs/installation/upgrading_to_airflow3.rst b/airflow-core/docs/installation/upgrading_to_airflow3.rst index 04c9bda2fa0..d1e094bfb45 100644 --- a/airflow-core/docs/installation/upgrading_to_airflow3.rst +++ b/airflow-core/docs/installation/upgrading_to_airflow3.rst @@ -41,6 +41,11 @@ Step 2: Clean and back up your existing Airflow Instance upgrade process. These schema changes can take a long time if the database is large. For a faster, safer migration, we recommend that you clean up your Airflow meta-database before the upgrade. You can use the ``airflow db clean`` :ref:`Airflow CLI command<cli-db-clean>` to trim your Airflow database. +- Ensure that there are no errors related to dag processing, such as ``AirflowDagDuplicatedIdException``. You should + be able to run ``airflow dags reserialize`` with no errors. If you have have to resolve errors from dag processing, + ensure you deploy your changes to your old instance prior to upgrade, and wait until your dags have all been reprocessed + (and all errors gone) before you proceed with upgrade. + Step 3: DAG Authors - Check your Airflow DAGs for compatibility ---------------------------------------------------------------- diff --git a/airflow-core/src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py b/airflow-core/src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py index bd0a8ea725f..b6b08632f9f 100644 --- a/airflow-core/src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py +++ b/airflow-core/src/airflow/migrations/versions/0047_3_0_0_add_dag_versioning.py @@ -66,6 +66,9 @@ def _airflow_2_fileloc_hash(fileloc): def upgrade(): """Apply add dag versioning.""" conn = op.get_bind() + + op.execute("DELETE FROM dag_code WHERE fileloc_hash NOT IN (SELECT fileloc_hash FROM serialized_dag)") + op.create_table( "dag_version", sa.Column("id", UUIDType(binary=False), nullable=False),
