The GitHub Actions job "Tests" on 
airflow.git/simplify-migration-re-dag-versions-etc has succeeded.
Run started by GitHub user dstandish (triggered by dstandish).

Head commit for run:
2e4808c8a2430ab789b2e2ad54bd84df03257d65 / Daniel Standish 
<[email protected]>
Clear out the dag code and serialized_dag tables on 3.0 upgrade

This will discard the v1 serdags and let them be reserialized after new dag 
processor starts up.

Rather than go through the trouble of migrating the data for serialized dag and 
dag code, we can simply delete it and let it be regenerated after upgrade / 
downgrade.

Why does this make sense?

Prior to airflow version 3, both serialized_dag and dag_code would have been 
deleted every time the dag was reprocessed.  So, it was always ephemeral in 
2.x.  And we typically did a `airflow dags reserialize` on upgrade.

So this is just deleting it one more time and reserializing it one more time on 
the way to 3.0, after which we we _don't_ delete everything with each run of 
dag processor.

There's little value in migrating the data when it can just be regenerated.

Similarly, when going back down to airflow 2.x from 3.0, rather than migrating 
the data, just delete it.  Because it will be regenerated in 2.x, and the PKs 
don't allow more than one version anyway.

Note there is also one downgrade fix where we null out schedule_interval -- if 
we do not do this, the scheduler crashes because it expects this value to be 
json.

Report URL: https://github.com/apache/airflow/actions/runs/14602832355

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to