kaxil commented on pull request #19637: URL: https://github.com/apache/airflow/pull/19637#issuecomment-971897110
>I am all for this. If we have that, I am happy without this "invalidaation". The only problem is HOW to write the tests to account for all the migration paths. This goes hand in hand with my suggestion on making sure that correct tests are added. Whenever something touches serialization -- we should add a test in https://github.com/apache/airflow/blob/main/tests/serialization/test_dag_serialization.py - We also already test serialization and deserialization of example_dags. The simplest of test in a container is to get older stable released version, run `airflow db upgrade` and run a sample DAG and verify that it succeeds. >Why don't we explicitly add such cleanup only when we know that we have a breaking change in-between? > Say we have a user migrating from 2.0.0 to 2.3.0 and we KNOW that 2.1.0 introduced breaking change - let's force the migration then. I mean -- I don't think the "migration" / "cleanup" is needed then. If it was an oversight in 2.1.0 and we broke serialization, we will fix the deserialization part in 2.1.1 . Until now we didn't have to change the serialization format without breaking compatibility. When it comes to it, we can add a DB migration to fix those and do smart "migration" where only affected records are cleaned up. I would still suggest of a separate CLI command like https://github.com/apache/airflow/pull/19471 or run `airflow db upgrade` with a flag (`--with-reserialize-migrations` or similar) to have a switch in case things go wrong. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
