zach-overflow commented on PR #59239: URL: https://github.com/apache/airflow/pull/59239#issuecomment-3696948258
> > > One radical thought if no other idea to fix: Can we drop support of migration from <2.10 when we release 3.2? To enforce people need to upgrate to 2.10 or 2.11 before getting to 3.2? Not sure what our backwards migration promise is, in the past we had cut for all <2.7. > > > > > > This might be a tough sell. We did that before for Airlfow 1 -> 2 migration and for some users it was double work essentially, so this time we decided to lower the minimum. I guess if we can workaround the root cause reliably, it's far lower impact. > > Otherwise - the problem is "just" the downgrade, correct? So in case of upgrading from <2.10 this would just mean downgrade is not possible, meaning you need to be more careful and make proper backup of your DB for case of rollback? > > Second alternative: we drop all trigger states in case of such migration and reset all tasks to re-run from start when downgrading... (just not needing to support downgrading trigger states serialized in DB) Yeah I was wondering about both these options, and neither seem ideal, particularly with regard to keeping upgrade pathways relatively simple for `2.X` users. I just thought of one other potential solution which I've tested locally. I'd be curious to hear people's opinions on this: 1. Leave the old trigger migration script in `2.7.0` unmodified. 2. In our migration CI checks we split [the current `3.2` → `2.7` downgrade command](https://github.com/DataDog/airflow/blob/dfb24d74b968d9362d68db5c46b2e789584480fb/.github/actions/migration_tests/action.yml#L41-L44) into 2 commands: ```diff -export AIRFLOW_3_CMD='airflow db migrate --to-revision heads && airflow db downgrade -n 2.7.0 -y && airflow db migrate' +export AIRFLOW_3_TO_2_8_CMD='airflow db migrate --to-revision heads && airflow db downgrade -n 2.8.0 -y && airflow db migrate' +export AIRFLOW_2_8_TO_2_7_CMD='airflow db migrate --to-revision heads && airflow db downgrade -n 2.7.0 -y && airflow db migrate' -breeze shell "${AIRFLOW_3_CMD}" --no-db-cleanup +breeze shell "${AIRFLOW_3_TO_2_8_CMD}" --no-db-cleanup +breeze shell "${AIRFLOW_2_8_TO_2_7_CMD}" --use-airflow-version 2.8.0 --no-db-cleanup ``` 3. Add a note in the user upgrade docs that if they have upgraded from `2.7.X`, and they decide to downgrade back to `2.7.X`, then they should _first_ downgrade to `2.8` and _then_ downgrade from 2.8 to `2.7.x`. If this sounds reasonable, I'll create a separate PR to modify the migration tests and downgrade documentation accordingly. Obviously, this is not as smooth of a downgrade experience for the user; but, it still preserves the same uninterrupted upgrade pathway to 3.x, without modifying the past migration script. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
