Hi all, Reviving this old thread.
Flask-AppBuilder version 5 was released at the beginning of September, so I thought it would be a good time to continue work on upgrading the FAB provider to use `flask-appbuilder` 5. Compared to the previous integration with the FAB 5 alpha, we no longer need the workarounds Jarek and I had added to fix SQLAlchemy-related issues. The PR is here and already green: https://github.com/apache/airflow/pull/50960. What are the next steps and the current status regarding SQLAlchemy? As a reminder: * The current version we use in Airflow (`flask-appbuilder` 4.x) is not compatible with SQLAlchemy 2. * `flask-appbuilder` 5 is compatible with SQLAlchemy 2. * However, during the FAB 5 upgrade I found that parts of the Airflow codebase itself are still not fully compatible with SQLAlchemy 2. That’s why in the PR above I pinned `sqlalchemy<2` in `providers/fab/pyproject.toml`. We’ll need to address these compatibility issues in Airflow before we can fully switch. **Proposed next steps:** 1. Review and merge https://github.com/apache/airflow/pull/50960. 2. Update the Airflow codebase to be compatible with SQLAlchemy 2. Most of the issues come from `mypy` errors, nothing difficult, just numerous. @Dev-iL created a nice and detailed plan to help us fix these issues here: https://github.com/user-attachments/files/22438407/SQLA_Migration_Plan.md. I’ll create an issue to track them so anyone in the community can help. I’ll also open a draft PR with SQLAlchemy 2 enabled and rebase it frequently so we can monitor progress. 3. Once all issues are resolved, we can enable SQLAlchemy 2 in Airflow. As always, feedbacks are appreciated. Best, Vincent On 2025/06/05 12:07:08 Vincent Beck wrote: > > It took a lot of guessing, debugging, experimenting and while it's fine for > > me, I guess Vincent's hair might have thinned a bit after the experience > > because we were literally pulling our hair out) > > HAHAHAHAHAHA. Thank you Jarek. You made my day ;') > > On 2025/06/05 10:26:30 Jarek Potiuk wrote: > > Hello here, > > > > *TL;DR; we are quite a bit closer to getting FAB 5 (alpha) support and we > > have a kind request to those who know more about inner-workings of > > SQLAlchemy to comment on our uncertainties and workarounds.* > > > > PR is here https://github.com/apache/airflow/pull/50960 and I explained in > > detail what we have done and why in the description, also pointed out some > > uncertainties and added comments on where we are unsure if what we came up > > with makes sense - and whether it can be improved or fixed better. > > > > Why is it important and why did we spend the better part of our last week > > on it? > > > > Having FAB 5 would make us closer to getting rid of some of the old > > dependency issues: > > > > * frees us to migrate to sqlalchemy 2 (not yet there- this is still > > sqlalchemy 1, but we can attempt sqlalchemy 2 migration as next step). > > * frees us to have proper Python 3.13 support > > > > We have a green PR now with all the issues solved or workarounds (we spend > > with Vincent quite a few long days and nights working on finding out how > > things work and fix things up - we've been depending on subtle internals of > > FAB and flask-sqlalchemy that have changed. > > > > It took a lot of guessing, debugging, experimenting and while it's fine for > > me, I guess Vincent's hair might have thinned a bit after the experience > > because we were literally pulling our hair out). But we are definitely past > > that - we understand much more how things work, > > > > But there are still some small workarounds that we would like to understand > > better if they can be improved. > > > > SQLAlchemists here - please take a look and help! > > > > J. > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
