Hi,
as I raised the discussion with one comment on the PR, wanted to share
my view:
I have no ties in SQLA1 and the argumentation that task execution == 99%
of providers except those who provide Executors should be free to use
any irrespective of what the core is pinned.
So thanks for the discussion, would be good to hear if some other
dependency (outside of known monorepo) exists that further require SQLA1
or if Airflow 3.2 (Core) can migrate to >=SQLA2 in its depednencies. But
would be a hard switch between Airflow <=3.1 and 3.2 if any dependency
we are not aware of.
Is there an estimate of efforts/costs keeping a SQLA1 CI test running
(similar like we had Pydantic v1 as well for a while)? Maybe
transitional until Airflow 3.3?
Hard opinions:
* I am +1 switchintg the default to SQLA2 for Airflow
* I am for making this in Airflow 3.2 as all seems to be ready and in-time
Thanks for all the efforts by the way to migrate! Great community work!
As we have many people on leave atm we might need to wait until after
Christmas break for responses? Or how long can we afford to wait?
Jens
On 12/25/25 20:25, Jarek Potiuk wrote:
+1
In January it will be 3 years since first SQLA 2 stable version was
released. Plenty of time for all the libraries we use to support it. And
they all do - demonstrated by the green build.
Internally we have no specific ties with SQLA1. So the only real -
potential - issue is if someone uses SQLA2 in their code for DAGs. Also
people are not supposed to use our SQLA Models for anything any more - just
APIs.
But - with the task isolation efforts - assuming we manage to complete it
with 3.2 - what will happen that those users will actually use task SDK -
which will have - by necessity - NO SQLA as dependency.
At all.
The only potential dependency to SQLA is small part of DBApi (coomon.sql)
that has some sql-specific (but effectively optional) methods. And we could
revert our test setup to test those with SQLA1 to only keep compatibility
there. That would be possible and not very difficult. Currently we have
separate sqla1 tests to run all tests but we could limit it to only
common.sql tests.
I don't see personally big issue with this one.
But maybe others - especially Airflow users currently who did some
migrations might also chime in.
J.
On Thu, Dec 25, 2025, 20:01 Dev iL<[email protected]> wrote:
Dear Community,
I’d like to start a discussion regarding a proposal to *migrate Airflow to
SQLAlchemy 2.0* for the upcoming *v3.2* feature release.
The Problem: Maintenance Overhead and Locked Ecosystem
As many of you know, there is a massive ongoing effort by community members
to improve and upgrade type hints across the entire codebase. However,
because our CI currently runs on SQLAlchemy 1.4, *Mypy does not "protect"
us* against introducing new violations that would be caught in a 2.0
environment. This creates a significant maintenance burden where
contributors and reviewers must manually account for type safety that the
CI is unable to enforce. Moving to 2.0 provides native PEP 484 support,
which will drastically improve ORM class coverage and allow us to catch
bugs during development rather than at runtime.
Staying on 1.4 has created a "dependency ceiling." Migrating to SQLAlchemy
2.0 will finally allow us to upgrade several key libraries that the
community has been asking for, including:
-
*pandas* >= 2.2 (released on Jan 2024)
-
*numpy* >= 2 (released on Jun 2024)
-
*flask-sqlalchemy* >= 3.1 (released on Sep 2023)
-
*sqlmodel* >= 0.12 (released on Nov 2023)
Current Progress
We already have a functional path forward. There is an outstanding PR that
implements this transition:
-
*PR:*https://github.com/apache/airflow/pull/59218
*Note:* This PR is currently *green* (passing all CI checks), demonstrating
that the core functional changes are stable and do not break our existing
test suite.
Seeking Community Feedback
We want to ensure this transition is as smooth as possible for all
stakeholders. We are particularly interested in your views on:
1.
*Custom Providers/Plugins:* Are there concerns regarding plugins that
rely heavily on SQLAlchemy 1.4 internals?
2.
*Database Backends:* Does anyone foresee specific issues with our
supported versions of Postgres, MySQL, or SQLite during this jump?
3.
*General Timeline:* Does the community feel Airflow 3.2 is the right
target for this breaking internal change? Should we aim for official
dual
support for a period of time?
Please share your thoughts, concerns, or support for this move. If there is
a general consensus, we plan to proceed with the review and merge of the PR
in the near future.
Best regards,
Dev-iL
