Hi Wei,
I don't mind either way. I suspect the effort needed by users is much
larger for 1.4->2.0 than 2.0->2.1.
Say we start work on 2.0 support. I'm assuming it's not much extra
effort to implement 2.1 support alongside it (at the very least use
syntax that's not deprecated in 2.1), but what I'm worried about is
excessive branching in e.g. class definitions of the sort I mentioned in
the OP:
class SomeModel(...)
if is_SQLA1():
fieldname: annotation
elif is_SQLA2():
...
else: # implicitly is_SQLA2_1()
...
Is there an existing example in the project of supporting multiple
library versions that require distinct approaches?
Best,
Dev-iL
On 2025/06/27 02:08:17 Wei Lee wrote:
> > 2. Should we work on supporting SQLA 2.0 or jump straight to 2.1 (which
> is currently in beta)?
>
> Unless there’s a major reason to upgrade to 2.1, I would suggest
supporting SQLA 2.0 instead. Upgrading to 2.0 alone could disrupt enough
package dependencies (from the user end), and moving to 2.1 might cause
even more issues.
>
> Best,
> Wei
>
> > On Jun 27, 2025, at 3:48 AM, Jens Scheffler <j_...@gmx.de.INVALID>
wrote:
> >
> > Hi,
> >
> > thanks for taking the lead on this!
> >
> > Regarding (1) I fear we need to make a dual version support for a
moment because some providers also use SQLA (fab, edge3 at least I know
of, standard also soonish if human operators are implemented). If we
directyl move to 2.0 then this would be sort of a breaking change and
all providers need to move as well - and then newer providers would not
be working on older Airflow versions. So I fear this is a coupling
problem... and also means we need to consider how long we actually keep
1.4 support.
> >
> > I am not an expert which implications and "messy" workarounds we
need to consider for a dual version compatibility. Would it be possible
also to use the same model and just make the field definitions
dynamically depending on the library version? Like...
> >
> > classDagRun(Base, LoggingMixin):
> > __tablename__="dag_run"
> > if SQLA3:
> > id: Mapped[int] = mapped_column(Integer, primary_key=True) dag_id:
Mapped[str] = mapped_column(String, ForeignKey("dag.id"))
> > else:
> > id=Column(Integer, primary_key=True)
> > dag_id=Column(StringID(), nullable=False)
> >
> > But there is also a larger migration guide. I assume you have
digested and considered this already?
> >
> > Jens
> >
> > On 26.06.25 14:49, Dev iL wrote:
> >> Hello everyone,
> >>
> >> With the release of Airflow 3 and the refactoring that moved
> >> flask-appbuilder (FAB) into a separate provider, as well as recent
work by
> >> the FAB developers to support SQLAlchemy v2.0 (SQLA2), it is finally
> >> possible to work on supporting SQLA2 in Airflow as well.
> >>
> >> To get this process going, I added two CI tasks that install SQLA2 (PR
> >> #52233) and am now slowly adding various workarounds to overcome
repeated
> >> test failures. After doing this for a while, there's a couple of
discussion
> >> points I want to raise:
> >>
> >> 1. What is the plan regarding dual support of 1.4 & 2.x? If we want to
> >> have that, it might lead to pretty messy code (see details below).
> >> 2. Should we work on supporting SQLA 2.0 or jump straight to 2.1
(which
> >> is currently in beta)?
> >>
> >> I think we should decide on the refactoring principles and start
moving in
> >> that direction. With some luck, it might be possible to get this
done in
> >> time for 3.1.
> >>
> >> I'd love your inputs on how to best approach this task. More
importantly,
> >> if anyone wants to participate - please leave your comment on my
PR, and
> >> let's coordinate. I suggest you use my branch as a starting point,
since it
> >> has the tests in place.
> >>
> >> Best,
> >> Dev-iL
> >>
> >> P.S.
> >> At the time of writing, the status of running the tests (via `breeze
> >> testing core-tests --upgrade-sqlalchemy --maxfail=1000`) is: 134
failed,
> >> 6469 passed, 47 skipped, 8 xfailed, 1 warning
> >> -------------------
> >>
> >> SQLA2 introduced stricter type annotation requirements for ORM mapped
> >> attributes. All mapped columns need to use the `Mapped[]` generic type
> >> annotation:
> >>
> >> <pre>
> >> class TaskInstance(Base):
> >> __tablename__ = "task_instance"
> >>
> >> # Before (SQLA 1.4)
> >> # id = Column(Integer, primary_key=True)
> >> # dag_id = Column(String, ForeignKey("dag.id"))
> >> # dag_model = relationship("DagModel")
> >>
> >> # After (SQLA 2.0)
> >> id: Mapped[int] = mapped_column(Integer, primary_key=True)
> >> dag_id: Mapped[str] = mapped_column(String, ForeignKey("dag.id"))
> >> dag_model: Mapped["DagModel"] = relationship()
> >> </pre>
> >>
> >> To address differences of similar nature, and assuming we're
interested in
> >> dual SQLA version support, I can think of two extremes:
> >>
> >> 1. We implement two copies of each model, one for each version of
SQLA,
> >> and use the relevant one based on a runtime check.
> >> 2. We use workarounds that disable the breaking features of SQLA2
(e.g.
> >>
https://github.com/apache/airflow/pull/52233/commits/90d738880553707fa01ee2a6b4acd40609e8cc2f)
> >> where possible, and resort to runtime branching where impossible, thus
> >> postponing the rewrite to whenever SQLA1.4 support will be dropped.
> >>
> >> ----------------------
> >>
> >> References:
> >>
> >> 1. Main SQLA2 support issue:
> >> https://github.com/apache/airflow/issues/28723
> >> 2. My PR adding SQLA2 to the CI:
> >> https://github.com/apache/airflow/pull/52233
> >> 3. FAB's SQLA2 support PR:
> >> https://github.com/dpgaspar/Flask-AppBuilder/pull/2241
> >>
>
>
