Hi Jarek,
one thought came into my mind... as you said it is a lot of trouble to
find which "dev" PR might be missing on a mainteance branch... should we
cherry-pick dev tooling PRs by default as well to active maintenance
branches? I mean... automating this in the CI like if `backport to`label
is set?
Jens
On 24.06.25 11:38, Jarek Potiuk wrote:
https://github.com/apache/airflow/pull/52148 -> Proposed PR. Let's iterate
there if there are comments.
On Tue, Jun 24, 2025 at 11:25 AM Jarek Potiuk <ja...@potiuk.com> wrote:
Yeah. I think what I am really after is expressed by Wei: +0.5 vote -
i.e. we should be fine with **some** selected cherry-picks that are not
bug-fixes that we proactively cherry-pick and do not "complain" that this
is wrong because technically it's not a bugfix. I think It would be great
if in every such cherry-pick we rationalise why we are doing it:
"I know it's not a bugfix, but because this is an actively developed part
that is cherry-picked heavily, let me cherry-pick that one to make it
easier for others to cherry-pick bugfixes".
If we are all on-board with that (seems so) then I can update the
cherry-pick description to add it as possible exception.
J.
On Tue, Jun 24, 2025 at 10:14 AM Kaxil Naik <kaxiln...@gmail.com> wrote:
It looks like everyone is mostly on the same page based on all the emails
-
so no comments :) .
On Tue, 24 Jun 2025 at 07:45, Wei Lee <weilee...@gmail.com> wrote:
I’m +1 for 1-3 (assuming the doc changes relate to the backported
version).
+0.5 for 4. I hope that changes not related to new features will be
backported when feasible; however, we can skip them if the required
effort
is substantial. This is because failing to backport these items could
potentially lead to future conflicts with points 1 or 3.
Best,
Wei
On Jun 24, 2025, at 5:32 AM, Jens Scheffler
<j_scheff...@gmx.de.INVALID>
wrote:
I am +1 for (1) to (3) [also assuming that (2) is mostly like a doc
bugfix!]
For (4) I am hesitant and would rather be conservative. Every
cherry-pick has a risk to break something in old codebase. As branches
change over time and backport PRs are clearly less cautious reviewed it
might lead to introduced inconsistencies which might not be covered in
tests. I would also not back-port just for the sake of easier later
maintenance for other cherry-picks. But for (4) the rule might not be
too
strict and every rule is made for exceptions which in (4) might be
likely.
But I would not advertise to backport refactorings if no clear benefit.
On 23.06.25 15:33, Pavankumar Gopidesu wrote:
Thanks Jarek, for starting this discussion,
I agree with all the points.
The real intention behind to backport
https://github.com/apache/airflow/pull/51992 is , this area has a
lot
of
ongoing development going and I felt it's worth porting to v3-0-test.
I myself faced situations where I tried to backport some doc changes
, I
failed because of conflicts that previous changes were not ported
back
on
the same file etc or conflicts with some lines.
So I am very strong on this if there are any areas with heavy
development
activity going. I feel it's worth backporting the changes and IMHO i
don't see any problem.
Pavan
On Mon, Jun 23, 2025 at 2:23 PM Jarek Potiuk <ja...@potiuk.com>
wrote:
I think We are approaching it from the same point of view, just
that
we
have different conclusions.
I agree. I think that simply there is one special case that we
should
"allow". Details below.
For 4, I do not have strong opinions on either front, but defining
what
to
do and what not should
probably be that if that change makes it easy to backport something
else,
maybe have it?
Yep. That's **exactly** my proposal. I do not want to backport
**all**
refactorings. That would be stupid. In my definition of 4. I want to
cherry-pick those pre-commit when it's small, and automated and when
we can
easily anticipate it will make it quite likely someone (soon) will
do
another cherry-pick that will be conflicting.
Yes. It's not "0/1" and quite a bit personal judgment, and yes it
**might
not** result in conflict (so we might end up with YAGNI), but I
think
we
should define (and trust committers judgment) when they do it,
because
they
want to prevent "others" to have problems.
For example in case of https://github.com/apache/airflow/pull/51992
-> the
only reason why I think it makes sense to cherry-pick it is because
it
is
in the area of task-sdk that has been recently bug-fixed quite a
number of
times and there are open related issues to this area that make it
(IMHO)
quite likely to result in conflict. I think there are other areas
like
that
(UI for example) where we clearly cherry-pick a number of changes,
because
there are some pretty active "bug-fixing" in this area as 3.0 had
still a
number of low priority but either known or anticipated bugs that are
likely
to be fixed in 3.0.3, 3.0.4
So just to clarify - I do not want to cherry-pick "all" refactoring,
but
leave to the judgment of the commiter merging the request (and
author)
to
decide to cherry-pick such change anticipating it will make life
easier for
others.
J
On Mon, Jun 23, 2025 at 2:57 PM Amogh Desai <
amoghdesai....@gmail.com>
wrote:
Agree with point 1 - 3 definitely.
For 4, I do not have strong opinions on either front, but defining
what
to
do and what not should
probably be that if that change makes it easy to backport something
else,
maybe have it?
For ex:
*PR 1 changes file1.py and file2.py*
*PR 2 changes some lines in file2.py*
*We backport PR 1 as it's a bug fix and do not for PR 2 as its some
refactoring.*
*Now while trying to backport PR 3 (bugfix), it conflicts and needs
PR 2
to
be picked*
*to land PR 3.*
Thanks & Regards,
Amogh Desai
On Mon, Jun 23, 2025 at 5:02 PM Ash Berlin-Taylor <a...@apache.org>
wrote:
I think We are approaching it from the same point of view, just
that
we
have different conclusions.
Points 1-3 I agree with.
We do already have this written up
https://github.com/apache/airflow/blob/130e9600443e06c08acc1b28c69a62c858d6e6a2/dev/README_RELEASE_AIRFLOW.md?plain=1#L116-L129
On 4 I think of it in terms that “every change carries some risk”
—
in
the
linked example of 51992 the risk is almost zero, but generally:
3.0.x
are
meant to be bug fix releases on top of 3.0.0, and if it’s not
fixing
a
bug
we don’t back port it. The one exception I have to this is if the
change
is
needed to make it easy to backport a change that is a bug without
conflicts.
I think our default approach has to be we don’t back port a change
unless
it is fixing a bug, otherwise the risk of “oh I’ll just fix this”
ends
up
introducing more bugs than we fix. Stability of a Minor release
series
is
my primary desire, and not changing things more than we have to is
the
best
way I know of doing that.
Things are slightly different now that we have automated
cherry-picks
but
I still don’t think it is worth porting refactoring automatically.
It’s
extra change and risk for almost zero benefit to users is my view.
-ash
On 23 Jun 2025, at 11:43, Jarek Potiuk <ja...@potiuk.com> wrote:
BTW. I'd be happy to capture result of this discussion if we can
get
to a
consensus or vote eventually in the "cherry-picking" guidelines.
On Mon, Jun 23, 2025 at 12:42 PM Jarek Potiuk <ja...@potiuk.com>
wrote:
I wanted to start a discussion on "things that we cherry-pick"
(to
vX_Z
branch).
I think there are different opinions on what kind of changes
should
be
cherry-picked and it might be a good idea to agree on a common
approach.
I think (following the comment of Ash here)
https://github.com/apache/airflow/pull/51992#issuecomment-2995632849
that
we can use a very simplistic and (I'd say) dogmatic approach
"only
cherry-pick bug fixes. Full stop". But I believe (and past
experience
from
a lot of cherry-picking that I've been doing - multiple times
helping
to
bring past branches to be green and spending countless hours on
it,
that it
should be a bit more nuanced.
I would love to see what others think, but from my experience
those
are
the things that we **should** cherry-pick:
1) bug-fixes (of course)
2) doc changes (when they are improvements or filling gaps)
3) dev tool changes (every time we did not, it resulted in
hours of
my
time when things were breaking and I tried to reconcile it)
4) results of automated refactorings that have very low risks
(in
the
areas that are likely to have cherry-picks)
t) - is non-controversial I think
2) - is also relatively non-controversial and very low risk and
gives
our
users a chance to get better docs earlier (even today for
example I
cherry
picked this one: https://github.com/apache/airflow/pull/52068 -
because
one of my friends who tries to learn Airflow 3 pinged me that
"ConfiuguringRuff" link that we have in 3.0.2 leads to 404 NOT
found
3) - it had always bitten us if we stopped cherry-picking dev
tool
changes. The thing is that external dependencies change all the
time
and we
are continuously catching up with those, also we improve, speed
up
and
simplify the tooling - and often things that worked when branch
was
cut,
does not work today - countless, countless hours lost in one or
two
branches when we stopped doing it - I think even once or twice I
had
to
just copy over most (but not all) the code from main to the
branch
and
commit one single "catch-up dev tooling with main" big change
4) Is likely most controversial - example here:
https://github.com/apache/airflow/pull/51992/ - those are the
kind
of
(really small) changes that are done in "active" area (i.e. area
that
had
and will have a lot of cherry-picks anyway, but they are done
with
automated refactoring - like renaming variables and such. This
introduces
clarity and readability, so this is good we are doing them. But
if
we
do
not cherry-pick them and then we cherry-pick any change that
touches
the
same code, this lead to a conflict. Conflicts are frustrating,
especially
those kinds - you never know what you should do - should you
"merge"
this
naming change with your change? or should you leave the original
namiing,
or should you try to find the past commit that changed it and
cherry-pick
it as well? This paired with the fact that we are using
cherry-picker
that
allows to cherry-pick stuff very quickly, automatically and
painlessly
when
there are no conflicts, make me think that yes - we should
cherry
-pick
those changes proactively as a service to those contributors who
will
follow up with their cherry-picking. It's just "good service"
and
helping
others who will come after you.
That's how my definition of "what we should cherry-pick" is...
I wonder what others think about it ?
J.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org
