Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Thanks for starting this discussion Jarek! I think it's very important for us to get on the same page as a community about this. I'd love to go with a more flexible/common sense approach for considering breaking changes, and in a perfect world I think this would be best. However, I also think

Re: [VOTE] November 2022 PR of the Month

+1 for 26457 too On Tue, Nov 22, 2022 at 5:44 PM Bas Harenslak wrote: > > +1 for 26457 > > Bas > > On 22 Nov 2022, at 17:31, Jeambrun Pierre wrote: > > Hello, > > I vote for https://github.com/apache/airflow/pull/26457. Not sure why it > wasn't selected by our heuristic, with 2k additions,

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Deprecate first to discover the possible footprint of any public API. With the deprecation, bump the minor version. Add documentation about the deprecation and appropriate deprecation code and warnings, with a notice for the EOL date or an EOL sem ver. — Darren On Tue, Nov 22, 2022 at 1:32 AM

Re: [VOTE] November 2022 PR of the Month

+1 for 26457 Bas > On 22 Nov 2022, at 17:31, Jeambrun Pierre wrote: > > Hello, > > I vote for https://github.com/apache/airflow/pull/26457 > . Not sure why it wasn't > selected by our heuristic, with 2k additions, 140 comments, 9 participants,

Re: [VOTE] November 2022 PR of the Month

Hello, I vote for https://github.com/apache/airflow/pull/26457. Not sure why it wasn't selected by our heuristic, with 2k additions, 140 comments, 9 participants, and even one #protm tag 樂. (I will try to take a look) Best, Le mar. 22 nov. 2022 à 16:40, John Thomas a écrit : > Another month

[VOTE] November 2022 PR of the Month

Another month passes, and another newsletter approaches! Heuristically, we have the below PRs nominated: Please vote by selecting the most important, interesting, or impactful PR from the list below (or an entirely new one!) and replying with the number Voting will close on 11/29 at 8:15a PT.

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

I beg to differ too and I do think that Alexander is right in what he wants to accomplish. Large installations do want to do rolling upgrades and not bring a cluster down for an upgrade. It should be possible to run Airflow Core 2.4 with for example 2.3 workers. It should however not happen

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

On topic: Airflow has a pretty loosely defined contract. E.g. operators do not have strict ordering of keywords, our naming is not very consistent (conn_id vs oss_conn_id and many more) and we do weird thing in tasks vs dag/scheduler/executor :-). Design wise we are very flexible (or ?). This is

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Jarek, No worries (:  -- ,,,^..^,,, On Tue, Nov 22, 2022 at 3:42 PM Jarek Potiuk wrote: > Cool. I understand. Good example. Might contribute to the discussion. > But let's not discuss in this thread if you can /cannot run workers. I > started this and that was a mistake - as it diverges from

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Cool. I understand. Good example. Might contribute to the discussion. But let's not discuss in this thread if you can /cannot run workers. I started this and that was a mistake - as it diverges from the main topic. So the comment was mainly to myself as I was diverging from the main topic. On

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Jarek, I'm probably not ready to introduce this topic and push it forward, but I just wanted to clarify what are the borders of breaking changes and how they are applied to Airflow itself. Practice shows they are not, so it's may be a problem that you raised on the topic. Or may be not. It's

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

I propose let's not "diverge" the discussion with this specific case and whether it's easy or not. Let's focus on general approach and whether the approach to make a policy makes sense in this (or different way) but let's not argue if it is easy to deploy airflow with mutliple different versions

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Hi, I Beg to differ with Alexander and agree with Jarek. There are multiple ways to deploy Airflow. Mostly commonly used is docker images, in that case using one image for all components is standard practice. If using native pip installations, airflow components are launched by a single pip

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

On Tue, Nov 22, 2022 at 1:37 PM Jarek Potiuk wrote: > BTW. "Workers from 2.2" used with "Airflow 2.4" is not even a thing. > This is something that you should never, ever, try to do. > This is even more common sense, and there are of course limits of what > you can describe in the docs (whatever

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

BTW. "Workers from 2.2" used with "Airflow 2.4" is not even a thing. This is something that you should never, ever, try to do. This is even more common sense, and there are of course limits of what you can describe in the docs (whatever you come up with, someone might have a super crazy idea that

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Changing the DB schema has never been "breaking". This is implicit, not really "explicitly" documented as breaking classification. And this is also the case which I refer to as the "common sense" - and also another example where "common sense" is not enough. Since we introduced a stable REST

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

The rule of breaking changes also applies to database schema as well, right? Because currently, almost every minor release is breaking since you cannot run workers from 2.2.x with a database from 2.4.x. -- ,,,^..^,,, On Tue, Nov 22, 2022 at 12:32 PM Jarek Potiuk wrote: > Hello everyone, > >

Re: [DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Also for those who prefer visual ways - obligatory XKCD which is the illustration of Hyrum's Law: https://xkcd.com/1172/ On Tue, Nov 22, 2022 at 10:31 AM Jarek Potiuk wrote: > > Hello everyone, > > We had a few discussions in PRs recently about removing some > functionality from the Airflow Core

[DISCUSSION] Assessing what is a breaking change for Airflow (SemVer context)

Hello everyone, We had a few discussions in PRs recently about removing some functionality from the Airflow Core and the question of backwards compatibility came up. Example two discussions: * https://github.com/apache/airflow/pull/27826 * https://github.com/apache/airflow/pull/27067 I think we