+1 for removing all deprecations from providers before Airflow 2.11. Unfortunately, we don't have a policy to remove providers' deprecations, as we see in some providers we still maintain deprecated features after more than 4 major releases, this is something we have discussed multiple times in the past, and I think we should re-discuss it and take a decision before releasing Airflow 3. I propose to remove deprecations after 2 major releases and to cover what Jarek mentioned about some of the providers managed by stakeholders, we can consider this as the default policy, which we can replace with a provider-level policy.
On Sun, Nov 17, 2024 at 11:56 AM Jens Scheffler <j_scheff...@gmx.de.invalid> wrote: > Strong +1 > > Which means for almost all providers we should attempt to release a > "breaking version" and effectively remove deprecated functions. > > Just looking into two examples: > > - providers/src/airflow/providers/standard/operators/bash.py:170 - > "skip_exit_code is deprecated. Please use skip_on_exit_code" > > - providers/src/airflow/providers/docker/decorators/docker.py:130 - > "`use_dill` is deprecated and will be removed in a future version." > > ..and then airflow-providers-docker would be bumped to 4.0.0... > > Correct understood? > > Jens > > On 17.11.24 10:59, Jarek Potiuk wrote: > > +1 . Very good and well thought proposal coming from a lot of discussions > > we held on slack with a number of people. > > > > Great for the users and also recognizing our stakeholders needs and their > > willingness to maintain their providers in the way it is best for their > > user base (both within their services and outside) > > > > While we know such an exclusion is now requested (and very consistent and > > good policy proposed) by Google (we should likely register the policy > > somewhere in Google provider's readme BTW) - similar ones.could be added > by > > other stakeholders not only as big as Google, Amazon and Microsoft. > > > > But also smaller providers could benefit from it if they had any stake > > lholder who would like to define and follow through their deprecation > > policy - for example Teradata could do the same already but others are > more > > than welcome to follow and both - set the rules and commit to maintaining > > them. This is a great opportunity for others to step and raise their hand > > and say 'hey I would like to take more responsibility for that provider > and > > here are the rule I propose and commit to keep it maintained. > > > > BTW. I know Microsoft team (kudos to Ambika!) is working on system test > > dashboard (yay!!!) so finally we may get have all the stakeholders from > the > > big clouds involved and active in our community, and also having > Astronomer > > as leading 3.0 and of course biggest contributor, and all the individual > > contributors - that makes it a very nice, sustainable community around > > Airflow and makes me really bullish on the future of Airflow! > > > > It's a really great to see how we all collaborate respect and recognize > > each-others needs. > > > > J > > > > niedz., 17 lis 2024, 09:30 użytkownik Elad Kalif <elad...@apache.org> > > napisał: > > > >> Hello everyone, > >> > >> We know from a previous survey that users tend to upgrade providers > >> together with Airflow core. This is also consistent with what I hear > from > >> talking to users in the community. Upgrading the provider separately is > >> done mostly when there is a need (relevant bug fix, needed feature, > etc...) > >> This is something that we should prepare for. While we treat providers > as > >> independent packages users may see this as part of Airflow core. > >> > >> I can say for myself that one of the challenges of upgrading 1.10 -> 2 > was > >> provider packages. > >> > >> My suggestion is to take the risk / problem away. If we remove all > >> deprecations users will have to handle them while they are on Airflow > 2.x > >> and before they are migrating to Airflow 3.x - To my perspective this > will > >> allow a smoother migration. > >> The reason I suggest to do it before Airflow 2.11 is because we treat > 2.11 > >> as a bridge release thus I'd like the constraints for this release to > have > >> the provider versions after removal of deprecations. > >> > >> This is a process we already started with cncf.kubernetes and amazon > >> providers that removed all their deprecations: > >> https://github.com/apache/airflow/pull/43689 > >> https://github.com/apache/airflow/pull/42450 > >> > >> *I'd like to focus the discussion around the following proposal:* > >> We will remove all deprecations from providers prior to Airflow 2.11 > >> (Providers that already did this recently don't have to do it again if > we > >> added new deprecations), Given the mixed governance model for providers > ( > >> > >> > https://github.com/apache/airflow/blob/main/PROVIDERS.rst#mixed-governance-model-for-3rd-party-related-community-providers > >> ) Stackholders who maintain a specific provider can ask to be excluded. > The > >> exclusion will be registered and referenced in the guides/tools we will > >> introduce for Airflow 3 migration. > >> > >> Regards, > >> Elad > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > For additional commands, e-mail: dev-h...@airflow.apache.org > >
