And BTW. airflow/contrib/operators/ecs_operator.py is potentially already broken multiple times - Every time we release Amazon major version of provider, it might break if you relied on it being backwards compatible.
On Thu, Aug 4, 2022 at 12:22 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Not if you add install `airflow[contrib]` - then it won't break. > > On Thu, Aug 4, 2022 at 12:21 PM Ash Berlin-Taylor <a...@apache.org> wrote: > >> Looking at the PR I see you are talking specifically about removing >> airflow/contrib/operators/ecs_operator.py. >> >> I disagree with you that this doesn't count as a breaking change. >> >> For example, if we remove that file: >> >> I have a dag on Airflow 2.3. I upgrade to 2.4. My dag breaks. >> >> That is 100% a breaking change to me. >> >> -a >> >> On Thu, Aug 4 2022 at 11:08:14 +01:00:00, Ash Berlin-Taylor < >> a...@apache.org> wrote: >> >> One question: Are you talking about removing things from airflow.contrib, >> or things already with in airflow.providers.*? >> >> -a >> >> On Thu, Aug 4 2022 at 11:55:52 +02:00:00, Jarek Potiuk <ja...@potiuk.com> >> wrote: >> >> Hello everyone, >> >> Following the discussion in https://github.com/apache/airflow/pull/25413 >> I have a proposal. >> >> Why don't we remove all "contrib" and other Airflow 1.10 deprecated >> classes to a separate package and add dependency to that package as >> [contrib] or [deprecated] extra in Airflow - and release one of the >> next 2.* (2.4/2.5) airflow versions without those classes ? >> >> This should be possible I think (We could do it for all "contrib" easily >> and for some other individual classes in "operators" and "hooks" that would >> likely require a little dynamic python magic to not override the folders). >> >> There are a number of benefits: >> >> 0) lots of old, defunc mostly deprecation code can be removed from the >> main repo - including lots of tests that verify that. >> >> 1) new users would not even know about those classes/contrib - less >> confusion >> >> 2) many of those "contrib" classes are not backwards-compatible with old >> 1.10 classes already as we had many "major" releases in many providers - >> so this might be a little misleading for those who are still in 1.10 that >> they can easily "migrate" without making any changes >> >> 3) there are many users who even now use "contrib" and we could use the >> opportunity to "guide them" into migration. To make it smoother, we could >> likely implement dynamic attribute checking in packages and raise >> appropriate instructions to those who still use it and migrate to 2.5 or >> 2.6 (and they will still have the option to install the "contrib" package). >> The instructions could be the same as in deprecation messages today (but >> they would fail in case the "contrib" package is not installed) >> >> 4) We give a great tool for admins of Airflow installation. Currently the >> admins have no tools to force their users to switch-off from using contrib >> if they still do. But with this one they will simply be able to install >> airflow without the "contrib" package and the users will have to adapt. We >> can even provide those "Admins" instructions on how to build your own >> "contrib" package if you want to do it gradually and ask your users to >> remove class-by-class or whatever way you want. >> >> Technically - we are not breaking SemVer compatibility - you can still >> get back to the contrib if you install the separate package. So we can do >> it without bumping Major version >> >> WDYT? >> >> J. >> >> >> >> >> >> >> >>
