I think it's a good idea to codify the rules, indeed what Elad mentioned, we **mostly** follow a very similar approach, but it's not written anywhere.
Re: points of Elad: 1. I am ok with 6 months, but I agree here we should not have "strict" limits. It's fine to have some default (i.e. accumulate deprecating changes for some time - max. 6 months) - but I think the policy should have an option to apply exceptions - for example when we have a major change in many libraries and we need to make some breaking changes (ads case) we shall do it (but in this case we don't have to immediately remove all deprecations - they can wait for the regular (say 6 months) breaking change that will be coming. The users can - if needed - even in new versions of Airflow - downgrade to previous provider versions in case they have problem with it. I like the idea of having some time (6 months) where we can very safely say "OK, enough is enough, lets remove deprecations" - and this proposal addresses it nicely. Currently it's purely subjective judgment. One small thing here to clarify for "accumulating" deprecations - I think we should make sure the deprecation is present in already released minor/patchlevel version to be able to remove it (so even if it is deprecated 2 months ago but we had release with the deprecation, it's ok to remove it in the next breaking change. 2. Agree (and again here "strict" is not good I think). I'd treat that 6 months as a baseline. Before that - we should have very good reason to make a breaking change (and we might decide to do it partially only) - after - we are free to remove things that have been already announced as deprecated and we should consider making breaking change whenever we release a new version for whatever reason (but again - not mandatory). I's just an indication that tells us "Last breaking release was 6 months ago, it's now OK to remove all the deprecations without having an extremely good reason". Very much agree with 3. -> this should be the same as for other providers. We should adopt the same rule for all of them. And if we apply the adjustments above - we should - I think - be ok to apply it to any provider I think. J On Thu, Feb 29, 2024 at 12:20 PM Elad Kalif <elad...@apache.org> wrote: > This is not too much different than what we already do. > We deprecate then we remove. We almost never create breaking change without > deprecating first. > > I'd like to raise a few notes: > > 1. Major releases can happen frequently. I see no problem with it. This is > more a question of what are the changes. If google has 3 major releases one > with breaking change to ads, another to leveldb and 3rd to cloud these are > not related one to another and unlikely that there is 1 user that uses all > of these together. We create major releases when needed and we keep a close > eye and challenge why a change must be a breaking change. If possible we > will always prefer to be backward compatible. > 2. Time based deprecations are not a good approach. Some changes require > much more time to be acted upon and others may be trivial. > As a rule we almost never cut a breaking change release just to remove > deprecations. > 3. Setting a policy per provider is not the right way to go. We have a > shared governance model for provider handling and I as release manager will > do what I can to accommodate requests from companies who participate in > managing the provider. Which means that if Google wants to remove > deprecations and ask for a breaking change release most likely we will > accommodate their request. My point of view is that the company knows what > is best for their own users. I don't think that what is right for Google > must bind AWS or Microsoft. 1 policy goes against the shared > governance idea. > > > > On Thu, Feb 29, 2024 at 11:04 AM Eugen Kosteev <eu...@kosteev.com> wrote: > > > Hi. > > > > I would like to discuss/propose a deprecation policy for > > apache-airflow-providers-google package, mostly for deprecating Airflow > > operators (but not limited to it). > > > > *Some background:* > > Airflow google provider package (as most of other provider packages in > > Airflow) has a tendency to constantly evolve. > > There are cases when its features or operators/hooks have to be > deprecated. > > The typical example of such a case is “refactoring” which leads to the > > renaming of operators, parameters or public methods. Such refactorings > > maybe consequence of: > > - using a new dependent library (that causes some methods to be > deprecated) > > - using a new version of API (that causes some parameters to be > deprecated) > > - restructuring/migration of the operators (that causes some operators to > > be deprecated) > > Given the need of a “deprecation” procedure, it looks essential to > > establish policy around it to have users of Airflow google provider > package > > clearly understand when and how to adapt their DAGs. > > > > *Preambula (versioning of the package):* > > As mentioned in “Airflow’s release process and version policy” ( > > > > > https://airflow.apache.org/docs/apache-airflow/stable/release-process.html#airflow-s-release-process-and-version-policy > > ) > > google provider package (and others) should follow SemVer, meaning that > any > > breaking changes should be released together with bumping major version > of > > the package. > > The change is considered to be a breaking (in the context of google > > providers package) if a DAG that was working before, stops to work after > > the change. > > > > *My proposal (for discussion):* > > The entire procedure of deprecating (either method, parameter or > operator) > > consists of two steps: > > - Emission of the deprecation warning message > > Example of the message: > > “”” > > The “given” parameter is deprecated and will be removed after dd.mm.yyyy. > > Please use “this” parameter instead. > > “”” > > - Once the date of the deprecated method/parameter/operator is passed, it > > can be removed (together with bumping major version of the package) > > > > The format of the warning message may vary, but it has to contain: > > - What should be used instead > > - The date after which the method/parameter/operator will be removed > > > > *Additional considerations:* > > - Bumping a major version of the Airflow google provider package > shouldn’t > > happen very frequently (presumably not more frequently than 6 months), > > therefore when it happens it may accumulate removals of something that > was > > communicated to be removed even a couple of months ago. (related > discussion > > https://github.com/apache/airflow/blob/main/PROVIDERS.rst) > > Example: if today is 2025-12-20 and we are releasing new major version of > > the package, we can/should remove all deprecated > > methods/parameters/operators which have date prior to 2025-12-20 - it can > > be November 2025, October 2025 or even earlier, since previous bump of > > major version happened e.g. in summer of 2025. > > - By default all deprecations should allow a 6 months time period until > > they will be removed and not available. This period will give Airflow > users > > enough time and flexibility to update their DAGs before actual removal > > happens. > > On a case by case basis this period can be adjusted given specific > > circumstances (e.g. in case deprecation is because of underlying API > sunset > > which can happen earlier than in 6 months). > > > > Apart from deprecation warning messages that will be printed as logs, > this > > information will be put as well in: > > - Airflow documentation > > - Release notes of Airflow google provider package > > > > Please, let me know what you think about this. > > > > -- > > Eugene > > >
