o-nikolas commented on PR #27067: URL: https://github.com/apache/airflow/pull/27067#issuecomment-1289736364
> > The cruelty of SemVer I suppose! > > I think this is just the price to pay to make our users confident they can upgrade without worrying about incompatibilities. This is the main reason we adopted SemVer, so that we can convinvce our users, that they should migrate early and often without a fear of doing so. I think (based on our survey - https://airflow.apache.org/blog/airflow-survey-2022/ ) we mostly succeeded as the distribution of Airlfow versions is getting very much to what I expected to be (very similar to Python version distribution - with the very same reason behind I think), where users are migrating to newer versions way faster than they did in Airlfow 1.10. I believe adopting SemVer and spreading the word about 'safer" migrations play important part in it (on top of adding really cool features of course). Of course with Providers introducing their own SemVer it makes it more difficult to migrate sometimes, but this is one more reason to migrate often and early - becasue you can address provider incompat iblities one-by-one. This is the "story" we keep on telling our users and using SemVer is part of that "story". It was a bit of a throw-away comment on my end (which often doesn't translate well over the internet :laughing: ), so that is my bad. But I am actually a huge proponent of SemVer. As a developer, knowing that I can upgrade to any minor version without compatibility issues is an almost unmeasurably great thing. > > I think as long as "cruelty" does not make a havy toll on maintianers to keep the backwards compatibility, this is not cruel really. Also I think we should figure ways where maintinag compatibility is done in the way that it does not have any impact on maintainability ( I think #26179 is a very good example of that - where the way we provide the compatibilty hsa prety much zero overhead on maintenance). Whenever possible I think we should find a way to keep compatibility in the way that has similar zero-overhead and does not make further development problematic. I see that as much better approach than removing deprecations lightly. And quite often it is a one-time effort to figure out how to do it without leaving a lot of maintainance effort to keep it (for example we can move deprecated code to separate packages/folders where it can be all-but-forgotten). I think this is a big one, the more legacy code you pull forward the more complicated your code becomes to maintain and test. Optimizations that reduce overhead like #26179 are not always possible, but we should definitely prioritize those if they are. But also, IMHO, we shouldn't be afraid of major upgrades if and when it becomes valuable to do so (to be clear though, I'm not saying that time is now, or even close to now). > Also one more point -- getting to the subject of this PR. > > I think the way it is currently done is a **very good one**. I personally think, there is no reason to document deprecated parameters, classes etc. This is precisely what we've done in #26179 - while the deprecated classes continue to work via dynamic attrs, you do not see them any more. This is one of the best ways you can handle deprecations - if someone used it in the past, they will continue to work, but lack of documentation on them and lack of "easy way' of using it will make new users ... not use them. While we keep compatibility. This is precisely the behaviour we want to induce: > > * make 'old users" continue without doing changes if they want (but continue to be annoyed by warnings) > > * make "new users" not even aware that there is a deprecated parameter they can use > > > I think it is good as it is now. I think there is a risk to incrementing the amount of un-documented/hidden behaviour and it is a double edged sword. It does mean you can get away with fewer major upgrades but also users will ultimately run into these old bits of functionality (by accident or by choice) and it can be confusing for them to navigate from there forward if things like documentation are wiped away. To me this is a bit like security through obscurity. In this particular case, we now have a metric that is being emitted to users and they 1) shouldn't use it and 2) have no documentation for it if they do notice it or start using it. So IMHO, we need to make a call whether the goal should be to ultimately remove this code, or decide to keep it long term for backcompat but in that case I'd actually like to see it added back to our docs as deprecated but while still describing what it is and what it does, instead of ignoring its existence. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
