potiuk commented on code in PR #30359: URL: https://github.com/apache/airflow/pull/30359#discussion_r1152798064
########## README.md: ########## @@ -473,6 +477,38 @@ of the contributors to perform the cherry-picks and carry-on testing of the olde The availability of stakeholder that can manage "service-oriented" maintenance and agrees to such a responsibility, will also drive our willingness to accept future, new providers to become community managed. +### Suspending releases for providers + +In case a provider is found to require old dependencies that are not compatible with upcoming versions of +the Apache Airflow or with newer dependencies required by other providers, the provider's release +process can be suspended. + +This means: + +* The provider's status is set to "suspended" +* No new releases of the provider will be made until the problem with dependencies is solved +* Sources of the provider remain in the repository for now (in the future we might add process to remove them) Review Comment: > I wonder if we can find a method to early detect the dependencies which could block the providers release (for ex two major versions older than the latest version), in this case we can create warn issues and invite the maintainers of dependencies of the provider to update its dependencies. Yes this is already partially happening (and I think it's "good enough". We could potentially try to do it automatically to improve the current process. BUT This is a complex and difficult (and I think impossible) matter to do it automatically - it took `pip` two years and a [grant](https://pyfound.blogspot.com/2019/12/moss-czi-support-pip.html) of ~ 400.000 USD from Mozilla Foundation and Chan-Zuckerberg initiative (https://pyfound.blogspot.com/2019/12/moss-czi-support-pip.html) to hash out algorithm that tries to find and resolve any conflicts (and even that was not enough the effort including fixing a lot of teething problems took far more than that. I think proactively detecting and flagging keeping and flagging such problems two major version (or anything like that) before is probably at least an order of magnitude more complex task. And no-one of us would will to do it "proactively" and manually check 650+ dependencies we have. But I think it is also something we already do manually when we find those. Generally speaking we **know** (at least people who are contributing to providers) that some dependencies are holding us back for some cases. Good example was a google provider. In October 2022 I created the issue about google provider dependencies (related to our Python 3.11 support) https://github.com/apache/airflow/issues/27292 and I pinged google team several times since then stating that those have to be solved, as this is not only blockin 3.11 but also cause the "ads" problem to materialize (as it did). Our system is designed around automating and testing dependencies when they **can** be automatically upgraded and tested, and I think sometimes we have good reasons to stay behind, I think we will be able to detect and flag cases that need attention quite early and create. So yeah agree we **should** give early warnings about those and in most cases we do. And again - the "effect" of suspension on it's own is nothing more than a very strong warning. It does not have immediate effects - it's just excluding the provider from the future release and blocking any new changes coming. And a clear sign to maintainers "you have to do something". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
