Yeah agreed. Each provider should be treated like a separate project/release - it's just that we batch up releases to save time of the release manager.
-a On 28 March 2023 07:05:13 BST, Daniel Standish <daniel.stand...@astronomer.io.INVALID> wrote: >It seems reasonable to me. > >On Mon, Mar 27, 2023 at 12:02 AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> Hello Everyone, >> >> TL;DR; I wanted to raise a discussion and make a proposal about option >> to skip some niche providers of our from releasing if they are holding >> us back, regarding the dependencies >> >> We are going through some troubles with dependencies of our providers >> - mostly around some outdated dependencies which are used for some - >> often niche - providers of ours. We mitigated the problem for a while >> but now when Google team is working on a major bump of all the >> dependencies of google provider: >> https://github.com/apache/airflow/pull/30067 we have some other >> non-google providers that holds us back from upgrading those. This is >> mainly about the protobuf dependency (all the new google dependencies >> will have protobuf > 4 and there are few dependencies that have >> protobuf < 4. >> >> There are few ways we can deal with this in the order of approach: >> >> 1) we can replace dependencies with other dependencies which have the >> problem removed >> 2) we can (and we do) raise the issue with the respective dependencies >> 3) If the feature, the provider depends on is somewhat optional - we >> can make it so >> 4) finally if those are not successful we can disable the provider >> from further releases (until the problem is fixed) >> >> The first 3 are not really something we need to decide on specifically >> (and we are going to individually work on fixing those if possible). >> >> But I have a question, if we are ok to apply 4) >> >> Good example to look at is yandex provider - I just opened an issue >> for it whether they are planning to lift the limit to protobuf: >> https://github.com/yandex-cloud/python-sdk/issues/71 I think if we do >> not get a response and update in a few days/week we might need to >> disable the provider from next releases and testing. This is just an >> example, we might have other cases similar, I just wanted to discuss >> our approach there. >> >> What I propose is that in this case: >> >> a) we deliberately mark the provider as not maintained any more (that >> will include documentation update markingi it so >> >> b) we remove it from contributing to our dependencies and generating >> our constraints >> >> c) we stop running any tests with it >> >> d) old relasess will of course remain and the users will be able to >> use it for as long as they will be able to keep it conflict-free (but >> our constraints will not help with it) >> >> This will help us to move forward with some dependencies, but not >> being held-back by them. >> >> WDYT? >> >> J. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> For additional commands, e-mail: dev-h...@airflow.apache.org >> >>
