I don't have any issue with this in general, in fact I think it's not a bad idea to trim out older unused/unmaintained providers. But what are the criteria for marking a provider package as unmaintained? Is it simply "once a package becomes a blocker AND nobody has stepped up to fix it in [two weeks?]". Also, to clarify, "unmaintained" in this context isn't going to prevent current users from using it, it's just a notation indicating that nobody is actively updating/upgrading it, correct?
- ferruzzi ________________________________ From: Ash Berlin-Taylor <a...@apache.org> Sent: Tuesday, March 28, 2023 7:11 AM To: dev@airflow.apache.org; Daniel Standish Subject: RE: [EXTERNAL][DISCUSS] Exclude some providers that hold us back from releasing CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. Yeah agreed. Each provider should be treated like a separate project/release - it's just that we batch up releases to save time of the release manager. -a On 28 March 2023 07:05:13 BST, Daniel Standish <daniel.stand...@astronomer.io.INVALID> wrote: >It seems reasonable to me. > >On Mon, Mar 27, 2023 at 12:02 AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> Hello Everyone, >> >> TL;DR; I wanted to raise a discussion and make a proposal about option >> to skip some niche providers of our from releasing if they are holding >> us back, regarding the dependencies >> >> We are going through some troubles with dependencies of our providers >> - mostly around some outdated dependencies which are used for some - >> often niche - providers of ours. We mitigated the problem for a while >> but now when Google team is working on a major bump of all the >> dependencies of google provider: >> https://github.com/apache/airflow/pull/30067 we have some other >> non-google providers that holds us back from upgrading those. This is >> mainly about the protobuf dependency (all the new google dependencies >> will have protobuf > 4 and there are few dependencies that have >> protobuf < 4. >> >> There are few ways we can deal with this in the order of approach: >> >> 1) we can replace dependencies with other dependencies which have the >> problem removed >> 2) we can (and we do) raise the issue with the respective dependencies >> 3) If the feature, the provider depends on is somewhat optional - we >> can make it so >> 4) finally if those are not successful we can disable the provider >> from further releases (until the problem is fixed) >> >> The first 3 are not really something we need to decide on specifically >> (and we are going to individually work on fixing those if possible). >> >> But I have a question, if we are ok to apply 4) >> >> Good example to look at is yandex provider - I just opened an issue >> for it whether they are planning to lift the limit to protobuf: >> https://github.com/yandex-cloud/python-sdk/issues/71 I think if we do >> not get a response and update in a few days/week we might need to >> disable the provider from next releases and testing. This is just an >> example, we might have other cases similar, I just wanted to discuss >> our approach there. >> >> What I propose is that in this case: >> >> a) we deliberately mark the provider as not maintained any more (that >> will include documentation update markingi it so >> >> b) we remove it from contributing to our dependencies and generating >> our constraints >> >> c) we stop running any tests with it >> >> d) old relasess will of course remain and the users will be able to >> use it for as long as they will be able to keep it conflict-free (but >> our constraints will not help with it) >> >> This will help us to move forward with some dependencies, but not >> being held-back by them. >> >> WDYT? >> >> J. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org >> For additional commands, e-mail: dev-h...@airflow.apache.org >> >>
