Good point. But we got it covered.

We actually can do it regardless and we already have a process for it
in the "mix-governance" process - branching off from the old version
and applying a fix is perfectly possible for this case.

J.



On Wed, Mar 29, 2023 at 8:29 PM Elad Kalif <elad...@apache.org> wrote:
>
> just one thought (sorry for bringing it just now)
> but we should also clarify what happens if during suspension a security
> risk is discovered.
> since it probably means we won't be able to raise/merge PR I think we
> should specifically mention that security risks can not be handled and
> users of suspended provider should be aware of this
>
> On Wed, Mar 29, 2023 at 9:25 PM Jarek Potiuk <ja...@potiuk.com> wrote:
>
> > Yeah. I proposed (see the LAZY CONSENSUS thread) that the
> > argumentation on suspension of the provider is followed by a
> > LAZY_CONSENSUS or VOTE - following the regular rules we have here.
> > This is the usual way we are resolving issues in Apache projects and
> > this is IMHO classic vote on procedural issues from here:
> > https://www.apache.org/foundation/voting.html
> >
> > This way it will never be single-handedly decided by anyone, and the
> > community will have a say and can either break a consensus or the
> > majority might vote against suspension.
> >
> > On Wed, Mar 29, 2023 at 8:18 PM Ferruzzi, Dennis
> > <ferru...@amazon.com.invalid> wrote:
> > >
> > > Yeah, I can see that.  I'd vote plus-one, but for the sake of
> > discussion:  my only real (non-blocking) concern is that there may be some
> > (perceived?) bias in the process if there isn't some kind of set policy.
> > It inherently favors the providers of the companies that have dedicated
> > contributors, for one.  And if we flip the coin on your example and Yandex
> > does respond and submits an update that Google doesn't yet support (ok,
> > that's maybe a bit of a stretch, but bear with me) do we really suspend the
> > bigger/popular provider?
> > >
> > > As long as we are alright handling the potential issues like adults,
> > then I think it's a solid idea.  I've just seen too many groups of grown
> > adults devolve into children when they perceive favoritism or bias.
> > >
> > >
> > >  - ferruzzi
> > >
> > >
> > > ________________________________
> > > From: Jarek Potiuk <ja...@potiuk.com>
> > > Sent: Wednesday, March 29, 2023 11:06 AM
> > > To: dev@airflow.apache.org
> > > Subject: RE: [EXTERNAL][DISCUSS] Exclude some providers that hold us
> > back from releasing
> > >
> > > CAUTION: This email originated from outside of the organization. Do not
> > click links or open attachments unless you can confirm the sender and know
> > the content is safe.
> > >
> > >
> > >
> > > I see a consensus building up :). So I will formally call for a lazy
> > > consensus in a moment with PR where I will spell out the details.
> > >
> > > @dennis @danniel   -> It will be difficult to get very predetermined
> > > criteria I think. There might be various reasons why we might think we
> > > decide that "enough is enough" for old dependencies (note that this is
> > > only for providers that cling to the dependencies where other
> > > providers want to upgrade to newer versions and they are held back -
> > > it is never in the other direction).
> > >
> > > But I think we do not have to have such criteria defined. Such change
> > > will be reversible and in the best case if the root cause of the
> > > problem is fixed quickly (whether with the involvement of a 3rd-party
> > > or not), we can always bring such providers back.
> > > This at most will result in not releasing new changes to it in the
> > > next wave of providers. But such changes won't happen - because in
> > > case of a provider that blocks us from doing something, changes to it
> > > will be rejected anyway as tests will be disabled - we will automate
> > > some of that).
> > >
> > > In this context I propose 1 week for notification. Should be enough,
> > > and since it is reversible. It's really a "soft" removal - so actually
> > > a "suspension" that can be "resumed".  We are not going to remove the
> > > code (for now at least but we might have some criteria for that in the
> > > future) - so bringing a change which fixes it (usually by a dependency
> > > release that unblocks the conflict) should be enough to "resume" such
> > > a provider.
> > >
> > > PR/Lazy consensus follows.
> > >
> > > J.
> > >
> > > On Wed, Mar 29, 2023 at 3:33 PM Josh Fell
> > > <josh.d.f...@astronomer.io.invalid> wrote:
> > > >
> > > > Agreed, this seems like a good move.
> > > >
> > > > On Wed, Mar 29, 2023 at 9:19 AM Beck, Vincent
> > <vincb...@amazon.com.invalid>
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > On 2023-03-29, 2:26 AM, "Elad Kalif" <elad...@apache.org <mailto:
> > > > > elad...@apache.org>> wrote:
> > > > >
> > > > >
> > > > > CAUTION: This email originated from outside of the organization. Do
> > not
> > > > > click links or open attachments unless you can confirm the sender
> > and know
> > > > > the content is safe.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I agree that we should exclude providers that block us.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Mar 28, 2023 at 9:58 PM Ferruzzi, Dennis
> > > > > <ferru...@amazon.com.inva <mailto:ferru...@amazon.com.inva>lid>
> > wrote:
> > > > >
> > > > >
> > > > > > I don't have any issue with this in general, in fact I think it's
> > not a
> > > > > > bad idea to trim out older unused/unmaintained providers. But what
> > are
> > > > > the
> > > > > > criteria for marking a provider package as unmaintained? Is it
> > simply
> > > > > > "once a package becomes a blocker AND nobody has stepped up to fix
> > it in
> > > > > > [two weeks?]". Also, to clarify, "unmaintained" in this context
> > isn't
> > > > > > going to prevent current users from using it, it's just a notation
> > > > > > indicating that nobody is actively updating/upgrading it, correct?
> > > > > >
> > > > > >
> > > > > > - ferruzzi
> > > > > >
> > > > > >
> > > > > > ________________________________
> > > > > > From: Ash Berlin-Taylor <a...@apache.org <mailto:a...@apache.org>>
> > > > > > Sent: Tuesday, March 28, 2023 7:11 AM
> > > > > > To: dev@airflow.apache.org <mailto:dev@airflow.apache.org>; Daniel
> > > > > Standish
> > > > > > Subject: RE: [EXTERNAL][DISCUSS] Exclude some providers that hold
> > us back
> > > > > > from releasing
> > > > > >
> > > > > > CAUTION: This email originated from outside of the organization.
> > Do not
> > > > > > click links or open attachments unless you can confirm the sender
> > and
> > > > > know
> > > > > > the content is safe.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Yeah agreed.
> > > > > >
> > > > > > Each provider should be treated like a separate project/release -
> > it's
> > > > > > just that we batch up releases to save time of the release manager.
> > > > > >
> > > > > > -a
> > > > > >
> > > > > > On 28 March 2023 07:05:13 BST, Daniel Standish
> > > > > > <daniel.stand...@astronomer.io.inva <mailto:
> > > > > daniel.stand...@astronomer.io.inva>LID> wrote:
> > > > > > >It seems reasonable to me.
> > > > > > >
> > > > > > >On Mon, Mar 27, 2023 at 12:02 AM Jarek Potiuk <ja...@potiuk.com
> > > > > <mailto:ja...@potiuk.com>> wrote:
> > > > > > >
> > > > > > >> Hello Everyone,
> > > > > > >>
> > > > > > >> TL;DR; I wanted to raise a discussion and make a proposal about
> > option
> > > > > > >> to skip some niche providers of our from releasing if they are
> > holding
> > > > > > >> us back, regarding the dependencies
> > > > > > >>
> > > > > > >> We are going through some troubles with dependencies of our
> > providers
> > > > > > >> - mostly around some outdated dependencies which are used for
> > some -
> > > > > > >> often niche - providers of ours. We mitigated the problem for a
> > while
> > > > > > >> but now when Google team is working on a major bump of all the
> > > > > > >> dependencies of google provider:
> > > > > > >> https://github.com/apache/airflow/pull/30067 <
> > > > > https://github.com/apache/airflow/pull/30067> we have some other
> > > > > > >> non-google providers that holds us back from upgrading those.
> > This is
> > > > > > >> mainly about the protobuf dependency (all the new google
> > dependencies
> > > > > > >> will have protobuf > 4 and there are few dependencies that have
> > > > > > >> protobuf < 4.
> > > > > > >>
> > > > > > >> There are few ways we can deal with this in the order of
> > approach:
> > > > > > >>
> > > > > > >> 1) we can replace dependencies with other dependencies which
> > have the
> > > > > > >> problem removed
> > > > > > >> 2) we can (and we do) raise the issue with the respective
> > dependencies
> > > > > > >> 3) If the feature, the provider depends on is somewhat optional
> > - we
> > > > > > >> can make it so
> > > > > > >> 4) finally if those are not successful we can disable the
> > provider
> > > > > > >> from further releases (until the problem is fixed)
> > > > > > >>
> > > > > > >> The first 3 are not really something we need to decide on
> > specifically
> > > > > > >> (and we are going to individually work on fixing those if
> > possible).
> > > > > > >>
> > > > > > >> But I have a question, if we are ok to apply 4)
> > > > > > >>
> > > > > > >> Good example to look at is yandex provider - I just opened an
> > issue
> > > > > > >> for it whether they are planning to lift the limit to protobuf:
> > > > > > >> https://github.com/yandex-cloud/python-sdk/issues/71 <
> > > > > https://github.com/yandex-cloud/python-sdk/issues/71> I think if we
> > do
> > > > > > >> not get a response and update in a few days/week we might need
> > to
> > > > > > >> disable the provider from next releases and testing. This is
> > just an
> > > > > > >> example, we might have other cases similar, I just wanted to
> > discuss
> > > > > > >> our approach there.
> > > > > > >>
> > > > > > >> What I propose is that in this case:
> > > > > > >>
> > > > > > >> a) we deliberately mark the provider as not maintained any more
> > (that
> > > > > > >> will include documentation update markingi it so
> > > > > > >>
> > > > > > >> b) we remove it from contributing to our dependencies and
> > generating
> > > > > > >> our constraints
> > > > > > >>
> > > > > > >> c) we stop running any tests with it
> > > > > > >>
> > > > > > >> d) old relasess will of course remain and the users will be
> > able to
> > > > > > >> use it for as long as they will be able to keep it
> > conflict-free (but
> > > > > > >> our constraints will not help with it)
> > > > > > >>
> > > > > > >> This will help us to move forward with some dependencies, but
> > not
> > > > > > >> being held-back by them.
> > > > > > >>
> > > > > > >> WDYT?
> > > > > > >>
> > > > > > >> J.
> > > > > > >>
> > > > > > >>
> > ---------------------------------------------------------------------
> > > > > > >> To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> > <mailto:
> > > > > dev-unsubscr...@airflow.apache.org>
> > > > > > >> For additional commands, e-mail: dev-h...@airflow.apache.org
> > <mailto:
> > > > > dev-h...@airflow.apache.org>
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> > > > > For additional commands, e-mail: dev-h...@airflow.apache.org
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> > > For additional commands, e-mail: dev-h...@airflow.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
> > For additional commands, e-mail: dev-h...@airflow.apache.org
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org
For additional commands, e-mail: dev-h...@airflow.apache.org

Reply via email to