Aside from a few details that still need refinement, I support this idea. I believe it’s the only viable way to continue expanding the number of integrations Airflow supports without overloading the provider release manager. The responsibility for releasing providers would be scaled out, rather than concentrated as it is today, which I see as a great improvement. That said, I agree with Jarek’s comments—responsibilities across individuals will need to be clearly defined, agreed upon, and well understood to ensure the Airflow ecosystem continues to function smoothly.
On 2025/09/17 12:39:45 Stephen Mallette wrote: > As a more casual observer of what happens here in Airflow, I don't have any > first hand knowledge of the problems faced beyond what was described in the > Motivation section of the proposal. That said, I did read through it and > was curious about the deprecation path that was described. I understand > that the path leaves open room for exception and that the quantitative > definitions are likely there to help guide PMC decisions, but I wonder if > it would make sense to describe wider exceptions for more mature providers > that don't require lots of new features/maintenance. That might help reduce > the scope of the PMC quarterly provider review while also reducing the > chance of providers gaming the system and/or sparing them a bit of worry > that they miss a quarterly chance to discuss an exception with the PMC. > > > > On Tue, Sep 16, 2025 at 9:14 PM Jarek Potiuk <[email protected]> wrote: > > > Comment from my side: I think this is a really good idea to discuss > > **what** we need from the community side to make it "easy" for providers to > > be contributed and maintained - without over-burdening the maintainers. > > This is by far the most important aspect of this proposal - and I think it > > would be fantastic if we hear feedback from Amazon, Google. Astronomer, > > Teradata and others who recently submitted providers to Airflow - and of > > course all maintainers that might be worried about impact it will have on > > our maintenance efforts. > > > > I think before we approve this one and start following it - all parties > > involved (maintainers, release managers ,contributors, PMC members, > > stakeholders) will need to agree on understanding: > > > > * what is required from them (everyone - stakeholders, contributors. > > maintainers, PMC members, release managers will have their parts in the > > process). > > * what they will have to commit to > > * what kind of tooling will be needed to make it as efficient as possible > > without overburdening everyone > > > > We are way ahead with tooling and modularisation of airflow than we were a > > year ago and I think changes and improvements in the tooling and process > > updates needed to get the process in place are within our reach now. This > > is the reason why I think it's the **right** time to discuss it. It will > > take a couple of months to complete all the needed changes, but it all > > seems doable. > > > > Also - I have discussed it with Apache Software Foundation leadership at > > Community Over Code in Minneapolis - and they are interested and supported > > in us trying this approach where we want to get Stakeholders more involved > > and more accountable, while keeping the Apache Way approach, so we should > > have green light from the Foundation on it. > > > > J. > > > > > > On Tue, Sep 16, 2025 at 12:02 PM Vikram Koka via dev < > > [email protected]> > > wrote: > > > > > Dear Airflowers, > > > > > > Every few days, we get a request for adding a new Provider (aka > > > Integration) into Apache Airflow. We would like to say yes to most if not > > > all of these, but we have had challenges in being able to absorb many > > more > > > providers into Airflow. > > > > > > We have had multiple conversations around this in the past, and we would > > > like to discuss a proposal to enable us to adopt many more integrations > > to > > > Airflow. At this stage, this is an early draft intended to get your > > > feedback. > > > > > > I know a lot of us are working on releasing Airflow 3.1.0, so I don't > > > expect a lot of conversation this week, but wanted to socialize this with > > > the broader community at the same time. > > > > > > Proposal > > > < > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Provider+lifecycle+update+proposal > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Provider+lifecycle+update+proposal > > > > > > Best regards, > > > Vikram, Jarek, Kaxil, Pavan, and Ash > > > -- > > > > > > Vikram Koka > > > Chief Strategy Officer > > > Email: [email protected] > > > > > > > > > <https://www.astronomer.io/> > > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
