> Will the people who maintain the providers' packages have the commiter
Nope - similarly as we do in v2-2* or what we did in 1.10.* cherry-picking can be done in separate branches (and in this case in forks). Then the branch can be fast-forwarded by the committer in the "airflow" repo. No problem with that. J. On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <[email protected]> wrote: > Will the people who maintain the providers' packages have the commiter > status? I guess it is necessary for people to have write access to the > repository and therefore to be able to make cherry-pick changes to the > branch. > > pon., 20 cze 2022 o 09:13 Elad Kalif <[email protected]> napisał(a): > > > > +1 > > From my side the proposal handles all concerns I raised in previous > threads. > > I think mixed-governance is a step in the right direction. > > > > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <[email protected]> wrote: > >> > >> Hello everyone, > >> > >> This is a follow-up after a few discussions started about providers > that were put on hold around the summit. I held a number of discussions > during theSummit and after, and as result I think I have a proposal that > can move forward some of the "stalled" decisions we need to make. > >> > >> TL;DR; > >> > >> My proposal is a "mixed-governance" model where "stakeholders" are more > responsible for cherry-picking and testing their providers (including > system testing) while Airflow PMC members will continue to be responsible > for releasing them. > >> > >> Why do we need that? > >> > >> Google, Amazon and possibly others teams who are interested in > maintaining more backwards compatible versions of their providers will > commit to make PRs of the cherry-picks for older release branches of their > providers. Those providers we release in parallel with the latest versions > during the normal provider cycle. We can deprecate changes more > aggressively in the "latest" release if we do that. > >> > >> Those cherry-picked PRs will be driven, tested and performed by the > stakeholder teams (Google/Amazon, Databricks, others) and will only contain > cherry-picks, while we - as PMC - will release them following the ASF rules > (this is very important for the ASF to follow strict release policies > regarding who and how performs releases). > >> > >> This also allows us to introduce similar rules for new provider's > acceptance for new providers for "main releases". It also allows running > the "system tests" for the provider under control of the stakeholder (after > applying AIP-47 changes). > >> > >> Example 1: Google team can cherry-pick changes to a google-provider-6 > branch and then we release a google 6.8.1 or 6.9.0 provider with some of > the bug-fixes and features (together with - say latest 8.1.0). > >> > >> Example 2: DataLake provider from Databricks - can get accepted if > Databricks commits to maintaining it. We will release the provider as long > as Databricks maintains it. > >> > >> Longer context: > >> > >> I have - in my mind so far - a longer roadmap for providers that will > lead them to be separated from the core and I want to write an AIP about > that soon. This AIP will detail all the steps needed - I will work with > multiple interested parties on it and it will take some time to agree and > complete. But I want to start with something tangible that will solve > quite a few problems that were raised recently and something that seems to > be possible to be solved in the current provider release cycle (till the > end of June) and test some of the governance approach. > >> > >> This proposal simply builds on our semver approach - we do not change > it, we just start releasing some providers (those that have some backing > from stakeholders) in more than one version - including "latest" and > earlier, more backwards-compatible branches. Not all providers - just some. > Not all branches - just those that the stakeholders will commit to maintain. > >> > >> We need such a commitment from stakeholders, because we - as the > Airflow community and maintainers, want to only actively maintain the > latest releases, where it is in the interest of the stakeholders to > cherry-pick and test also earlier, more backwards compatible releases of > their choice. > >> > >> What problems this proposal solves: > >> > >> * Problem 1: DEPRECATION REMOVAL > >> > >> we can remove deprecations faster in "main" versions of the providers - > no need to introduce a deprecation policy - the stakeholders for the > providers will take care about cherry-picking and maintaining more > "backwards-compatible" versions. We are free to remove deprecations in > major releases (in a cherry-pickable way of course). > >> > >> * Problem 2: PROVIDERS DIVERGENCE > >> > >> We avoid the problem (already happened with the composer release) that > the stakeholders in a given provider had to release their own version which > was not available in their community - with some cherry-picks. We want to > avoid "diverging" there - by releasing the cherry-picked providers by the > community, we also give other users an opportunity to follow "slower" > deprecation policies for as long as it is maintained. > >> > >> * Problem 3: PROVIDERS GOVERNANCE MODEL > >> > >> We are going to test a governance model that we might apply when we > split providers. We are talking about it for quite some time - but this is > what helps us to test the model where stakeholders provide more > "maintenance" while the community still takes care about releases. We (as > community) can commit to releasing such a version of a provider as long as > the stakeholder will actively maintain it. We can stop at any moment if we > do not have support from the stakeholder. If it works - we can keep it as a > long-term solution. In the future we can think of other scenarios (passing > ownership of a provider to stakeholders who want it - providing we want it > too) but we can decide about it when we learn from the mixed-governance > model and see if it works. > >> > >> * Problem 4: ACCEPTING NEW PROVIDERS > >> > >> If this is an acceptable approach - we can also apply a very similar > governance model to adding new providers and that should unblock some of > the PRs that are waiting for our decision. Knowing that we are going to > split and that we can expect "commitment" from a stakeholder, we should be > able to accept new providers. This might be possible assuming that the > stakeholder will make a similar commitment - but for new providers, that > commitment might also have to cover reviewing and testing new changes. We > might also decide as a community to stop releasing new providers there if > such support is missing. This way we can set the expectations we have as a > community for new providers - we will release them as long as the > stakeholder will actively make sure it is maintained. > >> > >> * Problem 5. SPLITTING PROVIDERS FROM CORE > >> > >> We all know we want to split providers from core. By introducing > mixed-governance we can test if it will work for the providers before we > split them. It will take some time (and detailed AIP) to split, but in the > meantime we can see if we will be able to apply the mixed-governance after > the split. We will see if we can agree when it comes to expectations and > find solutions before we actually split. > >> > >> J. >
