Will the people who maintain the providers' packages have the commiter status? I guess it is necessary for people to have write access to the repository and therefore to be able to make cherry-pick changes to the branch.
pon., 20 cze 2022 o 09:13 Elad Kalif <[email protected]> napisał(a): > > +1 > From my side the proposal handles all concerns I raised in previous threads. > I think mixed-governance is a step in the right direction. > > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <[email protected]> wrote: >> >> Hello everyone, >> >> This is a follow-up after a few discussions started about providers that >> were put on hold around the summit. I held a number of discussions during >> theSummit and after, and as result I think I have a proposal that can move >> forward some of the "stalled" decisions we need to make. >> >> TL;DR; >> >> My proposal is a "mixed-governance" model where "stakeholders" are more >> responsible for cherry-picking and testing their providers (including system >> testing) while Airflow PMC members will continue to be responsible for >> releasing them. >> >> Why do we need that? >> >> Google, Amazon and possibly others teams who are interested in maintaining >> more backwards compatible versions of their providers will commit to make >> PRs of the cherry-picks for older release branches of their providers. Those >> providers we release in parallel with the latest versions during the normal >> provider cycle. We can deprecate changes more aggressively in the "latest" >> release if we do that. >> >> Those cherry-picked PRs will be driven, tested and performed by the >> stakeholder teams (Google/Amazon, Databricks, others) and will only contain >> cherry-picks, while we - as PMC - will release them following the ASF rules >> (this is very important for the ASF to follow strict release policies >> regarding who and how performs releases). >> >> This also allows us to introduce similar rules for new provider's acceptance >> for new providers for "main releases". It also allows running the "system >> tests" for the provider under control of the stakeholder (after applying >> AIP-47 changes). >> >> Example 1: Google team can cherry-pick changes to a google-provider-6 branch >> and then we release a google 6.8.1 or 6.9.0 provider with some of the >> bug-fixes and features (together with - say latest 8.1.0). >> >> Example 2: DataLake provider from Databricks - can get accepted if >> Databricks commits to maintaining it. We will release the provider as long >> as Databricks maintains it. >> >> Longer context: >> >> I have - in my mind so far - a longer roadmap for providers that will lead >> them to be separated from the core and I want to write an AIP about that >> soon. This AIP will detail all the steps needed - I will work with multiple >> interested parties on it and it will take some time to agree and complete. >> But I want to start with something tangible that will solve quite a few >> problems that were raised recently and something that seems to be possible >> to be solved in the current provider release cycle (till the end of June) >> and test some of the governance approach. >> >> This proposal simply builds on our semver approach - we do not change it, we >> just start releasing some providers (those that have some backing from >> stakeholders) in more than one version - including "latest" and earlier, >> more backwards-compatible branches. Not all providers - just some. Not all >> branches - just those that the stakeholders will commit to maintain. >> >> We need such a commitment from stakeholders, because we - as the Airflow >> community and maintainers, want to only actively maintain the latest >> releases, where it is in the interest of the stakeholders to cherry-pick and >> test also earlier, more backwards compatible releases of their choice. >> >> What problems this proposal solves: >> >> * Problem 1: DEPRECATION REMOVAL >> >> we can remove deprecations faster in "main" versions of the providers - no >> need to introduce a deprecation policy - the stakeholders for the providers >> will take care about cherry-picking and maintaining more >> "backwards-compatible" versions. We are free to remove deprecations in major >> releases (in a cherry-pickable way of course). >> >> * Problem 2: PROVIDERS DIVERGENCE >> >> We avoid the problem (already happened with the composer release) that the >> stakeholders in a given provider had to release their own version which was >> not available in their community - with some cherry-picks. We want to avoid >> "diverging" there - by releasing the cherry-picked providers by the >> community, we also give other users an opportunity to follow "slower" >> deprecation policies for as long as it is maintained. >> >> * Problem 3: PROVIDERS GOVERNANCE MODEL >> >> We are going to test a governance model that we might apply when we split >> providers. We are talking about it for quite some time - but this is what >> helps us to test the model where stakeholders provide more "maintenance" >> while the community still takes care about releases. We (as community) can >> commit to releasing such a version of a provider as long as the stakeholder >> will actively maintain it. We can stop at any moment if we do not have >> support from the stakeholder. If it works - we can keep it as a long-term >> solution. In the future we can think of other scenarios (passing ownership >> of a provider to stakeholders who want it - providing we want it too) but we >> can decide about it when we learn from the mixed-governance model and see if >> it works. >> >> * Problem 4: ACCEPTING NEW PROVIDERS >> >> If this is an acceptable approach - we can also apply a very similar >> governance model to adding new providers and that should unblock some of the >> PRs that are waiting for our decision. Knowing that we are going to split >> and that we can expect "commitment" from a stakeholder, we should be able to >> accept new providers. This might be possible assuming that the stakeholder >> will make a similar commitment - but for new providers, that commitment >> might also have to cover reviewing and testing new changes. We might also >> decide as a community to stop releasing new providers there if such support >> is missing. This way we can set the expectations we have as a community for >> new providers - we will release them as long as the stakeholder will >> actively make sure it is maintained. >> >> * Problem 5. SPLITTING PROVIDERS FROM CORE >> >> We all know we want to split providers from core. By introducing >> mixed-governance we can test if it will work for the providers before we >> split them. It will take some time (and detailed AIP) to split, but in the >> meantime we can see if we will be able to apply the mixed-governance after >> the split. We will see if we can agree when it comes to expectations and >> find solutions before we actually split. >> >> J.
