> Cherry-picking to branch v2-2* or 1.10.* can only be done by the committers, because only they have write permission to the apache/airflow repository. As far as I know, Github does not allow us to grant write-only permissions to the selected branch.
Kamil - you misunderstood it. The branch will be in the FORK of those users's choice - not in airflow repo. committer will merge that branch in the same way we do as today - but with fast-forwarding rather than squashing. > We can keep these branches in forks managed by stakeholders teams, but I am afraid of the benefit that it will be then copied by us to our repository and then released by us. If the release was prepared by an external team, I think we should make it clear that it was prepared by another team, including by publishing on the Pypi account of the team that dealt with it. Yes this is exactly what I proposed. > I think that everything Apache PMC releases should be prepared and created fully within the apache / airflow repositories. If stakeholders team do not have such a possibility, we should figure out that these teams become part of the community, and therefore work together with the entire community, not in isolation. Only then will we be able to act in accordance with the Apache Way <https://www.apache.org/theapacheway/>, in particular each individual person will be able to contribute to the community as an individual, and not as a company or stakeholders team (Community of Peers) and no person will get special privileges just on the basis of their employment status (Earned Authority) Very true. And exactly follows my proposal :) J. On Mon, Jun 20, 2022 at 8:16 PM Kaxil Naik <[email protected]> wrote: > +1 -- We have discussed this during the Airflow Summit in-person with Ash, > Rafal (and his team), Jarek and I about this for a long time, and I think > this is a good step forward. > > Regards, > Kaxil > > On Mon, 20 Jun 2022 at 17:26, Kamil Breguła <[email protected]> wrote: > >> I think we should continue to be strictly vendor-neutral. No >> organization should be able to gain special privileges or control a >> project’s direction. >> >> pon., 20 cze 2022 o 18:14 Kamil Breguła <[email protected]> napisał(a): >> >>> Cherry-picking to branch v2-2* or 1.10.* can only be done by the >>> committers, because only they have write permission to the apache/airflow >>> repository. As far as I know, Github does not allow us to grant write-only >>> permissions to the selected branch. >>> >>> We can keep these branches in forks managed by stakeholders teams, but I >>> am afraid of the benefit that it will be then copied by us to our >>> repository and then released by us. If the release was prepared by an >>> external team, I think we should make it clear that it was prepared by >>> another team, including by publishing on the Pypi account of the team that >>> dealt with it. >>> >>> I think that everything Apache PMC releases should be prepared and >>> created fully within the apache / airflow repositories. If stakeholders >>> team do not have such a possibility, we should figure out that these teams >>> become part of the community, and therefore work together with the entire >>> community, not in isolation. Only then will we be able to act in accordance >>> with the Apache Way <https://www.apache.org/theapacheway/>, in >>> particular each individual person will be able to contribute to the >>> community as an individual, and not as a company or stakeholders team >>> (Community of Peers) and no person will get special privileges just on the >>> basis of their employment status (Earned Authority) >>> >>> >>> >>> pon., 20 cze 2022 o 15:54 Jarek Potiuk <[email protected]> napisał(a): >>> >>>> > Will the people who maintain the providers' packages have the commiter >>>> >>>> Nope - similarly as we do in v2-2* or what we did in 1.10.* >>>> cherry-picking can be done in separate branches (and in this case in >>>> forks). >>>> Then the branch can be fast-forwarded by the committer in the "airflow" >>>> repo. No problem with that. >>>> >>>> J. >>>> >>>> On Mon, Jun 20, 2022 at 2:53 PM Kamil Breguła <[email protected]> >>>> wrote: >>>> >>>>> Will the people who maintain the providers' packages have the commiter >>>>> status? I guess it is necessary for people to have write access to the >>>>> repository and therefore to be able to make cherry-pick changes to the >>>>> branch. >>>>> >>>>> pon., 20 cze 2022 o 09:13 Elad Kalif <[email protected]> napisał(a): >>>>> > >>>>> > +1 >>>>> > From my side the proposal handles all concerns I raised in previous >>>>> threads. >>>>> > I think mixed-governance is a step in the right direction. >>>>> > >>>>> > On Wed, Jun 15, 2022 at 1:12 AM Jarek Potiuk <[email protected]> >>>>> wrote: >>>>> >> >>>>> >> Hello everyone, >>>>> >> >>>>> >> This is a follow-up after a few discussions started about providers >>>>> that were put on hold around the summit. I held a number of discussions >>>>> during theSummit and after, and as result I think I have a proposal that >>>>> can move forward some of the "stalled" decisions we need to make. >>>>> >> >>>>> >> TL;DR; >>>>> >> >>>>> >> My proposal is a "mixed-governance" model where "stakeholders" are >>>>> more responsible for cherry-picking and testing their providers (including >>>>> system testing) while Airflow PMC members will continue to be responsible >>>>> for releasing them. >>>>> >> >>>>> >> Why do we need that? >>>>> >> >>>>> >> Google, Amazon and possibly others teams who are interested in >>>>> maintaining more backwards compatible versions of their providers will >>>>> commit to make PRs of the cherry-picks for older release branches of their >>>>> providers. Those providers we release in parallel with the latest versions >>>>> during the normal provider cycle. We can deprecate changes more >>>>> aggressively in the "latest" release if we do that. >>>>> >> >>>>> >> Those cherry-picked PRs will be driven, tested and performed by the >>>>> stakeholder teams (Google/Amazon, Databricks, others) and will only >>>>> contain >>>>> cherry-picks, while we - as PMC - will release them following the ASF >>>>> rules >>>>> (this is very important for the ASF to follow strict release policies >>>>> regarding who and how performs releases). >>>>> >> >>>>> >> This also allows us to introduce similar rules for new provider's >>>>> acceptance for new providers for "main releases". It also allows running >>>>> the "system tests" for the provider under control of the stakeholder >>>>> (after >>>>> applying AIP-47 changes). >>>>> >> >>>>> >> Example 1: Google team can cherry-pick changes to a >>>>> google-provider-6 branch and then we release a google 6.8.1 or 6.9.0 >>>>> provider with some of the bug-fixes and features (together with - say >>>>> latest 8.1.0). >>>>> >> >>>>> >> Example 2: DataLake provider from Databricks - can get accepted if >>>>> Databricks commits to maintaining it. We will release the provider as long >>>>> as Databricks maintains it. >>>>> >> >>>>> >> Longer context: >>>>> >> >>>>> >> I have - in my mind so far - a longer roadmap for providers that >>>>> will lead them to be separated from the core and I want to write an AIP >>>>> about that soon. This AIP will detail all the steps needed - I will work >>>>> with multiple interested parties on it and it will take some time to agree >>>>> and complete. But I want to start with something tangible that will solve >>>>> quite a few problems that were raised recently and something that seems to >>>>> be possible to be solved in the current provider release cycle (till the >>>>> end of June) and test some of the governance approach. >>>>> >> >>>>> >> This proposal simply builds on our semver approach - we do not >>>>> change it, we just start releasing some providers (those that have some >>>>> backing from stakeholders) in more than one version - including "latest" >>>>> and earlier, more backwards-compatible branches. Not all providers - just >>>>> some. Not all branches - just those that the stakeholders will commit to >>>>> maintain. >>>>> >> >>>>> >> We need such a commitment from stakeholders, because we - as the >>>>> Airflow community and maintainers, want to only actively maintain the >>>>> latest releases, where it is in the interest of the stakeholders to >>>>> cherry-pick and test also earlier, more backwards compatible releases of >>>>> their choice. >>>>> >> >>>>> >> What problems this proposal solves: >>>>> >> >>>>> >> * Problem 1: DEPRECATION REMOVAL >>>>> >> >>>>> >> we can remove deprecations faster in "main" versions of the >>>>> providers - no need to introduce a deprecation policy - the stakeholders >>>>> for the providers will take care about cherry-picking and maintaining more >>>>> "backwards-compatible" versions. We are free to remove deprecations in >>>>> major releases (in a cherry-pickable way of course). >>>>> >> >>>>> >> * Problem 2: PROVIDERS DIVERGENCE >>>>> >> >>>>> >> We avoid the problem (already happened with the composer release) >>>>> that the stakeholders in a given provider had to release their own version >>>>> which was not available in their community - with some cherry-picks. We >>>>> want to avoid "diverging" there - by releasing the cherry-picked providers >>>>> by the community, we also give other users an opportunity to follow >>>>> "slower" deprecation policies for as long as it is maintained. >>>>> >> >>>>> >> * Problem 3: PROVIDERS GOVERNANCE MODEL >>>>> >> >>>>> >> We are going to test a governance model that we might apply when we >>>>> split providers. We are talking about it for quite some time - but this is >>>>> what helps us to test the model where stakeholders provide more >>>>> "maintenance" while the community still takes care about releases. We (as >>>>> community) can commit to releasing such a version of a provider as long as >>>>> the stakeholder will actively maintain it. We can stop at any moment if we >>>>> do not have support from the stakeholder. If it works - we can keep it as >>>>> a >>>>> long-term solution. In the future we can think of other scenarios (passing >>>>> ownership of a provider to stakeholders who want it - providing we want it >>>>> too) but we can decide about it when we learn from the mixed-governance >>>>> model and see if it works. >>>>> >> >>>>> >> * Problem 4: ACCEPTING NEW PROVIDERS >>>>> >> >>>>> >> If this is an acceptable approach - we can also apply a very >>>>> similar governance model to adding new providers and that should unblock >>>>> some of the PRs that are waiting for our decision. Knowing that we are >>>>> going to split and that we can expect "commitment" from a stakeholder, we >>>>> should be able to accept new providers. This might be possible assuming >>>>> that the stakeholder will make a similar commitment - but for new >>>>> providers, that commitment might also have to cover reviewing and testing >>>>> new changes. We might also decide as a community to stop releasing new >>>>> providers there if such support is missing. This way we can set the >>>>> expectations we have as a community for new providers - we will release >>>>> them as long as the stakeholder will actively make sure it is maintained. >>>>> >> >>>>> >> * Problem 5. SPLITTING PROVIDERS FROM CORE >>>>> >> >>>>> >> We all know we want to split providers from core. By introducing >>>>> mixed-governance we can test if it will work for the providers before we >>>>> split them. It will take some time (and detailed AIP) to split, but in the >>>>> meantime we can see if we will be able to apply the mixed-governance after >>>>> the split. We will see if we can agree when it comes to expectations and >>>>> find solutions before we actually split. >>>>> >> >>>>> >> J. >>>>> >>>>
