Cool. I am looking forward to see how it unfolds :).
On Wed, Apr 20, 2022 at 5:33 PM Jed Cunningham <[email protected]> wrote: > > Cool, thanks Jarek. Yes, I am prepared and willing to deal with issues that > come up from this until we can harden the process a bit. > > I've just merged the PR as having the combined release notes for 2.3.0 will > be helpful for our users. I'll hop on the automation/requirements path once > we get that release out. > > On Wed, Apr 13, 2022 at 10:46 AM Jarek Potiuk <[email protected]> wrote: >> >> Sure :) if you are willing to sync and reconcile it at release tiem - >> I am all for it :). Happy to help along the process too as needed and >> try out and improve automation as we go (hopefully it means not too >> much of a chaos). >> >> I am likely more on being perfectionist (which has good and bad sides) >> and I am just usually erring on the side of prevention of some >> foreseeable problems rather than dealing with them when they are >> overwhelming. >> But those are the two extremes and usually we land somewhere in between. >> >> As long as we are aware of the potential issues and we take a >> conscious decision that we want to (and will have capacity) to deal >> with them when they start - this is cool. >> >> J. >> >> On Wed, Apr 13, 2022 at 6:18 PM Jed Cunningham >> <[email protected]> wrote: >> > >> > A few snippets from Jarek I'd like to comment on: >> > >> > So if we have some way of explaining what to do and we think we can >> > explain both committers and contributors what to do and we are ready >> > to deal with the aftermath of those being misunderstood/forgotten/not >> > followed, it's fine to skip automation initially (and it makes sense >> > if we do not know what to do exactly and want to learn and we >> > implement "little" or "no" automation). >> > >> > I think this is the case - I'm not exactly sure what automation we really >> > need and the best way to figure that out is to start using it beyond a >> > POC. I have some good ideas kicking around in my head for making this all >> > less of a burden once we start requiring newsfragments, but I'd rather not >> > big bang it. I think it's more likely we cause hiccups or pain for the >> > wider committer group (instead of just the release managers like our >> > current system). Either way, we have to deal with "old" label style and >> > "new" newsfragment style (or do a reconcile/true up), and dealing with >> > both for a bit really doesn't change much on the release manager side of >> > things. That's why I lean this way. >> > >> > I am afraid if we just >> > introduce it "softly" we will not even have a chance to learn what >> > burden it will introduce and whether it is "bearable". >> > >> > I think there is a balance here. If this all works out, there will be more >> > work required to merge a PR, however the smoother we make it when we start >> > requiring it the better. I'd rather come to the committers and say, "Hey, >> > these are now required, but it hopefully won't be too bad because you can >> > do X and Y with bots to help with the tedium of it". Said another way, I >> > don't want to launch with 5/10 pain on all committers then dial it back to >> > 2/10 after a bit, I'd rather try and shoot for 2/10 initially even if it >> > prolongs extra legworks from release managers. I want the "bearability" >> > test to start when we put our best foot forward. >> > >> > And we can always revert it if we find it too "problematic". >> > >> > Absolutely! I hope it won't come to that, but really the bulk of this >> > change is changelog + updating -> release notes. The towncrier piece >> > starts really mattering to non release managers when we put requirements >> > in place. Until that point, it's basically prep work, and as I said above >> > we will have to handle both "old" and "new" for some period of time >> > regardless. >> > >> > Outside of the significant newsfragments I already converted, I expect the >> > next release to otherwise be essentially "newsfragment free" with the bulk >> > coming from type labels. I expect that trend to continue until we put >> > requirements in place, and doing it this way doesn't add much burden to >> > release managers. >> > >> > On Mon, Mar 21, 2022 at 4:25 AM Jarek Potiuk <[email protected]> wrote: >> >> >> >> And BTW - we don't even have to implement the "preparation of the >> >> release" for any of those initially. We can just "gather" the news >> >> for some time (few weeks) and add release preparation from those when >> >> we prepare the release. So we really need to automate just news >> >> "entry" as the first step IMHO (following the POC that already told us >> >> that we will be able to use it). >> >> >> >> On Mon, Mar 21, 2022 at 11:18 AM Jarek Potiuk <[email protected]> wrote: >> >> > >> >> > And to add to it - setting automation for those is "easy". We already >> >> > have a selective check that determines the "type of change" (it's >> >> > being rewritten in python to make it easier). We could easily update >> >> > it to asses automatically that things are "trivial" (we actually >> >> > already do that - we have a number of PRs when there is no need to >> >> > build images at all (those are by definition trivial) and we can also >> >> > immediately see if there is any change to any python files in Airflow >> >> > code tree. We could even automatically set the "trivial" label for >> >> > those PRs. Same with changes that only touch the "dev" folder where we >> >> > only do some house-keeping of the dev environment. >> >> > >> >> > All the other changes IMHO should have a "news" piece (maybe empty >> >> > "trivial" news that will be skipped - but it should be there). >> >> > >> >> > There is of course question about splitting the news between providers >> >> > and core - but this should also be easy - we could require NEWS piece >> >> > to be created for "airflow" if there is a change in "airflow" and >> >> > another NEWS piece in any of the "providers" if there is a change for >> >> > that provider - that's all very easy to automate (happy to do so). >> >> > Similarly another news piece for "chart". I can very easily imagine >> >> > that the "NEWS" pieces will be actually different for each of those - >> >> > there might be change concerning both "airflow core" and "provider", >> >> > but you might want to emphasise different things in those two >> >> > different news. >> >> > >> >> > I think starting to require the news by automation is really the only >> >> > way we can introduce it quickly and "firmly". I am afraid if we just >> >> > introduce it "softly" we will not even have a chance to learn what >> >> > burden it will introduce and whether it is "bearable". And we can >> >> > always revert it if we find it too "problematic". >> >> > >> >> > J. >> >> > >> >> > >> >> > On Fri, Mar 18, 2022 at 6:35 PM Leah Cole <[email protected]> >> >> > wrote: >> >> > > >> >> > > @Kaxil Naik thank you for those examples! I've looked at all of those >> >> > > before so it's cool to know what's behind the scenes now :) >> >> > > >> >> > > I definitely echo Jarek's concerns about enforcement and the need for >> >> > > automation - I think that having automation helps ensure consistency >> >> > > and it reduces the mental load on release managers and committers to >> >> > > remember to add it. For folks who are like me and work in repos in >> >> > > different spaces with different policies daily, this is crucial. I >> >> > > agree that CICD can be used as communication. Would an alternative >> >> > > instead of having it be optional be doing something like a trial >> >> > > period, where we set a date to evaluate how it's going? >> >> > > >> >> > > Leah >> >> > > >> >> > > On Sun, Mar 13, 2022 at 4:48 PM Jarek Potiuk <[email protected]> wrote: >> >> > >> >> >> > >> I am just afraid that "optional" without stating what optionality >> >> > >> means might lead to chaos (when is it optional ? how do we check it?) >> >> > >> >> >> > >> No need for automation as long as we "know" what to do and >> >> > >> communicate >> >> > >> to both committers and contributors (and we are able to deal with the >> >> > >> aftermath of those not being followed).. >> >> > >> >> >> > >> I see CI automation mostly as a "communication" tool. What it does - >> >> > >> it passes our intentions of what we want in an automated way - it >> >> > >> should run the same checks that "committers" would do, and >> >> > >> "communicate" what should be fixed to the "contributors' '. >> >> > >> >> >> > >> I see implementing/not implementing automation in CI as a trade-off >> >> > >> between: >> >> > >> >> >> > >> * automating intentions of what we want to do (and teach people to >> >> > >> react to errors) >> >> > >> >> >> > >> vs. >> >> > >> >> >> > >> * explaining and teaching all committers our intention (and at least >> >> > >> attempting to make sure they read and understood it) >> >> > >> * handling the aftermath for all cases where they did not >> >> > >> understand/remember/knew/wanted to follow what to do (both committers >> >> > >> and contributors) >> >> > >> >> >> > >> So if we have some way of explaining what to do and we think we can >> >> > >> explain both committers and contributors what to do and we are ready >> >> > >> to deal with the aftermath of those being misunderstood/forgotten/not >> >> > >> followed, it's fine to skip automation initially (and it makes sense >> >> > >> if we do not know what to do exactly and want to learn and we >> >> > >> implement "little" or "no" automation). But the risk is that it won't >> >> > >> be understood/followed/we won't learn from it and the aftermath will >> >> > >> be difficult to handle. >> >> > >> >> >> > >> J. >> >> > >> >> >> > >> On Wed, Mar 9, 2022 at 10:56 PM Jed Cunningham >> >> > >> <[email protected]> wrote: >> >> > >> > >> >> > >> > Good points Jarek. I mentioned it in the initial email, but I >> >> > >> > think we should keep it optional to start with. I'd rather get the >> >> > >> > basics in place first, as I think we are going to find some >> >> > >> > interesting scenarios as we try and put rules around it. Even if >> >> > >> > only release managers touch it in the short term, we still benefit >> >> > >> > from it (or at least I will). >> >> > >> > >> >> > >> > I have every intention of making it required down the road. I'm >> >> > >> > eager to help spread the changelog workload around a bit more :) >> >> > >> > >> >> > >> > Just a single example of where things can get interesting is due >> >> > >> > to our monorepo: setup.py changes could require a core >> >> > >> > newfragment, a provider newsfragment (when we get there), or both. >> >> > >> > I'm sure we can sort it all out, but I don't think we need to wait >> >> > >> > and have it all on day 1. We will end up needing to handle 2 >> >> > >> > sources of changelog entries in the short term regardless (and is >> >> > >> > part of my todo list). My gut tells me we should do this >> >> > >> > incrementally instead. As long as we "catch up", when we start >> >> > >> > requiring it shouldn't matter (the 2.3.0 fork is a natural time, >> >> > >> > but we don't have to hit it dead on). In fact, I'll likely start >> >> > >> > putting requirements on the helm chart first, as it is more >> >> > >> > isolated. >> >> > > >> >> > > >> >> > > >> >> > > -- >> >> > > >> >> > > Leah Cole (she/her) | Developer Programs Engineer, Data Analytics | >> >> > > [email protected] | +1 (925) 257-2112 >> >> > > My working hours may not be your working hours. Please ping me >> >> > > anytime if you'd like a status update on anything we are working on >> >> > > together - my goal is to never be a blocker for you. >> >> > >
