Chris & Bolke, do you have a TODO list / wiki detailing the step-by-step process?
Max On Thu, May 18, 2017 at 11:46 AM, Maxime Beauchemin < [email protected]> wrote: > @Andrewm, we can only assume that the author of each commit in master on > top of 1.8.1 wants their commits into 1.8.2. > > ------------------------- > > Ok cool, I'll take this on then, and I'm asking Arthur to see if he wants > to help / oversee the process. > > I'm planning to make 1.8.2 essentially same as 1.8.1 plus the set of > "cherries" that we use at Airbnb in production and every bugfix / minor > feature that looks benign to us. Given that, we're committing to try out RC > along with everyone else. > > What cadence are we aiming at? What should be the target date for the RC? > > Max > > On Thu, May 18, 2017 at 11:29 AM, Bolke de Bruin <[email protected]> > wrote: > >> Hi Max, >> >> Sounds reasonable. For the Release Manager it is really mostly a >> management job. Chasing, prioritising etc. While it is nice to have a rm >> also being able to run the RCs themselves I don’t think it is an absolute >> requirement. Especially, as I think we should trust the community to test >> and then vote. >> >> As mentioned the 1.8.X release series should focus on bug fixes, >> performance issue and minor feature updates (UI fixes, fixes to some >> hooks/operators). 1.9.X is for the larger changes. So indeed please keep >> 1.8.2 simple! >> >> Fully understand that business priorities can take precedence. I (and I >> guess Chris as well) were just hoping that also some of the other >> committers would chime in. >> >> Cheers >> Bolke >> >> >> > On 18 May 2017, at 20:18, Maxime Beauchemin <[email protected]> >> wrote: >> > >> > Hey, >> > >> > Sorry about the delay answering, I wanted to sync up with the Airflow >> team >> > here at Airbnb before I replied here. >> > >> > Quick note to say that the folks at Airbnb are putting a plan together >> as >> > to how we can move towards smooth releases with higher confidence in the >> > future. That plan involves improving the build/test process as well as >> our >> > staging infrastructure, possibly enabling progressive rollouts >> internally. >> > >> > For context, the team that works on Airflow at Airbnb is "Data Platform" >> > and is also on the hook for big chunks of non-Airflow-related >> > infrastructure work that hit us recently and accounts for more than the >> > team's bandwidth at this time. Given that, the team doesn't want to >> commit >> > the time/risk to deploy RCs in production in the short term. Clearly >> > Airflow is still a priority for the team, but on the short term we have >> > critical things prioritized above that. >> > >> > Part of the solution is for us to hire more engineers, and one of the >> open >> > seats is a dedicated role on Airflow tackling things from feature >> building >> > to release management. Hopefully we can widen our bandwidth shortly. >> > >> > In the meantime, I can commit the time to handle a release, but this >> > release won't hit production at Airbnb for a little while, which makes >> me >> > wonder whether it's worth committing the time. Maybe there's a >> > Fedora/RHEL-type scenario here (using a cutting-edge community edition >> to >> > stabilize LTS releases), but we know it's not ideal for Airbnb and for >> the >> > community. The end goal is clearly to have steady, high-confidence, >> mostly >> > automated, regular releases and it feels like time is best spent >> working in >> > that direction. >> > >> > Another option is to make [upcoming] 1.8.2 very simple, as 1.8.1 + the >> few >> > cherries we run in production already at Airbnb, holding the 50+ extra >> > commits in master for 1.8.3. This is marginally useful but helps getting >> > the release mechanics oiled up. >> > >> > I'm trying to be as transparent as I can here, and open to discuss the >> > different ways we can move forward. >> > >> > Max >> > >> > On Sun, May 14, 2017 at 4:44 AM, Bolke de Bruin <[email protected]> >> wrote: >> > >> >> Hi Folks, >> >> >> >> With 1.8.1 we have very much improved the reliability airflow, which is >> >> great as many new features entered 1.8.0 and the gap from 1.7.1 was >> huge. >> >> What is also great is that we are slowly but surely increasing the test >> >> coverage which mitigates some of the risk of regressions going >> forward. As >> >> you know the 1.8.X releases will continue to focus on improved >> reliability, >> >> performance improvements and minor feature updates. The 1.9.X release >> >> cycle, which should start around September, will allow for larger >> feature >> >> updates. >> >> >> >> I expect 1.8.2 not to have too many PRs, so it will be a relatively >> simple >> >> release process: >> >> >> >> 1. Apply bug fixes >> >> 2. Add performance fixes >> >> 3. Fix some outstanding Apache requirements (Author, Licensing etc) >> >> >> >> The process of creating a distribution has been detailed by Chris here: >> >> https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow >> < >> >> https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow> >> >> >> >> Now we just need a volunteer (preferably from the committers) to be the >> >> Release Manager for 1.8.2 :-). >> >> >> >> Who is willing to take this on and make history? >> >> >> >> Regards, >> >> Bolke >> >> >> >> >> >> >> >> >
