Hey, Sorry about the delay answering, I wanted to sync up with the Airflow team here at Airbnb before I replied here.
Quick note to say that the folks at Airbnb are putting a plan together as to how we can move towards smooth releases with higher confidence in the future. That plan involves improving the build/test process as well as our staging infrastructure, possibly enabling progressive rollouts internally. For context, the team that works on Airflow at Airbnb is "Data Platform" and is also on the hook for big chunks of non-Airflow-related infrastructure work that hit us recently and accounts for more than the team's bandwidth at this time. Given that, the team doesn't want to commit the time/risk to deploy RCs in production in the short term. Clearly Airflow is still a priority for the team, but on the short term we have critical things prioritized above that. Part of the solution is for us to hire more engineers, and one of the open seats is a dedicated role on Airflow tackling things from feature building to release management. Hopefully we can widen our bandwidth shortly. In the meantime, I can commit the time to handle a release, but this release won't hit production at Airbnb for a little while, which makes me wonder whether it's worth committing the time. Maybe there's a Fedora/RHEL-type scenario here (using a cutting-edge community edition to stabilize LTS releases), but we know it's not ideal for Airbnb and for the community. The end goal is clearly to have steady, high-confidence, mostly automated, regular releases and it feels like time is best spent working in that direction. Another option is to make [upcoming] 1.8.2 very simple, as 1.8.1 + the few cherries we run in production already at Airbnb, holding the 50+ extra commits in master for 1.8.3. This is marginally useful but helps getting the release mechanics oiled up. I'm trying to be as transparent as I can here, and open to discuss the different ways we can move forward. Max On Sun, May 14, 2017 at 4:44 AM, Bolke de Bruin <[email protected]> wrote: > Hi Folks, > > With 1.8.1 we have very much improved the reliability airflow, which is > great as many new features entered 1.8.0 and the gap from 1.7.1 was huge. > What is also great is that we are slowly but surely increasing the test > coverage which mitigates some of the risk of regressions going forward. As > you know the 1.8.X releases will continue to focus on improved reliability, > performance improvements and minor feature updates. The 1.9.X release > cycle, which should start around September, will allow for larger feature > updates. > > I expect 1.8.2 not to have too many PRs, so it will be a relatively simple > release process: > > 1. Apply bug fixes > 2. Add performance fixes > 3. Fix some outstanding Apache requirements (Author, Licensing etc) > > The process of creating a distribution has been detailed by Chris here: > https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow < > https://cwiki.apache.org/confluence/display/AIRFLOW/Releasing+Airflow> > > Now we just need a volunteer (preferably from the committers) to be the > Release Manager for 1.8.2 :-). > > Who is willing to take this on and make history? > > Regards, > Bolke > > >
