On Wed, Feb 25, 2015 at 10:31:58AM +0100, Thierry Carrez wrote: > Robert Collins wrote: > >> It's also worth noting that we were on a 3-month cycle at the start of > >> OpenStack. That was dropped after a cataclysmic release that managed the > >> feat of (a) not having anything significant done, and (b) have out of > >> date documentation and translations. > > > > Oh! > > https://wiki.openstack.org/w/index.php?title=Obsolete:3MonthReleaseCycle&direction=prev&oldid=14016 > > appears to be the page about that, but my google-fu isn't turning up a > > post-mortem of the C release which prompted the change. Perhaps some > > old-hand like you can fill us in on the details :). > > After the Austin release (a code drop, really), we were on a three-month > time-based cycle with no intermediary milestone. The first cycle (Bexar) > worked relatively well, the second one (Cactus) not so well. There were > multiple reasons for that: > > - Software was growing up in Bexar and we introduced more testing, but > we were unable to detect and fix all the release blockers in time for > the release date, resulting in a broken Bexar release. To address that > in Cactus, we increased the duration of the frozen period, but then that > didn't leave enough time for development, hence nothing getting done.
I wasn't involved in OpenStack back then, but is it not true that OpenStack has changed almost beyond recognition since those early days. The number of projects has certainly increased massively, as has the number of contributors, both individual & corporate. There is a hell of alot of testing now and people doing continuous deployment of trunk, so I find it hard to believe that trunk is so unstable that we can't do a release at any time we choose, nor that we have to have such long freezes that we don't do get time for dev. > - Not starting a release cycle with a global F2F gathering (no > intermediary Design Summit) actually proved quite disastrous: no reset > of community, no alignment on goal, no discussion on what to work on. > That resulted in a limbo period after the Bexar release. That is why I'm > so insistent on aligning our development cycles with our Design Summits. > I've seen where we go when we don't. I don't know about other projects so much, but I don't really see the design summit as a positive thing wrt planning the next release. For a start the design summit is atbout 5 weeks after the trunk opens for development, so if people wait for the summit do do planning they have thrown away half of the first milestones' development time. It is also not inclusive as a decision making forum because we cannot assume every one is able to make it to the summit in person, and even if they are present, people often can't get involved in all sessions they wish to due to conflicting schedules. If release planning were done via email primarily, with IRC for cases where realtime planning is needed, it would be more effective IMHO. IOW i think the summit would be better off if were explicitly not associated with releases, but rather be a general forum for collaboration were we talk through ideas or do code sprints, and more. > > [...] > >> I may appear defensive on my answers, but it's not my goal to defend the > >> current system: it's just that most of those proposals generally ignore > >> the diversity of the needs of the teams that make OpenStack possible, to > >> focus on a particular set of contributors' woes. I'm trying to bring > >> that wider perspective in -- the current system is a trade-off and the > >> result of years of evolution, not an arbitrary historic choice that we > >> can just change at will. > > > > I agree that its not arbitrary and that changing it requires some > > appropriate wide-spread consultation; OTOH the benefits of rolling and > > higher frequency releases are really substantial: but we have to > > backup the change with the appropriate engineering (such as reducing > > our frictions that cause teams practicing CD to be weeks behind tip) > > to make it feasible. My greatest concern about the proposals happening > > now is that we may bite off more than we can chew. OTGH the reality is > > that all the negative things multiply out, so we probably need to just > > start somewhere and /do/ to give us the space to fix other things. > > As I said, I'd very much like us to "release" a lot more often. I just > don't see how we can do it without: > > - abandoning the idea of translating the software > - admit that docs will always lag code > - dropping stable branch maintenance (and only fix vulnerabilities in > master) I don't really buy that. If we have 6 month cycles with 1 month freeze for docs & i18n, vs 2 months cycles with 2 weeks freeze for docs & i18n the latter is a win in terms of dev time vs stablization time, giving docs & i18n 50% more time in aggregate. > That would all make my life (and the life of most feature-focused > developers) a *lot* easier. I'm not convinced that's the best solution > for our downstream users, though. I think they like docs, > translations(?), stable branches and backported security fixes. > > PS: as a data point, at the last summit, in the stable branch session, > there was a whole group of downstream users crashing the session to ask > for releasing *less* often ("we can't keep up"). I think that's crazy > too, but that shows 6 months is really a trade-off. I'd be wary of taking that those users literally. IME working on RHEL everyone wants very infrequent unchanging releases, except for their own favourite new features to be added. So you end up with infrequent releases but with major feature backports to those so called "stable" branches. This diverts massive amounts of developer resource away from doing net new upstream work, while at the same time destablizing the stable branches. I already see this happening with OpenStack due to the severe time delay in getting access to features over the 6 month dev cycle. BTW, when I talk about stable branches in this context, I'm more referring to the vendor's OpenStack products, rather than upstream stable branches. Upstream stable branches dont accept any features at all, which leaves users who want plain upstream OpenStack stuck waiting for 6 months. This actually pushes users towards vendors who are willing to provide & support the features they want on a more reasonable time scale. This is great for us vendors but IMHO is really a reflection on the upstream release model By doing more frequent releases you would reduce the pressure to backport features to stable branches, and reduce pressure to jam in lots of major features at the last minute in master which risks destablizing trunk too which is bad. I think the key take away is that you must not force users to upgrade to new releases in an unreasonable timeframe - you need to give them the ability to have a reasonably long deployment life when they pick a release, and allow them to skip releases if they want to upgrade. Since we only official supporting N-1 to N upgrades, we are effectively forcing people to try to deploy every 6 months. If they want to skip a cycle and stay deloyed for 12 months, then we're causing them pain by saying they have to upgrade to this intermediate release first, before going to the one they actually want to get to. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev