Hi Ricardo, I find interesting some ideas in the proposed changes, especially I like this: "uniform substring like “unstable”" (with startup-time warnings if they change or become stable) and "Maintenance releases should be no-brainers to install... This means that once you’re on 3.4.0, you can always upgrade within the 3.4 series with a minimum risk. It also means you get no optimizations, features, and the like.", though I wouldn't limit the fixes to only the critical ones. Any bugfix that doesn't change the behavior is included in the stable maintenance releases.
But I couldn't understand from the description what are the benefits of tying major releases to certain calendar dates vs to make a release when certain desired features are implemented and well tested. What happens if some major new feature, that is a must for a new major release to be published in a week, just isn't stable enough yet? Would it have to wait for an entire year to be included in the next major release? Or would you release it anyway as a stable release with known issues? Then, when you implement a new large feature, who would test it? Today, for example, I (as an advanced user and a potential community dev) can run 3.1 branch at some semi-production deployments (and I sometimes do) and report issues. If, with the new scheme, you only guarantee that the unstable branch just compiles, certainly I wouldn't be using it anywhere, and probably neither would other users. Then pre-production testing of new features would be exclusively the developers' task, with obvious limitations. So when the devs are sure that a new feature works well (in their setups and for their use cases), it is included in the next major stable release... and suddenly a lot of migrating users start finding issues. That could create an impression that the new stable releases of Cyrus are not that stable at all. > As feature X is delayed (for various reasons that we can’t easily > eliminate), it doesn’t just delay the feature, but also all the other > minor bugfixes and optimizations that we’ve made in the master branch. Why would a new feature of a stable release (3.2.0) delay bugfixes in the current stable branch (3.0)? > Also, it sets up the idea that we delay releases for the sake of > fixes, instead of releasing the fixes that are ready. I don't understand what you mean here, but with the current scheme (AFAIK) the bug fixes go to the current stable branch (3.0) and all users receive them without delays. New development happens in dev. When some new feature is stable according to the devs (well tested in all environments available to them), it is published as a new minor release in unstable branch (3.1.x). This is expected to be fully-functional releases, just not proven by the time and the community to be bugs-free. I'm not sure how new major releases are managed today, but it could be done this way: at some point in time, when devs decide that the unstable/3.1 branch has accumulated enough features to be published, 3.1 is frozen for new features and it becomes 3.2.0-RC1 so the community in general could start testing the new candidate stable version in their test deployments. If issues are found, they are fixed in RC2, RC3, and so on until no issues are reported for, say, 1 month. Then, the last issues-free RC becomes 3.2.0 release. At the same time, when 3.1 is frozen for new features, a 3.3 branch is created and new features start landing there. And the current stable branch 3.0 receives bug fixes as usual during all this time. New optimizations probably won't be included in the 3.0.x maintenance versions, but that's OK IMO. It's stable, not cutting-edge after all. But it is bugs-free to the extent possible. All bugs, major and minor (without behavior changes), are fixed there immediately. This is a typical release cycle of many server projects. The main advantage over date-bound releases is that the releases are published when they are ready, not when we reach some specific point in time. The disadvantage of the potential for delays could be mitigated by defining certain criteria for the features to be included in each major release. Also, some flexible dates could be defined, e.g. to publish a major release every 6-12 months. Regards, Anatoli On 13/12/19 11:59, Ricardo Signes wrote: > Hey, remember last month when I asked about releasing Cyrus v3.2 > <https://lists.andrew.cmu.edu/pipermail/cyrus-devel/2019-November/004509.html>? > > That thread had some more conversation about what needs to get done > before v3.2, and I wanted to come back to it and turn some things on > their head. > > Right now, we’re talking about Cyrus releases being feature-bound. > “We’ll release v3.2 when feature X is done.” I think we’re not being > well-served by that. As feature X is delayed (for various reasons that > we can’t easily eliminate), it doesn’t just delay the feature, but also > all the other minor bugfixes and optimizations that we’ve made in the > master branch. Also, it sets up the idea that we delay releases for the > sake of fixes, instead of releasing the fixes that are ready. > > That is: every additional criteria for a new release is another doorway > to delay. Instead of opening those doors, I would rather try to > eliminate all of them. > > I propose that instead of tying releases to milestones, we tie them to > the calendar. For the sake of full disclosure: I am modeling this > suggestion on the release cycle of perl > <https://metacpan.org/pod/perlpolicy>, which I ran for several years. I > found the process more than satisfactory, then. > > 1. > > A new /unstable release/ of Cyrus is made every month. We promise > only that it compiled and passed the Cassandane test suite on the > release manager’s computer. It might contain regressions from > previous unstable releases, it might have crashers or corruptors. We > try to avoid any of these, but the goal here is a snapshot for easy > month-to-month testing. These are the odd-middle-digit releases. (3.3.x) > > 2. > > A new /major release/ of Cyrus is made every year. We will have > tested it on as many configurations as we can readily test. We will > have, some time before the release, frozen the branch for risky > changes, to reduce churn. In the meantime, new work lives in feature > branches. (The changelogs from each unstable release provide a good > basis for the whole-year changelog!) These are the even-middle-digit > third-digit-zero releases. (3.4.0) > > 3. > > A new /maintenance release/ of Cyrus is made for the last two stable > releases when there are enough fixes to critical bugs to warrant it. > These are the even-middle-digit third-digit-nonzero releases (3.4.1) > > For the above to work, some more properties need to be maintained. > > Maintenance releases should be no-brainers to install, so they must only > fix regressions, crashers, security vulnerabilities, and the like. This > means that once you’re on 3.4.0, you can always upgrade within the 3.4 > series with a minimum risk. It also means you get no optimizations, > features, and the like. > > Major releases must clearly document any incompatible changes or upgrade > steps required. Because non-regression bugfixes aren’t backported, we > want everyone to be able to upgrade from major release to major release, > so incompatible changes must be kept to a minimum. > > In part, this is just “don’t kill off a feature people use just because > it’s a little annoying.” The more important one is “don’t introduce > half-baked things that might need to change,” because people will come > to rely on them before you get the updates finished. For features that > will require multiple years to get right, they have to go behind a > default-off configuration option. I’d strongly suggest they all have a > uniform substring like “unstable”. That way, when a complaint comes in > that the behavior of JMAP calendaring has changed, we can reply, “well, > to use it, you had to turn on the unstable_jmap_calendaring” option. > > If we go with this policy, we’ll need to… > > 1. > > identify what issues are /blockers/ to v3.2.0, meaning they’re > regressions from v3.0 and would reasonably prevent someone from > upgrading; this does /not/ include all known bugs, since they may be > bugs that already exist in the last stable release! > > 2. > > pick a release target for v3.2.0; I will arbitrarily suggest March 2 > as “not too far off, but far off enough that we can get things in > order”; also, if you’re American, March 2 is 3/2 ;-) > > 3. > > produce a changleog, and especially identify what changes in master > need documentation as “incompatible changes” > > 4. > > produce a list of changes in master that should be put behind an > unstable configuration option and then do it > > 5. > > decide when to stop merging non-release-related things to master > > 6. > > make a plan for who will do monthly snapshot releases > > I’ve spoken with ellie and Bron about just a few of these, such that I > don’t think it’s all crazy. (ellie notes, correctly, I think, that the > first set of releases like this will be the hard ones, where we work out > things like “how do we keep track of incompatibilities, upgrade steps, > and also how do we make snapshots dead easy to release.”) If there’s > general agreement, I am definitely ready to pitch in and help try to > make it work! > > — > rjbs > >
