Dear devs, a long post with my thoughts. Each time we discuss a new release, we return to the problem of lack of resources and our cumbersome release process. Here are my thoughts.
One of the main questions is how much more we could automate the procedure, given limited resources to do so. Another question we ask ourselves from time to time is whether we could release some fixes more quickly with less effort by releasing them as a patch (that is, a diff) instead of the full sets of source code (tarballs). == Streamlining Patch releases == We would like to streamline our release processes. What can we do given limited resources? Concentrate on streamlining patch releases. We should also be able to streamline the minor (1.x) releases, but I would concentrate on patch releases for now because they are more frequent and simpler, so more benefit can be gained for each one and in total. We would expect some of those benefits to trickle down to the less frequent and more manually intensive "minor" releases. - For "minor" releases (1.x) we traditionally put in significant effort such as manually curating the changelist and release notes, reviewing API changes, and so on. This makes sense when a number of relatively major and minor changes are bundled up together. OK. - Patch releases (1.x.y) are different: it should be a simple matter of "here's another little fix" every single time. I see no reason why we should: - keep on bundling multiple changes together, if we can streamline the process so that releasing one change is easy. - be doing significant manual work for any one change/fix/patch release, beyond that which we already require to propose and accept a fix into a back-port branch. One change, one patch release: - Releasing little and often, for maintenance releases, is generally accepted good practice. For us one of the biggest advantages is then the releases will no longer be a special extra burden that we have to discuss and plan and resource, as they become a small, easy, routine step in everyone's work. - Requires making our patch release process much easier, so that there is little or no extra work involved that could be combined more efficiently by bundling multiple changes in one release. - We should apply the same amount of manual review and testing to each patch, but instead of delaying it until the RM says it's time to review, approve and release many months' changes, we would more easily do it close to the time of writing the patch. - We could surely we can make it so the very same set of review and test effort serves to authorise both the backport merge of the patch and the consequent release, combining what are currently two separate review-and-test efforts. Yes that means the per-release effort would no longer be spread across a number of patches. The proposition, and dependency, is that that per-release effort should be almost entirely automated and so would cost little. - How would the downstream recipients be affected if we make faster smaller releases? For minor (1.x) releases with API bumps we have been made aware that at least in some cases (I'm thinking of ViewVCS) these generate very considerable effort downstream. For patch releases, as far as I know, that would not be the case at all. Manual work: We already have release tooling for most of the patch release process. For example, that can place a new entry in the changelist; we should just need a simple review that it looks ok and optionally minor editing for formatting and wording. It's still some work but it's something anyone can do in a few minutes rather than having to set aside hours. Automation: I feel we could improve our automation without excessive effort. We are living a decade or two behind the times in terms of build-test-release pipeline automation. Every time a change is approved on a release branch we should have an automated tarball produced, with version bump and accompanying external bits like release announcements drafted, ready for a release manager to press the "Ship it!" button if signatures have been collected properly and they agree it's ready. It's much easier nowadays to take advantage of containers or VMs, and build pipelines such as GitLab's. We might also find some relevance in binary build/distribution systems such as Open Build Service. Yes this is just talk, to float the idea and see if anyone is inspired to try to actually do this. I'm quite interested myself but I don't know if I'll spare the time to do so. I have some little bits of automation scripts left over from previous RM work, that I haven't turned into contributions, but not to the extent I'm now talking about. == Releasing just the patch instead of full tarballs == We've said a few times in the past, we could perhaps save work and get a small patch release out sooner if we just released the patch instead of the full tarballs. As far as I know, that would be both allowable and reasonable. When we have discussed this before, my recollection is people have always said it would be fine. We need to stop making more work for ourselves than absolutely necessary. Where possible, do what is simplest. If that's simplest, and we decide it will be worthwhile, then we should agree the minimal steps, here and now or in a similar thread, and start doing it. Let's see what this might look like in more detail. There are two styles of patch we could release: 1. A stand-alone patch that just fixes the particular issue. - The recipient (an end user, IT department, or a downstream packager) can choose to apply the patch if it seems relevant to them. - The recipient should manage their own version numbering to indicate patched versions. 2. A diff between one specific version and the next in our release versioning scheme that we control, equivalent to a mechanical diff between tarball releases. - We apply our own version numbering scheme, just the same as if we were releasing complete tarballs for each release. - The diff includes a version bump, and successive patches apply in sequence to generate successive release versions of Subversion. - This need not imply we also release the complete code of each version as tarballs. - We retain the option to release the tarballs as well as or instead of the patch, if and when we wish, at the same time or any later (or earlier) time. It would be good to include in any such patch: - which versions we assert it does / might / does not apply to; - a changelist addition; - an addition to a list of "patches included" if we were to have that (but we currently don't). But would it be less work? Mustn't we still produce full tarballs in order to test the result, or would we require that only for a patch that involves build system changes? And doing so would only be a small automated step away from having a patch. And there would be discussions required and the procedural set-up to start doing this. So I'm not sure it would be significantly less work; in fact it might be more. Can anyone point out significant work that would be saved by doing this? Is it work that would be saved in general, every time, or is it just because the tarball-generation steps of the process are onerous because they are not yet sufficiently automated? It mainly feels to me that what we need to concentrate on is automation and simplification, up to and including a standard pracice of making a patch release for every bug fix. Your thoughts? - Julian
