On Fri, Jan 18, 2013 at 1:05 AM, Oswald Buddenhagen <[email protected]> wrote: > as some certainly noted, the completeness of dist/changes-* severely > deteriorated over the last few minor releases, and in particular 5.0.0 > is one of the poorest qt changelogs seen in a while. > > also, sergio is now attempting to make 5.0.1 changelogs in a heroic > one-man-effort - not entirely surprisingly, this turns out to be a tad > more tedious than planned.
I was quietly wondering to myself how long it would take for someone to notice that I stopped acting as editor for the changelogs when I left Nokia. As Sergio has no doubt discovered, manually generating the changes file is a tedious and time-consuming process, particularly when the level of co-operation from developers is low to non-existent. Back in the Trolltech and Nokia days, I made a few (largely unsuccessful) attempts to get people to describe their important changes in the changes-file, but mostly I only succeeded in getting myself flamed (and maybe this email will have the same effect), to the point where one senior developer tried to wash his hands of this particular duty by stating on an open mailing list that "the release branch is the Release Manager's hostile fork of Qt for which the developers have no responsibility". While that gave the folks in Qt's former QA team a good laugh, I would hope that attitudes have shifted a little since then. During my tenure as Qt's Release Manager, I came to the following conclusions about the changes file: (a) There is no feasible way to automatically produce a high-quality changes file, (b) The changes file is an important resource for Qt users, (c) Jira is not an acceptable substitute for the changes file, and (d) The git commit history is not an acceptable substitute for the changes file. Therefore, I basically gave up chasing developers to update the changes file and did it myself by examining both the bugtracker's list of closed tasks for the release and the git log, making decisions about what items were worthy of a mention in the changes file, what kind of change they were (feature, fix, doc, approved BC/SC break, etc) and trying to turn the task and commit descriptions into a brief, high-level, human-readable description of what changed. This could take anything from one to five days work for each release depending on how many commits were in the release. The quality of the changes file for each release was largely a factor of whether I could actually spare the time to do it properly in addition to all the other things a release manager has to do. In my opinion, as Qt's most experienced past release manager, this should have gotten a lot better with modularization, as the ultimate responsibility for documenting the important changes for each module is now very clear -- it belongs to the module maintainer. Now, let me explain my reasons for the conclusions above: (a) The changes file can't be automatically generated, partly because the Qt Project's development processes are too flexible (i.e. inconsistent, poorly-defined and, in some cases, highly resistant to change) and partly because writing the changelog is a complex decision-making process for which suitable AI will probably not exist within our lifetimes. You need to decide what's important to end-users and what's not. You need to filter out stuff that user's never saw in an official release (i.e. stuff that was broken and fixed again between releases, bugs fixed before a new feature was first released, fixes that got reverted, patches that were effectively empty because of later changes and/or resolution of merge conflicts). You need to figure out when multiple commits equal one fix from the user's viewpoint. You need to figure out when one commit equals multiple fixes from the user's viewpoint. You need to filter out duplicate bug-tracker tasks. You need to filter out changes to code that was later completely removed or rewritten. You need to figure out where in the changes file a particular change belongs. You need to turn one or more commit descriptions into a user-understandable description. Sometimes you even need to read diffs to figure out what really changed. That said, I believe that it is possible to create a tool to assist/manage the process of manually writing the changes file. The old changelog tool we had in Trolltech basically showed you a list of the commits in a release and allowed you to tick off the ones you had already dealt with, so that you knew which ones you still had to make decisions about. Unfortunately, the tool was written for the source-control system we used before moving to git and fell into disuse thereafter because writing the changes file became less the developers' problem and more my problem. (b) End users do care about the changes file. I know this because I always received emails and IRC messages each time Nokia's web team forgot to post the changes file on the web when a new release went out. The changes file was the first place many customers looked when they were trying to decide whether there was any value in them upgrading to a new version. Many Qt users, and particularly commercial customers, don't automatically upgrade every time a new release is made, because that carries a significant cost and a significant risk of their code being broken by regressions in Qt (CI doesn't fix that -- last time I looked our test coverage was barely 50% in most cases). Those people look at what's changed and only attempt to upgrade if a new release gives them features or fixes that they actually need. (c) We can't drop the changes file and tell people to look at Jira. There are many commits that have no corresponding Jira task. Even if there was a task for every commit, the Jira tasks usually describe symptoms, not the root cause or what was changed as a result of diagnosing those symptoms. There is also considerable reluctance among developers to modify Jira task summaries and descriptions and to delete irrelevant comments. I don't agree with that reluctance, but I also don't think I'm ever going to change it. (d) We can't drop the changes file and tell people to look in the git history. For one thing, the changes file has a very different audience to the git history. Many of the people who pinged me when the changes file wasn't published weren't Qt developers. Some were developers using Qt to write their own apps. Others were managers and project-leads who were trying to decide whether taking a new release was worth the potential disruption to their projects. Those aren't people who rarely have the desire to grovel through the thousands of commit messages in each release, nor the detailed knowledge of Qt internals to draw conclusions about whether those commit messages means that the release includes changes that their project needs. So, based on my bitter experience, here are my recommendations: 1. Let's acknowledge that writing the changes file is going to have to be a manual process due to the complexity of the decision-making and the lack of a strict development process. 2. Let's agree that module maintainers should be responsible for their module's changes file, and clearly document that responsibility on the part of the wiki that lists a maintainer's duties. 3. Let's call for volunteers to prototype a new changelog tool along similar lines to the old one, but this time with a better UI, the ability to share data between users via a git repo, and some kind of checklist to assist the decision-making process. I'm happy to help with setting requirements, but I regret that I won't be able to do much more than that as my new full-time job does not involve Qt. 4. Let's try to make the job of our maintainers that little bit easier by writing good commit summaries and diligently reviewing the commit summaries of our peers. Ok, I'm wearing my asbestos jumpsuit, so you may now flame me at will..... -- Jason _______________________________________________ Development mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/development
