Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
> On 3 Nov 2020, at 10:52, Richard Gustavsen wrote: > > Would it be better if we required all commits to have a ChangeLog entry, so > that no one > forgets? And if your commit shouldn't be in the log, you need to state that > explicitly, e.g: [NoChangeLog] No, that would just add noice to the git log. The commit message template should already have a reminder to add a changelog entry _if applicable_. Tor Arne ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
> But I didn’t like it when we generate a point-release changelog that says > there are only minor changes, when I know for a fact that some significant > bugs got fixed, and should be mentioned, even though the author forgot to > add a ChangeLog to the git commit. So a few years ago, before each release > I was actually scanning the git log manually to look for such bug fixes > that ought to be mentioned. I got tired of that because it always requires > several windows at the same time: a terminal to view the git log, an editor > for the changes file, and a browser with a couple of tabs open to look up > bugs and code changes. > Yep, same here. Would it be better if we required all commits to have a ChangeLog entry, so that no one forgets? And if your commit shouldn't be in the log, you need to state that explicitly, e.g: [NoChangeLog] -Richard Fra: Development på vegne av Thiago Macieira Sendt: mandag 2. november 2020 16:59 Til: development Emne: Re: [Development] Proposal: let's use Markdown for the dist/changes changelog On Monday, 2 November 2020 02:26:31 PST Shawn Rutledge wrote: > So in practice, changelogs still require manual editing. Probably there is > someone who knows just what settings to use in just what editor to make it > effortless, but for me it always involves a lot of tedious manual spacing, > because paragraph indentation is so damn weird (5 spaces usually), etc. > Creator doesn’t know how to wrap a paragraph in a changelog, for example. > There is no tool to center the text inside the star-box headings except for > the generator scripts themselves, AFAIK. Emacs works just fine. You just need to add spaces before and after the - lines so it detects what's a paragraph, then use M-q to rewrap. The centering can be done by removing the leading and tailing stars, then use M-x center- line. At least, this is how it was done when I created the script. But I wholeheartedly agree on changingi to either Markdown or ReStructuredText for the headings and the bullets. > One possible bikeshed is whether we should use the easier format for > headings like > # QtQuick > > ## Animations > > or one that perhaps looks a bit better in the terminal, but only allows 2 > levels of headings (which is enough in practice, so far): > QtQuick > === > > Animations > -- RST allows for far more levels even in the more verbose way. Anyway, since we only use two levels anyway and I think we should avoid adding more levels, I would prefer the more verbose way. Note: the levels here are "Libraries" and "QtQuick". "Animations" is a big bullet and "alwaysRunToEnd==true" (a bad entry) is a sub-bullet. > But I didn’t like it when we generate a point-release changelog that says > there are only minor changes, when I know for a fact that some significant > bugs got fixed, and should be mentioned, even though the author forgot to > add a ChangeLog to the git commit. So a few years ago, before each release > I was actually scanning the git log manually to look for such bug fixes > that ought to be mentioned. I got tired of that because it always requires > several windows at the same time: a terminal to view the git log, an editor > for the changes file, and a browser with a couple of tabs open to look up > bugs and code changes. Yep, same here. > So I modified Simon’s go script to also find the > git log entries that mention bugs, and append the git commit message plus > extra text taken from the jira subject line. And I just quietly used that > script to overwrite the generated changelogs, and then edited by hand to > format it nicely, in qtdeclarative only. So probably qtdeclarative has had > more-complete change logs than most other modules during the last few > years. I’d selectively use it to add a few qtbase changes that I wanted to > highlight too though, now and then. Other people were using it in other > modules sometimes. Extracting extra entries is a welcome addition. If you can just mark the entries thus extracted so we know what was written as an intentional changelog and what is speculative and thus requires more attention. > Then it was proposed that we might as well use the go script all the time. > And the result is what you can see now: > https://codereview.qt-project.org/c/qt/qtbase/+/319071 Nobody wants to > edit such a long changelog, instead they are just complaining that it’s a > mess. I was able to deal with this in qtdeclarative; I don’t mind the > mess, because it means I get all the information I need in the editor I’m > using, and I take it for granted that the changes file needs manual editing > anyway. But in qtbase it’s a lot of changes, and it will be a lot, each > time. Some people say if the author didn’
Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
> On 2 Nov 2020, at 20:03, Oswald Buddenhagen wrote: > > On Mon, Nov 02, 2020 at 10:26:31AM +, Shawn Rutledge wrote: >> [stuff about changelogs] > > for completeness: > > some of your points re-iterate what has already been said in > https://bugreports.qt.io/browse/QTQAINFRA-1960 (which presumably needs a > revision for the now used go program). Commented > and the bigger-picture issues have overlap with > https://bugreports.qt.io/browse/QTQAINFRA-933 Yes crowd-sourcing it sounds interesting, but then it’s another tool to fork and maintain, and maintain infrastructure for. > >> We could also make the sanity bot complain when a change that includes a >> bug-fix does not include a ChangeLog entry. >> > https://codereview.qt-project.org/c/qt/qtrepotools/+/185645 is awaiting > appropriate admin action for *years*. And it has +2 but is that enough? ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
On 2 Nov 2020, at 16:59, Thiago Macieira mailto:thiago.macie...@intel.com>> wrote: On Monday, 2 November 2020 02:26:31 PST Shawn Rutledge wrote: So in practice, changelogs still require manual editing. Probably there is someone who knows just what settings to use in just what editor to make it effortless, but for me it always involves a lot of tedious manual spacing, because paragraph indentation is so damn weird (5 spaces usually), etc. Creator doesn’t know how to wrap a paragraph in a changelog, for example. There is no tool to center the text inside the star-box headings except for the generator scripts themselves, AFAIK. Emacs works just fine. You just need to add spaces before and after the - lines so it detects what's a paragraph, then use M-q to rewrap. I tried. (I’m not an emacs user normally, but I’ve tried before; this time had to read some docs again to refresh my memory.) The wrapping still didn’t seem right to me. But I wholeheartedly agree on changingi to either Markdown or ReStructuredText for the headings and the bullets. I’m glad you agree. RST allows for far more levels even in the more verbose way. Anyway, since we only use two levels anyway and I think we should avoid adding more levels, I would prefer the more verbose way. OK. Extracting extra entries is a welcome addition. If you can just mark the entries thus extracted so we know what was written as an intentional changelog and what is speculative and thus requires more attention. I think it’s clear already if one reads all the way through: the generated entries end with the text “The bug was:” with the Jira subject line, and often a lot of verbosity from the git commit message. Then it was proposed that we might as well use the go script all the time. And the result is what you can see now: https://codereview.qt-project.org/c/qt/qtbase/+/319071 Nobody wants to edit such a long changelog, instead they are just complaining that it’s a mess. I was able to deal with this in qtdeclarative; I don’t mind the mess, because it means I get all the information I need in the editor I’m using, and I take it for granted that the changes file needs manual editing anyway. But in qtbase it’s a lot of changes, and it will be a lot, each time. Some people say if the author didn’t write a ChangeLog, tough luck: the change will not be mentioned. 6.0 is a very special case. For other releases, especially point releases, I can see the value. Yes but that link is for the 5.15.2 changelog. Which I’ve now done a few hours’ manual editing on. It’s amazing how many interesting bug fixes would be omitted if we only used the ChangeLog entries. The category is the class name. The script can get the categorization from the ChangeLog entry if there is one; but in the case that there isn’t, I doubt I will try to make it smart enough to figure out what was the main class that got changed, since patches often touch multiple classes. So for humans to edit afterwards, broader categories make it easier and more succinct. But yes, I see many changes files are using class names as categories. A full listing of what has been fixed, appended to the file, is not a bad idea anyway. I could try to make the bugs into reference links: inline they'd look the same as they already do: - [QTBUG-12345] Fixed a bad bug in QVariant. and then at the bottom of the whole file, a long list of URL definitions for every bug mentioned: [QTBUG-12345]: https://bugreports.qt.io/browse/QTBUG-12345 "QVariant conversion to QChar fails" Then HTML rendering will make real clickable links out of the inline references and omit the list at the bottom (it also works directly in QTextBrowser when you read that markdown: links are clickable); whereas otherwise [QTBUG-12345] is rendered as plain text in brackets if the link url is not defined elsewhere. Reference links are something else that QTextMarkdownWriter doesn’t rewrite yet, but it can be done. (And no problem to make the go script ganerate them, anyway.) ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
On Mon, Nov 02, 2020 at 10:26:31AM +, Shawn Rutledge wrote: [stuff about changelogs] for completeness: some of your points re-iterate what has already been said in https://bugreports.qt.io/browse/QTQAINFRA-1960 (which presumably needs a revision for the now used go program). and the bigger-picture issues have overlap with https://bugreports.qt.io/browse/QTQAINFRA-933 We could also make the sanity bot complain when a change that includes a bug-fix does not include a ChangeLog entry. https://codereview.qt-project.org/c/qt/qtrepotools/+/185645 is awaiting appropriate admin action for *years*. ___ Development mailing list Development@qt-project.org https://lists.qt-project.org/listinfo/development
Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
On Monday, 2 November 2020 02:26:31 PST Shawn Rutledge wrote: > So in practice, changelogs still require manual editing. Probably there is > someone who knows just what settings to use in just what editor to make it > effortless, but for me it always involves a lot of tedious manual spacing, > because paragraph indentation is so damn weird (5 spaces usually), etc. > Creator doesn’t know how to wrap a paragraph in a changelog, for example. > There is no tool to center the text inside the star-box headings except for > the generator scripts themselves, AFAIK. Emacs works just fine. You just need to add spaces before and after the - lines so it detects what's a paragraph, then use M-q to rewrap. The centering can be done by removing the leading and tailing stars, then use M-x center- line. At least, this is how it was done when I created the script. But I wholeheartedly agree on changingi to either Markdown or ReStructuredText for the headings and the bullets. > One possible bikeshed is whether we should use the easier format for > headings like > # QtQuick > > ## Animations > > or one that perhaps looks a bit better in the terminal, but only allows 2 > levels of headings (which is enough in practice, so far): > QtQuick > === > > Animations > -- RST allows for far more levels even in the more verbose way. Anyway, since we only use two levels anyway and I think we should avoid adding more levels, I would prefer the more verbose way. Note: the levels here are "Libraries" and "QtQuick". "Animations" is a big bullet and "alwaysRunToEnd==true" (a bad entry) is a sub-bullet. > But I didn’t like it when we generate a point-release changelog that says > there are only minor changes, when I know for a fact that some significant > bugs got fixed, and should be mentioned, even though the author forgot to > add a ChangeLog to the git commit. So a few years ago, before each release > I was actually scanning the git log manually to look for such bug fixes > that ought to be mentioned. I got tired of that because it always requires > several windows at the same time: a terminal to view the git log, an editor > for the changes file, and a browser with a couple of tabs open to look up > bugs and code changes. Yep, same here. > So I modified Simon’s go script to also find the > git log entries that mention bugs, and append the git commit message plus > extra text taken from the jira subject line. And I just quietly used that > script to overwrite the generated changelogs, and then edited by hand to > format it nicely, in qtdeclarative only. So probably qtdeclarative has had > more-complete change logs than most other modules during the last few > years. I’d selectively use it to add a few qtbase changes that I wanted to > highlight too though, now and then. Other people were using it in other > modules sometimes. Extracting extra entries is a welcome addition. If you can just mark the entries thus extracted so we know what was written as an intentional changelog and what is speculative and thus requires more attention. > Then it was proposed that we might as well use the go script all the time. > And the result is what you can see now: > https://codereview.qt-project.org/c/qt/qtbase/+/319071 Nobody wants to > edit such a long changelog, instead they are just complaining that it’s a > mess. I was able to deal with this in qtdeclarative; I don’t mind the > mess, because it means I get all the information I need in the editor I’m > using, and I take it for granted that the changes file needs manual editing > anyway. But in qtbase it’s a lot of changes, and it will be a lot, each > time. Some people say if the author didn’t write a ChangeLog, tough luck: > the change will not be mentioned. 6.0 is a very special case. For other releases, especially point releases, I can see the value. > We could also make the sanity bot complain when a change that includes a > bug-fix does not include a ChangeLog entry. But maybe that’s going too > far? Many non-native speakers would write badly worded changelog messages > anyway, so just getting more developers to write “something” there does not > remove the need for manual editing of the changes file after it’s > generated. But at least you’d get less-messy output from the changelog > generator, and reliably categorized. There's always going to be a need for an editor who understands what the change is about. > Maybe the commit message template should include commented-out example > changelog lines for every possible category, because otherwise you have to > scan the git log and look for examples each time you try to write one in a > commit message. At least I can’t remember all the categories that have > been used. The category is the class name. > So maybe that’s one of the obstacles, why people are not > writing them. But the commit message template could get really long that > way. Or we could write some tool for editing
Re: [Development] Proposal: let's use Markdown for the dist/changes changelog
Just as a note we are using markdown for Qt Creator changelogs since a long time too. With the “right” heading styles it is also nicely readable in ASCII. You can actually use ### as the third level of headings without compromising readablility. https://code.qt.io/cgit/qt-creator/qt-creator.git/tree/dist/changes-4.13.0.md And when viewing on other platforms it also gets nicely rendered https://github.com/qt-creator/qt-creator/blob/master/dist/changes-4.13.0.md Can’t comment on auto-generating, since for Qt Creator it’s all hand-crafted. Br, Eike > On Nov 2, 2020, at 11:26, Shawn Rutledge wrote: > > It has been frustrating for maintainers to generate changelogs in the > venerable wizened format that we use. One hope was that if everyone would > write ChangeLog messages in their git commits, the changelogs could be > generated automatically; and we have two competing scripts to do that (more > below about that). But in practice, it’s often neglected. I often forget > too, unless I get reminded somehow. Getting nagged about it on codereview is > not much fun for either party. > > So in practice, changelogs still require manual editing. Probably there is > someone who knows just what settings to use in just what editor to make it > effortless, but for me it always involves a lot of tedious manual spacing, > because paragraph indentation is so damn weird (5 spaces usually), etc. > Creator doesn’t know how to wrap a paragraph in a changelog, for example. > There is no tool to center the text inside the star-box headings except for > the generator scripts themselves, AFAIK. > > Markdown is much more standardized, and there are many editors and other > tools for working with it. Even Qt has built-in support in QTextDocument > since 5.14. So for comparison purposes, I’ll attach the 5.15.2 changelog for > qtdeclarative, and the proposal for how it could look in Markdown, which I > was editing just now in my QTextBrowser-based editor. In read-only mode in > QTextBrowser, it looks like this: > > > > Perhaps later we could use the markdown changelogs to render html for the web > site, or something. That’s not my concern yet; I just want to make editing > easier, and I think that users who are accustomed to viewing plain-text > changelogs are not losing anything either, because markdown also looks quite > fine in the terminal, IMO. There are even terminal formatting tools that > render it a bit more fancy (I wrote one version of mdcat in go, but there are > multiple versions of that out there). Vi can syntax-highlight markdown, and > there are hacks to make less use vi syntax highlighting, and stuff like that. > > One possible bikeshed is whether we should use the easier format for headings > like > > # QtQuick > > ## Animations > > or one that perhaps looks a bit better in the terminal, but only allows 2 > levels of headings (which is enough in practice, so far): > > QtQuick > === > > Animations > -- > > The advantage of the hash format is QTextDocument already writes markdown > that way; but I think I could (and probably should) somehow add a flag to > configure it to write either way. It’s able to read it either way, already. > > Now, more about the tooling. A perl script has been used for some years to > generate the dist/changes files from git ChangeLog entries. At some point > Simon wrote a go script which did more or less the same thing (because it’s a > minority of us who are happy working with perl, I guess; I also prefer go > because it’s easier to read, can be compiled, etc. But it was my first time > using go for anything real.) > > But I didn’t like it when we generate a point-release changelog that says > there are only minor changes, when I know for a fact that some significant > bugs got fixed, and should be mentioned, even though the author forgot to add > a ChangeLog to the git commit. So a few years ago, before each release I was > actually scanning the git log manually to look for such bug fixes that ought > to be mentioned. I got tired of that because it always requires several > windows at the same time: a terminal to view the git log, an editor for the > changes file, and a browser with a couple of tabs open to look up bugs and > code changes. So I modified Simon’s go script to also find the git log > entries that mention bugs, and append the git commit message plus extra text > taken from the jira subject line. And I just quietly used that script to > overwrite the generated changelogs, and then edited by hand to format it > nicely, in qtdeclarative only. So probably qtdeclarative has had > more-complete change logs than most other modules during the last few years. > I’d selectively use it to add a few qtbase changes that I wanted to highlight > too though, now and then. Other people were using it in other modules > sometimes. > > Then it was proposed that we might as well use the go
