I think the goal would be to minimize this editing, instead, try to improve/standardize formatting of commit messages to do whatever you need.
You can get some ideas from other projects doing it this way: Linux kernel: * detailed changelog: https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.9.12 * short changelog/release announcement: https://lkml.org/lkml/2020/12/2/277 Haproxy: * release announcement: https://www.mail-archive.com/[email protected]/msg39066.html I'm sure you can find more, but you get the idea that you can generate these notes without such manual editing, while improving commit message quality at the same time (these have no maintenance cost but a lot of other benefits). Maybe messages need to be standardized to include things like subsystem, severity, fix type, etc to get what you need. On Fri, Dec 4, 2020 at 11:16 AM David Smiley <[email protected]> wrote: > > Nice idea Rob... I like it! I didn't know other projects typically do it > this way. > > I suppose the release process would involve a manual collaborative editing > period of the generated change files, perhaps facilitated by the Confluence > wiki where we could clean up the raw output. Or maybe create a temporary > branch where we edit the file in any way we're comfortable (CLI/IDE/GitHub UI > editor). Since the file ought to be Markdown, I suppose the latter approach > is better than Confluence. > Tasks to be done by RM and helpers: > * group by Bug, New Feature, Improvement, Optimization, ... (etc.) > -- automated via JIRA lookup? Optimization vs Improvement distinction isn't > in JIRA but could be done manually. > * consolidation of fix-ups > * some editorializing cleanup > > Questions: > * Do we even need an HTML version of this? The alternative to HTML would be > Markdown committed to source control on the release branch (e.g. branch_8_8), > and it'd only contain changes for that release. We could add a link on the > bottom (in GitHub) to the previous release. A link from a release > announcement could point to the GitHub based link since GitHub renders the > Markdown nicely. A trivial script could link-ify LUCENE/SOLR JIRA issue IDs > to be URLs. If/when the HTML version goes, so does the changes2html.pl > script and related complexities. > * What is releasedJirasRegex.py used for? I suspect it may become obsolete. > > I had proposed module grouping, at least for Solr's contribs & Docker. We > could still do that... but I suggest delaying that until the above. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Thu, Dec 3, 2020 at 6:51 AM Robert Muir <[email protected]> wrote: >> >> git-log is better than JIRA for this. A lot of projects generate >> release notes from it. >> >> here's a dirty stab at LUCENE 8.6.2 release notes looking similar to >> CHANGES.txt: git log --format=format:"* %s%n%b (%an)%n" --grep >> "^LUCENE" releases/lucene-solr/8.6.1..releases/lucene-solr/8.6.2 >> >> --------- >> * LUCENE-9224: (ant) RAT report complains about ... solr/webapp >> rat-report.xml >> (Simon Willnauer) >> >> * LUCENE-9478: Prevent DWPTDeleteQueue from referencing itself and >> leaking memory (#1779) >> In LUCENE-9304 we introduced some fixes that unfortunately hold on to >> the previous >> DWPTDeleteQueue which is essentially leaking IW memory and cause >> applications to fail. >> This fixes the memory leak and adds a test to ensure its not leaking >> memory. (Simon Willnauer) >> -------- >> >> So it looks reasonable and can easily work, the problem is that commit >> messages need to be more consistent if you want to do fancy stuff like >> grouping and all that. >> >> On Wed, Dec 2, 2020 at 6:38 PM Rahul Goswami <[email protected]> wrote: >> > >> > Couldn't help pitching in here and making a humble request. The >> > CHANGES.txt has been of immense help for us for determining the right >> > upgrade version for our production deployments. So CHANGES.txt or no >> > CHANGES.txt, I hope we'll retain a mechanism to clearly be able to track >> > the changes in subsequent versions. >> > >> > Thanks, >> > Rahul >> > >> > On Mon, Nov 30, 2020 at 9:04 AM David Smiley <[email protected]> wrote: >> >> >> >> I get your point on different audiences... sometimes I peer-review us on >> >> dubiously written CHANGES.txt entries to be more user friendly. However, >> >> this attention could and should be given to JIRA issue summaries as well. >> >> We all benefit from that. Also, for Solr in particular, the need for >> >> examining CHANGES / JIRA is reduced because we have a >> >> solr-upgrade-notes.adoc which is editorialized and covers just the >> >> important stuff; no minor matters. We link to this from release >> >> announcements. >> >> >> >> ~ David Smiley >> >> Apache Lucene/Solr Search Developer >> >> http://www.linkedin.com/in/davidwsmiley >> >> >> >> >> >> On Mon, Nov 30, 2020 at 3:29 AM Adrien Grand <[email protected]> wrote: >> >>> >> >>> I have a preference for maintaining a separate CHANGES file because it >> >>> allows us to keep JIRA focused for a committer/contributor audience >> >>> while the CHANGES file can describe changes that matter for users. >> >>> Elasticsearch uses a similar mechanism for release notes to what you are >> >>> proposing, using GitHub instead of JIRA. It works well, but in my >> >>> opinion the Lucene/Solr process produces better curated release notes. >> >>> >> >>> On Mon, Nov 30, 2020 at 12:25 AM David Smiley <[email protected]> wrote: >> >>>> >> >>>> Well the commit history remains there as well and was converted from >> >>>> SVN and may eventually be converted to something else. My point is >> >>>> that it has been retained. On release boundaries, we could not only >> >>>> distribute Changes.html (a JIRA export) in the assembly (tar.gz) but we >> >>>> could also commit it to source control on each release branch, and thus >> >>>> will transfer along with source control into the future, which is way >> >>>> more convenient than digging up an old binary. >> >>>> >> >>>> ~ David Smiley >> >>>> Apache Lucene/Solr Search Developer >> >>>> http://www.linkedin.com/in/davidwsmiley >> >>>> >> >>>> >> >>>> On Sun, Nov 29, 2020 at 5:55 AM Dawid Weiss <[email protected]> >> >>>> wrote: >> >>>>> >> >>>>> Changes in the repository stay there forever (think >> >>>>> cvs/svn/git/whatever comes next...). External tools change all the >> >>>>> time. This is the benefit I see. >> >>>>> >> >>>>> Dawid >> >>>>> >> >>>>> On Sun, Nov 29, 2020 at 6:32 AM David Smiley <[email protected]> >> >>>>> wrote: >> >>>>> > >> >>>>> > After recently proposing per-module CHANGES.md... I think I'd >> >>>>> > actually rather not have any CHANGES file at all to maintain. I'd >> >>>>> > rather go to JIRA with a bit better hygiene for metadata like >> >>>>> > components==contrib/module, and have some convenient links sprinkled >> >>>>> > about so that it's a convenient click away from each module. This >> >>>>> > proposal may not be as compelling for Lucene which has no >> >>>>> > solr-upgrade-notes.adoc file. >> >>>>> > >> >>>>> > Maintaining this CHANGES file (or files) is a pain. Formatting it >> >>>>> > just-so & conversion to HTML & other scripts manipulating it in >> >>>>> > dev-tools (e.g. add version), and branch syncing. It's commonly a >> >>>>> > source of merge conflicts more than any other file. It's an >> >>>>> > annoying step with GitHub PRs in particular. Why do we bother? >> >>>>> > Instead, on releases, provide a JIRA link to display all fixed >> >>>>> > issues grouped by issue type. We could export it to a file for >> >>>>> > direct inclusion in the distribution. JIRA even has a feature for >> >>>>> > this -- here's a direct link for 8.7: >> >>>>> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310230&version=12348463 >> >>>>> > Notice the HTML version at the bottom. It could be dumped into >> >>>>> > the release binaries. >> >>>>> > Issue summaries tend to be much shorter than CHANGES.txt bullets but >> >>>>> > I think that's okay because it's not the only information available >> >>>>> > for those who want to know more. Remember there is also all the >> >>>>> > other metadata in JIRA a user can examine, there are commit >> >>>>> > messages, sometimes PRs, and there's solr-upgrade-notes.adoc which >> >>>>> > ought to be the starting point for someone interested in a release. >> >>>>> > >> >>>>> > It's been argued that contributors should get attribution here but >> >>>>> > we could maintain a separate contributors file to acknowledge people >> >>>>> > by name for inclusion with the Solr distribution -- one that has a >> >>>>> > link to JIRA and GitHub even. >> >>>>> > >> >>>>> > ~ David Smiley >> >>>>> > Apache Lucene/Solr Search Developer >> >>>>> > http://www.linkedin.com/in/davidwsmiley >> >>>>> >> >>>>> --------------------------------------------------------------------- >> >>>>> To unsubscribe, e-mail: [email protected] >> >>>>> For additional commands, e-mail: [email protected] >> >>>>> >> >>> >> >>> >> >>> -- >> >>> Adrien >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
