Hi all,
(Sorry, big topic...)
In this [1] draft pull request I have started a way to
semi-automatically do releases, following the Apache rules (draft RC ->
vote -> publish -> announce). It is intended to be run in Github actions
and handles all the various manual steps, which can be time consuming
and are not that easy to get right all the time.
Building releases is in general a sensitive task that should be
performed in a separate environment in a reproducible way. Doing
releases in Github workflows and is a suitable environment for releases.
The necessary secrets (credentials for GPG, dist.apache.org, Apache
Sonatype and later for container registries) are only needed by the
release related workflows. It also makes it easier for non-(P)PMC
members to do releases.
While the scripts in the mentioned PR are intended to be run in GH
workflows, those can also be run locally for development and testing
purposes. Local runs do intentionally not touch Apache's Sonatype nor
dist.apache.org (SVN is simulated using local SVN repos, so the only
difference is the checkout location).
The "Git logic" is based on the principle to _not_ touch (change/push
to) the 'main' branch (or a version branch). This is quite important for
multiple reasons:
* Doing releases based on a specific commit that's not the "head" of a
branch
* Relying on pushes to the source branch from the release process can
easily race w/ concurrent commits/merges and let the release process fail
Instead, the release script(s) create a new commit with the necessary
changes (version.txt, generated release notes and some release meta
information) and creates a Git tag on that one. As an example:
O <-- Git tag 'apache-polaris-1.2.3-rc1'
/
---O---O---O---O--- <-- 'main' branch (or some version branch like
polaris-1.2)
Once the release has been drafted (all artifacts pushed/staged, the
RC-commit/tag has been created and pushed, vote email proposal
generated), the release vote can happen. If the vote fails, a new RC can
be created - the scripts take care of incrementing the RC number
(iteration). If the vote passes, the release can be published.
Publishing means that the Sonatype staging repository is released (Maven
artifacts published), a release Git tag is added to the RC Git tag (e.g.
'apache-polaris-1.2.3' - without the '-rcX' suffix), the source tarball
files are "moved" to the releases-SVN, website updated with the new
release, announcement-email proposal generated.
All scripts have some help and validation and runtime check logic. This
covers things like:
* The Git worktree must not be dirty
* Cannot create a draft (RC) for an already released version
* Cannot publish an already released version
* Automatically add the "-incubating" suffix to the version
(conditional, checks the "podling" status in whimsy.apache.org)
But also:
* Automatically increment the patch version - for example trying to a
version 1.2 release and 1.2.3 has already been released (Git tag
exists), an 1.2.4-rc1 will be created
* Automatically increment RC iteration (latest Git tag for a 1.2 release
is already an RC) - is is the case when a release vote failed
Noteworthy:
* The (previous) content of the version.txt file does not matter. The
release scripts overwrite the file with the "correct" version.
* The Git tags do not have the "-incubating" suffix, because that makes
handling the Git tags overly complex - especially when becoming a TLP.
The version itself however gets the "-incubating" suffix
How the scripts are actually invoked in production:
* releases/bin/draft-release.sh --major 2 --minor 4
* releases/bin/publish-release.sh --major 2 --minor 4
For local tests, start with adding the '--dry-run' option. When trying
it out without the '--dry-run' option, make sure your local
'versioned-docs' branch does NOT have 'apache/polaris' as its Git
upstream (publish-release.sh would push to it!). Also note that
artifacts will be signed, so you need to have gpg setup (uses the gpg
agent locally).
The release notes are generated from two major sources as markdown:
* Git commits between the previous version and the released version
* Contents of the (new) NEXT_RELEASE_NOTES.md file
The generated release notes start with a "thank you" to the contributors
for that release, followed by the content of the NEXT_RELEASE_NOTES.md
(if not empty) and the Git commits (subject, author and body), which are
enriched with links to GitHub (PR + commit)
Sorry for the wall of text,
Robert
PS: On top of the actual release scripts, there are also scripts to
handle version branches like 1.x, 1.0.x - those are not necessary for
the release process though.
[1] https://github.com/apache/polaris/pull/485
--
Robert Stupp
@snazy
