This is an automated email from the ASF dual-hosted git repository.
stack pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hbase.git
The following commit(s) were added to refs/heads/master by this push:
new 2e21da7 HBASE-25437 Update refguide RC section; point at the
dev-tools/create-releases/README.txt rather than repeat its content (#2804)
2e21da7 is described below
commit 2e21da7597d546fe15d1aafe253ee1917db63dda
Author: Michael Stack <[email protected]>
AuthorDate: Sat Jan 2 11:34:55 2021 -0800
HBASE-25437 Update refguide RC section; point at the
dev-tools/create-releases/README.txt rather than repeat its content (#2804)
M dev-support/create-release/README.txt
Remove redundant text. Add some extra help around figuring state of
gpg-agent.
M dev-support/create-release/do-release.sh
Undo my mistaken commit where I undid test of gpg signing if under docker
M dev-support/create-release/release-build.sh
Handle '-h'
M src/main/asciidoc/_chapters/developer.adoc
Point to the README.txt under dev-tools/create-release rather than
repeat the text in here. Be more insistent about using scripts.
---
dev-support/create-release/README.txt | 71 ++++----
dev-support/create-release/do-release.sh | 23 ++-
dev-support/create-release/release-build.sh | 4 +
src/main/asciidoc/_chapters/developer.adoc | 248 ++++++++--------------------
4 files changed, 118 insertions(+), 228 deletions(-)
diff --git a/dev-support/create-release/README.txt
b/dev-support/create-release/README.txt
index aff5624..42959cd 100644
--- a/dev-support/create-release/README.txt
+++ b/dev-support/create-release/README.txt
@@ -1,18 +1,10 @@
-Creates a HBase release candidate. The script will update versions, tag the
branch,
-build HBase binary packages and documentation, and upload maven artifacts to a
staging
-repository. There is also a dry run mode where only local builds are
performed, and
-nothing is uploaded to the ASF repos.
-
-Run with "-h" for options. For example, running below will do all
-steps above using the 'rm' dir under Downloads as workspace:
-
- $ ./do-release-docker.sh -d ~/Downloads/rm
+Creates an HBase release candidate.
The scripts in this directory came originally from spark
-(https://github.com/apache/spark/tree/master/dev/create-release). They were
then
-modified to suite the hbase context. These scripts supercedes the old
-../make_rc.sh script for making release candidates because what is here is more
-comprehensive doing more steps of the RM process as well as running in a
+(https://github.com/apache/spark/tree/master/dev/create-release). They were
+then modified to suit the hbase context. These scripts supercede the old
+_../make_rc.sh_ script for making release candidates because what is here is
+more comprehensive doing more steps of the RM process as well as running in a
container so the RM build environment can be a constant.
It:
@@ -24,49 +16,52 @@ It:
* Pushes release tgzs to the dev dir in a apache dist.
* Pushes to repository.apache.org staging.
-The entry point is here, in the do-release-docker.sh script. Requires a local
-docker; for example, on mac os x, Docker for Desktop installed and running.
+The entry point is the do-release-docker.sh script. It requires a local
+docker; for example, on mac os x, a Docker for Desktop installed and running.
- $ ./do-release-docker.sh -h
+(To run a build w/o invoking docker (not recommended!), use _do_release.sh_.)
-To run a build w/o invoking docker (not recommended!), use _do_release.sh_.
-
-Both scripts will query interactively for needed parameters and passphrases.
+The scripts will query interactively for needed parameters and passphrases.
For explanation of the parameters, execute:
$ release-build.sh --help
+The scripts run in dry-run mode by default where only local builds are
+performed and nothing is uploaded to the ASF repos. Pass the '-f' flag
+to remove dry-run mode.
+
Before starting the RC build, run a reconciliation of what is in JIRA with
what is in the commit log. Make sure they align and that anomalies are
-explained up in JIRA.
-
-See http://hbase.apache.org/book.html#maven.release
+explained up in JIRA. See http://hbase.apache.org/book.html#maven.release
+for how.
Regardless of where your release build will run (locally, locally in docker,
on a remote machine, etc) you will need a local gpg-agent with access to your
-secret keys. A quick way to tell gpg to clear out state and start a gpg-agent
-is via the following command phrase:
-
- $ gpgconf --kill all && gpg-connect-agent /bye
-
-Before starting an RC build, make sure your local gpg-agent has configs
-to properly handle your credentials, especially if you want to avoid
-typing the passphrase to your secret key.
-
-e.g. if you are going to run and step away, best to increase the TTL
-on caching the unlocked secret via ~/.gnupg/gpg-agent.conf
+secret keys. Before starting an RC build, make sure your local gpg-agent has
+configs to properly handle your credentials, especially if you want to avoid
+typing the passphrase to your secret key: e.g. if you are going to run
+and step away (the RC creation takes ~5 hours), best to increase the TTL on
+caching the unlocked secret by setting the following into local your
+~/.gnupg/gpg-agent.conf file:
# in seconds, e.g. a day
default-cache-ttl 86400
max-cache-ttl 86400
+A quick way to tell gpg to clear out state, re-read the gpg-agent.conf file
+and start a new gpg-agent is via the following command phrase:
+
+ $ gpgconf --kill all && gpg-connect-agent /bye
+
+You can verify options took hold with '$ gpg --list-options gpg-agent'.
+
Similarly, run ssh-agent with your ssh key added if building with docker.
Running a build on GCE is easy enough. Here are some notes if of use.
-Create an instance. 4CPU/15G/10G disk seems to work well enough.
+Create an instance. 4CPU/15G/20G disk seems to work well enough.
Once up, run the below to make your machine fit for RC building:
-# Presuming debian-compatible OS, do these steps on the VM
-# your VM username should be your ASF id, because it will show up in build
artifacts.
+# Presuming debian-compatible OS, do these steps on the VM.
+# Your VM username should be your ASF id, because it will show up in build
artifacts.
# Follow the docker install guide:
https://docs.docker.com/engine/install/debian/
$ sudo apt-get install -y \
apt-transport-https \
@@ -129,7 +124,3 @@ $ git clone https://github.com/apache/hbase.git
$ mkdir ~/build
$ cd hbase
$ ./dev-support/create-release/do-release-docker.sh -d ~/build
-
-# for building the main repo specifically you can save an extra download by
pointing the build
-# to the local clone you just made
-$ ./dev-support/create-release/do-release-docker.sh -d ~/build -r .git
diff --git a/dev-support/create-release/do-release.sh
b/dev-support/create-release/do-release.sh
index 5566b36..904d813 100755
--- a/dev-support/create-release/do-release.sh
+++ b/dev-support/create-release/do-release.sh
@@ -88,20 +88,19 @@ if [ "$RUNNING_IN_DOCKER" = "1" ]; then
else
# Outside docker, need to ask for information about the release.
get_release_info
-
- # Run this stuff when not in docker to check gpg.
- gpg_test_file="${TMPDIR}/gpg_test.$$.txt"
- echo "Testing gpg signing ${GPG} ${GPG_ARGS[@]} --detach --armor --sign
${gpg_test_file}"
- echo "foo" > "${gpg_test_file}"
- if ! "${GPG}" "${GPG_ARGS[@]}" --detach --armor --sign "${gpg_test_file}" ;
then
- gpg_agent_help
- fi
- # In --batch mode we have to be explicit about what we are verifying
- if ! "${GPG}" "${GPG_ARGS[@]}" --verify "${gpg_test_file}.asc"
"${gpg_test_file}" ; then
- gpg_agent_help
- fi
fi
+# Check GPG
+gpg_test_file="${TMPDIR}/gpg_test.$$.txt"
+echo "Testing gpg signing ${GPG} ${GPG_ARGS[@]} --detach --armor --sign
${gpg_test_file}"
+echo "foo" > "${gpg_test_file}"
+if ! "${GPG}" "${GPG_ARGS[@]}" --detach --armor --sign "${gpg_test_file}" ;
then
+ gpg_agent_help
+fi
+# In --batch mode we have to be explicit about what we are verifying
+if ! "${GPG}" "${GPG_ARGS[@]}" --verify "${gpg_test_file}.asc"
"${gpg_test_file}" ; then
+ gpg_agent_help
+fi
GPG_TTY="$(tty)"
export GPG_TTY
diff --git a/dev-support/create-release/release-build.sh
b/dev-support/create-release/release-build.sh
index 12cef12..cb13110 100755
--- a/dev-support/create-release/release-build.sh
+++ b/dev-support/create-release/release-build.sh
@@ -91,6 +91,10 @@ if [ $# -ne 1 ]; then
exit_with_usage
fi
+if [[ "$1" == "-h" ]]; then
+ exit_with_usage
+fi
+
if [[ "$*" == *"help"* ]]; then
exit_with_usage
fi
diff --git a/src/main/asciidoc/_chapters/developer.adoc
b/src/main/asciidoc/_chapters/developer.adoc
index a693992..59d6b71 100644
--- a/src/main/asciidoc/_chapters/developer.adoc
+++ b/src/main/asciidoc/_chapters/developer.adoc
@@ -625,54 +625,8 @@ upgrade protobuf later, then we don't have to add the
profile '-Paarch64' anymor
.Building against HBase 1.x
[NOTE]
====
-HBase 1.x requires Java 7 to build.
-See <<java,java>> for Java requirements per HBase release.
-====
-
-[[maven.settings.xml]]
-.Example _~/.m2/settings.xml_ File
-====
-Publishing to maven requires you sign the artifacts you want to upload.
-For the build to sign them for you, you a properly configured _settings.xml_
in your local repository under _.m2_, such as the following.
-
-[source,xml]
-----
-<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0";
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
- xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
- http://maven.apache.org/xsd/settings-1.0.0.xsd";>
- <servers>
- <!- To publish a snapshot of some part of Maven -->
- <server>
- <id>apache.snapshots.https</id>
- <username>YOUR_APACHE_ID
- </username>
- <password>YOUR_APACHE_PASSWORD
- </password>
- </server>
- <!-- To publish a website using Maven -->
- <!-- To stage a release of some part of Maven -->
- <server>
- <id>apache.releases.https</id>
- <username>YOUR_APACHE_ID
- </username>
- <password>YOUR_APACHE_PASSWORD
- </password>
- </server>
- </servers>
- <profiles>
- <profile>
- <id>apache-release</id>
- <properties>
- <gpg.keyname>YOUR_KEYNAME</gpg.keyname>
- <!--Keyname is something like this ... 00A5F21E... do gpg --list-keys to
find it-->
- <gpg.passphrase>YOUR_KEY_PASSWORD
- </gpg.passphrase>
- </properties>
- </profile>
- </profiles>
-</settings>
-----
+See old refguides for how to build HBase 1.x.
+The below is for building hbase2.
====
[[maven.release]]
@@ -699,30 +653,12 @@ target release will be included in the generated
_CHANGES.md/RELEASENOTES.md_
files that ship with the release so make sure JIRA is correct before you begin.
After doing the above, you can move to the manufacture of an RC.
-Building an RC is involved. We've tried to script it. In the next section
-we describe the script. It is followed by a description of the steps
-involved which the script automates.
-
-[[do-release-docker.sh]]
-==== Release Candidate Generating Script
-
-The _dev-support/create-release/do-release-docker.sh_ Release Candidate (RC)
-Generating script is maintained in the master branch but can generate RCs
-for any 2.x+ branch (The script does not work against branch-1). Check out
-and update the master branch when making RCs.
-The script builds in a Docker container to ensure we have a consistent
-environment building. It will ask you for passwords for apache and for your
-gpg signing key so it can sign and commit on your behalf. The passwords
-are passed to gpg-agent in the container and purged along with the container
-when the build is done.
-
-[NOTE]
-====
-_dev-support/create-release/do-release-docker.sh_ supercedes the previous
-_dev-support/make_rc.sh_ script. It is more comprehensive automating all
-steps, rather than a portion, building a RC.
-====
+Building an RC is involved so we've scripted it. The script builds in a Docker
+container to ensure we have a consistent environment building. It will ask you
+for passwords for apache and for your gpg signing key so it can sign and commit
+on your behalf. The passwords are passed to gpg-agent in the container and
+purged along with the container when the build is done.
The script will:
@@ -736,112 +672,25 @@ The script will:
* Pushes to repository.apache.org staging.
* Creates vote email template.
-The RC building script is _dev-support/create-release/do-release-docker.sh_.
-Pass _-h_ to _dev-support/create-release/do-release-docker.sh_ to
-see available options:
-
-```
-Usage: do-release-docker.sh [options]
-
-This script runs the release scripts inside a docker image.
-
-Options:
-
- -d [path] required. working directory. output will be written to "output"
in here.
- -n dry run mode. Checks and local builds, but does not upload
anything.
- -t [tag] tag for the hbase-rm docker image to use for building (default:
"latest").
- -j [path] path to local JDK installation to use building. By default the
script will
- use openjdk8 installed in the docker image.
- -s [step] runs a single step of the process; valid steps are: tag, build,
publish. if
- none specified, runs tag, then build, and then publish.
-```
-
-Running the below command will do all steps above using the
-'rm' working directory under Downloads as workspace:
-```
- $ ./dev-support/create-release/do-release-docker.sh -d ~/Downloads/rm
-```
-
-The script will ask you a set of questions about the release version
-and branch, the version to generate the compatibility report against,
-and so on, before it starts executing (If you set the appropriate
-environment variables, the script will skip asking you questions --
-which can come in handy if you end up having to re-run the script
-multiple times).
-
-On branch 2.1, a Release Candidate (RC) creation can take many hours
-(~8 hours) so run your build on a machine you know will be
-around for this swath of time. Start the build inside a _screen_
-or _tmux_ session in case you become disconnected from your
-build box.
-
-The build is made of three stages: tag, build, and
-publish. If the script fails, you may resort to 'fixing' the
-failure manually and then asking the script to run the
-subsequent stage rather than start over.
-
-When the scripts run, they use the passed working directory.
-Under the working directory is an _output_ dir. In here is
-where the checkouts go, where we build up the _svn_ directory
-to _svn_ commit to _apache/dist/dev_, etc. Each step also
-dumps a log file in here: e.g. _tag.log_ for the tagging
-step and _build.log_ for building.
-
-The _tagging_ step will checkout hbase, set the version number
-in all the poms – e.g. if branch-2.0 is at 2.0.6-SNAPSHOT
-and you are making a 2.0.5 RC, it will set the versions in
-all poms to 2.0.5 – appropriately. It then generate CHANGES.md
-and RELEASENOTES.md by checking out yetus and then
-calling its generator scripts. It then commits the poms with
-their new versions along with the changed CHANGES.md and
-RELEASENOTES.md, tags, and pushes up all changes to the
-apache repo.
-
-The _build_ step will checkout hbase, build all including
-javadoc and doc (javadoc takes the bulk of the time – 4 hours plus),
-run assemblies to produce src and bin tarballs, sign and hash it
-all, and then make a dir under apache dist dev named for the RC.
-It will copy all artifacts in here including top-level CHANGES.md
-and RELEASENOTES.md. It will generate api diff docs and put them
-into this RC dir too. When done, it commits the svn RC.
-
-The publish step will checkout hbase, build, and then copy up all
-artifacts to repository.apache.org (signed and hashed). When done,
-it will dump out an email template with all the correct links in place.
-
-Check the artifacts pushed to the dev distribution directory and up
-in repository.apache.org. If all looks good, check the generated
-email and send to the dev list.
-
-Under the create-release dir, scripts should make some sense:
-```
-do-release-docker.sh # Main entrance.
-do-release.sh . # More checks. Not usable really other than by setting
env variables before running it.
-release-tag.sh # Does tagging steps.
-release-build.sh . # Does the build and publish step.
-release-util.sh # Utility used by all of the above.
-vote.tmpl # Template for email to send out.
-hbase-rm # Has docker image we use.
-```
-
-If the RC fails, the script will do the right thing when it comes
-to edit of the _CHANGES.md_ and _RELEASENOTES.md_ removing the old
-and updating the files with the updated content (No harm verifying
-though).
-
-One trick for checking stuff especially in utility is to do as follows:
-
-```
-$ source release-util.sh ; generate_api_report ../../ rel/2.1.3 2.14RC1
-```
-
-i.e. source the release-util.sh script and then run one of its functions
-passing args. Helped debugging stuff.
-
-[[rc_procedure]]
+The _dev-support/create-release/do-release-docker.sh_ Release Candidate (RC)
+Generating script is maintained in the master branch but can generate RCs
+for any 2.x+ branch (The script does not work against branch-1). Check out
+and update the master branch when making RCs. See
+_dev-support/create-release/README.txt_ for how to configure your
+environment and run the script.
+
+[NOTE]
+====
+_dev-support/create-release/do-release-docker.sh_ supercedes the previous
+_dev-support/make_rc.sh_ script. It is more comprehensive automating all
+steps, rather than a portion, building a RC.
+====
+
==== Release Candidate Procedure
-Here we describe the steps involved generating a Release Candidate, the steps
-automated by the script described in the previous section.
+Here we outline the steps involved generating a Release Candidate, the steps
+automated by the _dev-support/create-release/do-release-docker.sh_ script
+described in the previous section. Running these steps manually tends to
+be error-prone so is not recommended. The below is informational only.
The process below makes use of various tools, mainly _git_ and _maven_.
@@ -859,6 +708,53 @@ MAVEN_OPTS="-Xmx4g -XX:MaxPermSize=256m" mvn package
You could also set this in an environment variable or alias in your shell.
====
+[[maven.settings.xml]]
+.Example _~/.m2/settings.xml_ File
+====
+Publishing to maven requires you sign the artifacts you want to upload.
+For the build to sign them for you, you a properly configured _settings.xml_
+in your local repository under _.m2_, such as the following.
+
+[source,xml]
+----
+<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0";
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
+ http://maven.apache.org/xsd/settings-1.0.0.xsd";>
+ <servers>
+ <!- To publish a snapshot of some part of Maven -->
+ <server>
+ <id>apache.snapshots.https</id>
+ <username>YOUR_APACHE_ID
+ </username>
+ <password>YOUR_APACHE_PASSWORD
+ </password>
+ </server>
+ <!-- To publish a website using Maven -->
+ <!-- To stage a release of some part of Maven -->
+ <server>
+ <id>apache.releases.https</id>
+ <username>YOUR_APACHE_ID
+ </username>
+ <password>YOUR_APACHE_PASSWORD
+ </password>
+ </server>
+ </servers>
+ <profiles>
+ <profile>
+ <id>apache-release</id>
+ <properties>
+ <gpg.keyname>YOUR_KEYNAME</gpg.keyname>
+ <!--Keyname is something like this ... 00A5F21E... do gpg --list-keys to
find it-->
+ <gpg.passphrase>YOUR_KEY_PASSWORD
+ </gpg.passphrase>
+ </properties>
+ </profile>
+ </profiles>
+</settings>
+----
+====
+
===== Update the _CHANGES.md_ and _RELEASENOTES.md_ files and the POM files.
Update _CHANGES.md_ with the changes since the last release. Be careful with
where you put
