[
https://issues.apache.org/jira/browse/HBASE-21935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16797476#comment-16797476
]
stack commented on HBASE-21935:
-------------------------------
[~zghaobac] Here's some doc in meantime. Below will serve as start on doc for
refguide. Get the lastest patch from above. Its on the 2.0 branch version. Its
[^HBASE-21935.branch-2.0.018.patch] . Its on the branch-2.0 (There may be am
ore recent one -- check -- I've been updating as I fix stuff).
Do a checkout and apply the patch. It should apply to any checkout version. The
patch adds a subdir under dev-support called create-release. Currently all
scripts and subscripts have the -x set on them so can follow along. See below
for more.
h2. Making an RC
Before you begin, make sure JIRA is clean with no JIRAs filed against the
version to RC.
To generate an RC, use the
.{{/dev-support/create/release/do-release-docker.sh}} RC'maker script. It will
tag, generate changes and release notes, build, sign, upload to
repository.apache.org, commit to apache svn dev directory, generate api checker
and finally, produce the email to send the m ailing list. Pass -h to see
options. Runs the build inside a Docker container. This helps ensure consistent
output and avoids sporadic build failures because of differences in build tools
as context change with Release Manager.
{code}
$ ./do-release-docker.sh -h
Usage: do-release-docker.sh [options]
This script runs the release scripts inside a docker image.
Options:
-d [path] required. working directory. output will be written to "output" in
here.
-n dry run mode. Checks and local builds, but does not upload
anything.
-t [tag] tag for the hbase-rm docker image to use for building (default:
"latest").
-j [path] path to local JDK installation to use building. By default the
script will
use openjdk8 installed in the docker image.
-s [step] runs a single step of the process; valid steps are: tag, build,
publish. if
none specified, runs tag, then build, and then publish.
{code}
When you run the script, it will ask you questions about the RC you want to
generate as well as you apache login and password as well as your signing key
and its passphrase (All should be safe as gpg passphrase gets loaded to
gpg-agent running in the container and the passwords are cleaned up and deleted
on script and container exit).
The build takes about 8 hours. Since the script is new and may fail, its
probably best to run the three steps -- tag, build, publish -- one at a time.
If any one step fails, manually finish it by fixing the fail problem and then
completing the last few steps of the stage (see the scripts for what more to
do) before starting up the next stage.
When the scripts run, they use the passed working directory. Under the working
directory is an output dir. In here is where the checkouts go, where we build
up the svn directory to svn commit to apache/dist/dev, etc. Each step also
dumps a log file in here: e.g. tag.log for the tagging step and build.log for
building.
The tagging step will checkout hbase, set the version number in all the poms --
e.g. if branch-2.0 is at 2.0.6-SNAPSHOT and you are making a 2.0.5 RC, it will
set the versions in all poms to 2.0.5 -- appropriately, then generate changes
and release notes by checking out yetus. It will then commit the poms with
their new versions along with the changed CHANGES.md and RELEASENOTES.md and
then tag pushing up changes to apache repo.
The build step will checkout hbase, build all including javadoc and doc
(javadoc takes the bulk of the time -- 4 hours plus), run assemblies to produce
src and bin tarballs, sign and hash it all, and then make a dir under apache
dist dev named for the RC. It will copy all artifacts in here including
top-level CHANGES.md and RELEASENOTES.md. It will generate api diff docs and
put them into this RC dir too. When done, it commits the svn RC.
The publish step will checkout hbase, build, and then copy up all artifacts to
repository.apache.org (signed and hashed). When done, it will dump out an email
with all the correct links in place.
One known issue at the moment is that when running the publish step, you need
to keep an eye for just after the hbase checkout for when it goes to make its
first gpg signing. It will put up a text dialog to ask for the password. Try
and be around for when it asks else it times out and you have to start over (no
harm done). I think this only happens when you do the steps one-at-a-time
rather than all in the one run. TODO.
Under the create-release dir, scripts should make some sense:
{code}
do-release-docker.sh Main entrance.
do-release.sh . More checks. Not usable really other than by
setting env variables before running it.
release-tag.sh Does tagging steps.
release-build.sh . Does the build and publish step.
release-util.sh Utility used by all of the above.
vote.tmpl Template for email to send out.
hbase-rm Has docker image we use.
{code}
One trick for checking stuff especially in utility is to do as follows:
{code}$ source release-util.sh ; generate_api_report ../../ rel/2.1.3
2.14RC1{code}
i.e. source the script and then run one of its functions passing args. Helped
debugging stuff.
Will get this into refguide.
> Replace make_rc.sh with customized spark/dev/create-release
> -----------------------------------------------------------
>
> Key: HBASE-21935
> URL: https://issues.apache.org/jira/browse/HBASE-21935
> Project: HBase
> Issue Type: Task
> Components: rm
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Labels: rm
> Attachments: HBASE-21935.branch-2.0.001.patch,
> HBASE-21935.branch-2.0.002.patch, HBASE-21935.branch-2.0.003.patch,
> HBASE-21935.branch-2.0.004.patch, HBASE-21935.branch-2.0.005.patch,
> HBASE-21935.branch-2.0.006.patch, HBASE-21935.branch-2.0.007.patch,
> HBASE-21935.branch-2.0.008.patch, HBASE-21935.branch-2.0.009.patch,
> HBASE-21935.branch-2.0.010.patch, HBASE-21935.branch-2.0.011.patch,
> HBASE-21935.branch-2.0.012.patch, HBASE-21935.branch-2.0.013.patch,
> HBASE-21935.branch-2.0.014.patch, HBASE-21935.branch-2.0.015.patch,
> HBASE-21935.branch-2.0.016.patch, HBASE-21935.branch-2.0.017.patch,
> HBASE-21935.branch-2.0.018.patch, HBASE-21935.branch-2.1.001.patch,
> HBASE-21935.branch-2.1.002.patch, HBASE-21935.branch-2.1.003.patch,
> HBASE-21935.branch-2.1.004.patch, HBASE-21935.branch-2.1.005.patch,
> HBASE-21935.branch-2.1.006.patch, HBASE-21935.branch-2.1.007.patch
>
>
> The spark/dev/create-release is more comprehensive than our hokey make_rc.sh
> script. It codifies the bulk of the RM process from tagging, version-setting,
> building, signing, and pushing. It does it in a container so environment is
> same each time. It has a bunch of spark-specifics as is -- naturally -- but
> should be possible to pull it around to suit hbase RM'ing. It'd save a bunch
> of time and would allow us to get to a place where RM'ing is canned,
> evolvable, and consistent.
> I've been hacking on the tooling before the filing of this JIRA and was
> polluting branch-2.0 w/ tagging and reverts. Let me make a branch named for
> this JIRA to play with (There is a dry-run flag but it too needs work...).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
