Repository: flume Updated Branches: refs/heads/trunk 2399329ee -> 9b65219cc
Add Developer Section / How to Release from cwiki.apache.org Reviewers: Mike Percy This closes #77 Project: http://git-wip-us.apache.org/repos/asf/flume/repo Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/9b65219c Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/9b65219c Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/9b65219c Branch: refs/heads/trunk Commit: 9b65219cca4dd68a06aecdd51e953c9e367c3826 Parents: 2399329 Author: Bessenyei Balázs Donát <[email protected]> Authored: Mon Oct 24 16:53:29 2016 +0200 Committer: Bessenyei Balázs Donát <[email protected]> Committed: Tue Oct 25 10:35:08 2016 +0200 ---------------------------------------------------------------------- dev-docs/HowToRelease.md | 440 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 440 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flume/blob/9b65219c/dev-docs/HowToRelease.md ---------------------------------------------------------------------- diff --git a/dev-docs/HowToRelease.md b/dev-docs/HowToRelease.md new file mode 100644 index 0000000..897b270 --- /dev/null +++ b/dev-docs/HowToRelease.md @@ -0,0 +1,440 @@ +<!--- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + http://www.apache.org/licenses/LICENSE-2.0 +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> +Apache Flume: How to Release +============================= + +This document describes how to make a release of Flume. It is a work in +progress and should be refined by the [Release +Manager](http://incubator.apache.org/guides/releasemanagement.html#glossary-release-manager) +(RM) as they come across aspects of the release process not yet +documented here. + +NOTE: For the purpose of illustration, this document assumes that the +version being released is 1.0.0, and the following development version +will become 1.1.0. + +Prerequisites +------------- + +### Policy documents + +The policy on releasing artifacts from an incubating Apache project is +stated in the [Guide to Release Management During +Incubation](http://incubator.apache.org/guides/releasemanagement.html#glossary-release-manager). +While Flume is now a top-level project, that guide is still a good +resource for new release managers since it contains what are considered +to be best practices. + +The Release Manager (RM) must go through the policy document to +understand all the tasks and responsibilities of running a release. + +### Give a heads up + +The RM should first create an umbrella issue and then setup a timeline +for release branch point. The time for the day the umbrella issue is +created to the release branch point must be at least two weeks in order +to give the community a chance to prioritize and commit any last minute +features and issues they would like to see in the upcoming release. + +The RM should then send the pointer to the umbrella issue along with the +tentative timeline for branch point to the user and developer lists. Any +work identified as release related that needs to be completed should be +added as a subtask of the umbrella issue to allow users to see the +overall release progress in one place. + +TODO: Add an email template to send out + + +### JIRA cleanup + +1\. Before a release is done, make sure that any issues that are fixed +have their fixVersion setup correctly. Run the following JIRA query to +see which resolved issues do not have their fix version set up +correctly: + +**`JIRA: project = Flume and Resolution = Fixed and fixVersion is EMPTY`** + +The result of the above query should be empty. If some issues do show up +in this query that have been fixed since the last release, please +bulk-edit them to set the fix version to the version being released. + +2\. Since the JIRA release note tool will list all JIRAs associated with +a particular fixVersion regardless of their Resolution, you may want to +remove fixVersion on issues declared as "Duplicate", "Won't Fix", "Works +As Expected", etc. You can use this query to find those issues, then +just bulk edit them and leave the fixVersion field blank when editing +them. (Note that you will need to replace "v1.4.0" below with the +version you are trying to release.) + +**`JIRA: project = Flume AND (Status != Resolved OR Resolution != Fixed) AND fixVersion = v1.6.0`** + +3\. Finally, check out the output of the **[JIRA release note +tool](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311321)** +to see which JIRAs are included in the release, in order to do a sanity +check. + +### Monitor active issues + +It is important that between the time that the umbrella issue is filed +to the time when the release branch is created, no experimental or +potentially destabilizing work is checked into the trunk. While it is +acceptable to introduce major changes, they must be thoroughly reviewed +and have good test coverage to ensure that the release branch does not +start of being unstable. + +If necessary the RM can discuss if certain issues should be fixed on the +trunk in this time, and if so what is the gating criteria for accepting +them. + +Creating Release Artifacts +-------------------------- + +### Communicate with the community + +1\. Send an email to [email protected] to + +Notify that you are about to branch. + +Ask to hold off any commits until this is finished. + +2\. Send another email after branching is done. + +### Update the LICENSE file + +The release manager is responsible for updating the LICENSE file to +provide accurate license information for all binary artifacts contained +in the codebase. This is a tedious and painstaking process, and must be +performed for each release that includes a binary artifact. + +### Prepare branches and create tag + +In this section, the release is X.Y.Z (e.g. 1.3.0) + +1\. Checkout the trunk. + + git clone http://git-wip-us.apache.org/repos/asf/flume.git flume + +2\. Update CHANGELOG in the trunk to indicate the changes going into the +new version. + +The change list can be swiped from the **[JIRA release note +tool](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311321)** +(choose the "text" format for the change log). See JIRA Cleanup above to +ensure that the release notes generated by this tool are what you are +expecting. Additionally, in the last two releases we have re-ordered the +issues to be New Features, Improvements, Bugs, etc as by default the +least important items are at the top. + +3\. Update the "version" value in RELEASE-NOTES in trunk to X.(Y+1).0: + + vim RELEASE-NOTES + git add RELEASE-NOTES + git commit RELEASE-NOTES -m "Updating Flume version in RELEASE-NOTES for release X.Y.Z" + +4\. Create a branch for the X.(Y+1) release series + +E.g. flume-1.5 should be branched off of trunk + + git checkout trunk + git checkout -b flume-X.(Y+1) + +4\. Update the "version" value of all pom.xml and documentation files in +trunk to X.(Y+1).0-SNAPSHOT + + git checkout trunk + find . -name pom.xml | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.$(Y+1).0-SNAPSHOT/" + find flume-ng-doc/ -name "*.rst" | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.$(Y+1).0-SNAPSHOT/" + git add . + git commit -m "Updating trunk version to X.$(Y+1).0-SNAPSHOT for Flume X.Y.Z release" + +5\. Checkout the release branch and remove -SNAPSHOT from the release +branch poms and docs and commit: + + git checkout flume-X.Y + + find . -name pom.xml | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.Y.0/" + find flume-ng-doc/ -name "*.rst" | xargs sed -i "" -e "s/X.Y.0-SNAPSHOT/X.Y.0/" + git add . + git commit -m "FLUME-XXXX: Removing -SNAPSHOT from X.Y branch" + +6\. Ensure RELEASE-NOTES has the appropriate version and description of +the release. + +7\. Push the branching changes upstream + + + git push -u origin trunk:trunk + git push -u origin flume-X.Y:flume-X.Y + + +8\. Tag a release candidate (in the example below, RC1): + + + git tag -a release-X.Y.Z-rc1 -m "Apache Flume X.Y.Z RC1" + git push origin release-X.Y.Z-rc1 + + +If an rc2, rc3 etc is needed, simply create a new rc tag: + + + git tag -d release-X.Y.Z-rc2 + git push origin release-X.Y.Z-rc2 + + +### Performing sanity check + +1\. Check out the candidate tag + + + git checkout release-X.Y.Z-rc1 + + +2\. Generate a tarball + + + mvn clean install -DskipTests + + +3\. Unpack the source tarball + + + cd flume-ng-dist/target + rm -rf ./apache-flume-X.Y.Z-src/ + tar xzvf apache-flume-X.Y.Z-src.tar.gz + + +4\. Do another full build inside the source tarball. This time, allow all +unit tests & integration tests to run and also include the docs + + + cd apache-flume-X.Y.Z-src + export LC_ALL=C.UTF-8 # Required to build the javadocs on some platforms and in some locales + mvn clean install -Psite -DskipTests + + +5\. Verify that the HTML docs that should have been generated inside the +binary artifact under /docs are there and do not have rendering errors. + +### Signatures and Checksums + +All artifacts must be signed and checksummed. In order to sign a release +you will need a PGP key. You should get your key signed by a few other +people. You will also need to recv their keys from a public key server. +See the [Apache release +signing](https://www.apache.org/dev/release-signing) +page for more details. + +1\. Add your key to the +[KEYS](https://dist.apache.org/repos/dist/release/flume/KEYS) +file: + + + (gpg --list-sigs <your-email> && gpg --armor --export <your-email>) >> KEYS + + +And commit the changes. + +2\. Create and sign the artifacts, including site docs. This pushes the +signed artifacts to the ASF staging repository. + +In order to do this, you will need a settings.xml file with your +username and password for the ASF staging repository. Typically this is +placed in \~/.m2/settings.xml and might look something like this: + + + <settings> + ââ<servers> + ââââ<server> + ââââââ<id>apache.staging.https</id> + ââââââ<username>your_user_id</username> + ââââââ<password>your_password</password> + ââââ</server> + ââ</servers> + </settings> + + +Once your settings.xml file is correct, you run the following from the +flume root directory to generate and deploy the artifacts: + + + mvn clean deploy -Psite -Psign -DskipTests + + +This will sign, hash, and upload each artifact to Nexus. + +Note: the checksum files will not be mirrored; They should be downloaded +from the main apache dist site. + +Do the same for javadoc and source artifacts: + + mvn javadoc:jar gpg:sign deploy:deploy + mvn source:jar gpg:sign deploy:deploy + + +3\. Publish Staging repository + +Login to <https://repository.apache.org> and select Staging Repositories +on the left under Build Promotion. + +Select org.apache.flume from the list of repositories, verify it looks +OK, and then click Close using "Apache Flume X.Y.Z" as the description +to allow others to see the repository. Note that the staging repository +will have a numeric id associated with it that will be used later + +4\. Copy artifacts to people.apache.org + +Copy the apache-flume-X.Y.Z-{bin,src}.tar.gz{,.{asc,md5,sha1}} files to +people.apache.org. + + $ ssh people.apache.org + $ cd public_html + $ mkdir apache-flume-X.Y.Z-rcN + $ cd apache-flume-X.Y.Z-rcN + $ wget --no-check-certificate https://repository.apache.org/content/repositories/orgapacheflume-XXXX/org/apache/flume/flume-ng-dist/X.Y.Z/flume-ng-dist-X.Y.Z-{src,bin}.tar.gz{,.{asc,md5,sha1}} + $ for file in flume-ng-dist-*; do mv $file $(echo $file | sed -e "s/flume-ng-dist/apache-flume/g");done + + +Running the vote +---------------- + +### Call for dev list votes + +Send an email to [email protected] list. For example, + + To: [email protected] + Subject: [VOTE] Release Apache Flume version X.Y.Z RC1 + + This is the XXXXX release for Apache Flume as a top-level project, + version X.Y.Z. We are voting on release candidate RC1. + + It fixes the following issues: + <Link-to-CHANGELOG-in-the-tag> + + *** Please cast your vote within the next 72 hours *** + + The tarball (*.tar.gz), signature (*.asc), and checksums (*.md5, *.sha1) + for the source and binary artifacts can be found here: + https://people.apache.org/~mpercy/flume/apache-flume-X.Y.Z-RC1/ + + Maven staging repo: + https://repository.apache.org/content/repositories/orgapacheflume-XXXXX/ + + The tag to be voted on: + https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=<commit-hash-of-the-tag> + + Flume's KEYS file containing PGP keys we use to sign the release: + https://svn.apache.org/repos/asf/flume/dist/KEYS + + +You need 3 +1 votes from Flume PMC members for a release. + + +Rolling out the Release +----------------------- + +### Upload the artifacts + +#### Source and convenience artifacts + + svn checkout https://dist.apache.org/repos/dist/release/flume dist-flume + cd dist-flume + mkdir 1.0.0 + cp <all release artifacts including asc/checksum files> 1.0.0/ + svn rm stable # remove older version link + ln -s 1.0.0 stable # update stable version link + svn add stable 1.0.0 + svn commit + + +It may take up to 24 hours for all mirrors to sync up. + +#### Deploy Maven artifacts + +General instructions on how to deploy the poms and jars to Maven Central +can be found at the [Apache page on publishing maven +artifacts](https://www.apache.org/dev/publishing-maven-artifacts.html). + +**Flume-specific instructions:** + +After you get your `settings.xml` file configured correctly (see above +link) then you can run: +`mvn clean deploy -Psite -DskipTests -Papache-release` to push the +artifacts to the staging repo.\ +You will need to go to the Apache Maven Repository @ +<http://repository.apache.org/> and log in with your Apache LDAP +credentials\ +Once you log in you will see Build Promotion > Staging Repositories +on the left hand side\ +You will want to edit the Flume artifacts that you don't want to push to +Maven (you can delete stuff like the release tarball)\ +Click Close to make the atrifacts available on the Staging repository.\ +To push to Central, you click Release. + +### Announce the release + +Send an email to [email protected] (the from: address must be [email protected]). For example, + + + To: [email protected], [email protected], [email protected] + Subject: [ANNOUNCE] Apache Flume 1.2.0 released + + The Apache Flume team is pleased to announce the release of Flume + version 1.2.0. + + Flume is a distributed, reliable, and available service for efficiently + collecting, aggregating, and moving large amounts of log data. + + This release can be downloaded from the Flume download page at: + http://flume.apache.org/download.html + + The change log and documentation are available on the 1.2.0 release page: + http://flume.apache.org/releases/1.2.0.html + + Your help and feedback is more than welcome. For more information on how + to report problems and to get involved, visit the project website at + http://flume.apache.org/ + + The Apache Flume Team + +### Update the website + +1. Checkout <https://svn.apache.org/repos/asf/flume/site/trunk> +2. Add a page to the `content/sphinx/releases` directory for the new + release, with the Changelog and links to the documentation (refer + previous release pages for details. The documentation should simply + use the same paths as the previous releases with correct versions - + the documentation will be checked in directly to the production + website as mentioned below). +3. Update `content/sphinx/releases/index.rst` to update the pointer to + the latest release. +4. Update `content/sphinx/download.rst` to also point to the + latest release. +5. Update `content/sphinx/index.rst` as necessary to add a News item to + the home page. +6. Copy the release version of the FlumeUserGuide.rst and + FlumeDeveloperGuide.rst to the Documentation directory. +7. Commit the changes to svn. Go to <https://cms.apache.org/flume/> - + Stage and publish the changes. +8. Checkout + <https://svn.apache.org/repos/infra/websites/production/flume>. +9. Create a directory with the current release's directory name under + content/releases/content (e.g., content/releases/content/1.3.1) +10. Copy both the HTML and PDF versions of the user guide and developer + guide (manually generate the PDF from the HTML files), and the + javadocs (apidocs directory) into this directory. +11. Commit the changes to svn. Done!
