Repository: parquet-cpp Updated Branches: refs/heads/master 641984e6d -> a1517582f
PARQUET-767: Add release scripts Based on the release scripts from Apache Aurora (as we don't have the luxury of the Maven release plugin, we need a bit more code). Kudos to @StephanErb for pointing me to them. - [x] Rename JIRA release tag from `cpp-0.1` to `cpp-0.1.0` ( @julienledem can you take care of this?) This is then in line with all other Apache Parquet artifacts. - [x] Test the scripts with dry-run mode (needs the above point) - [x] Reference Apache Aurora in LICENSE/NOTICE - [x] Add Guides - [x] Add script to verify the release Author: Korn, Uwe <[email protected]> Author: Uwe L. Korn <[email protected]> Closes #199 from xhochy/PARQUET-767 and squashes the following commits: 0f3ca22 [Korn, Uwe] Add script to verify release 58a24f9 [Uwe L. Korn] Add guide 37c4877 [Korn, Uwe] Mention Apache Aurora in the License be828bd [Korn, Uwe] Fix path to changelog script 1adf248 [Korn, Uwe] PARQUET-767: Add release scripts Project: http://git-wip-us.apache.org/repos/asf/parquet-cpp/repo Commit: http://git-wip-us.apache.org/repos/asf/parquet-cpp/commit/a1517582 Tree: http://git-wip-us.apache.org/repos/asf/parquet-cpp/tree/a1517582 Diff: http://git-wip-us.apache.org/repos/asf/parquet-cpp/diff/a1517582 Branch: refs/heads/master Commit: a1517582fb2bcdde970e7c36be2e4a78cc9dbb5b Parents: 641984e Author: Korn, Uwe <[email protected]> Authored: Sat Dec 10 09:29:24 2016 +0100 Committer: Uwe L. Korn <[email protected]> Committed: Sat Dec 10 09:29:24 2016 +0100 ---------------------------------------------------------------------- .parquetcppversion | 1 + LICENSE.txt | 11 ++ dev/committers-guide.md | 118 ++++++++++++ dev/release/changelog | 155 +++++++++++++++ dev/release/release | 250 ++++++++++++++++++++++++ dev/release/release-candidate | 308 ++++++++++++++++++++++++++++++ dev/release/verify-release-candidate | 92 +++++++++ 7 files changed, 935 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/.parquetcppversion ---------------------------------------------------------------------- diff --git a/.parquetcppversion b/.parquetcppversion new file mode 100644 index 0000000..b694fe3 --- /dev/null +++ b/.parquetcppversion @@ -0,0 +1 @@ +0.1.0-SNAPSHOT http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/LICENSE.txt ---------------------------------------------------------------------- diff --git a/LICENSE.txt b/LICENSE.txt index 1bf56ab..5e63e2b 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -244,3 +244,14 @@ Home page: http://impala.apache.org/ License: http://www.apache.org/licenses/LICENSE-2.0 -------------------------------------------------------------------------------- + +This product includes code from Apache Aurora. + +* dev/release/{release,changelog,release-candidate} is based on the scripts from + Apache Aurora + +Copyright: 2016 The Apache Software Foundation. +Home page: https://aurora.apache.org/ +License: http://www.apache.org/licenses/LICENSE-2.0 + +-------------------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/committers-guide.md ---------------------------------------------------------------------- diff --git a/dev/committers-guide.md b/dev/committers-guide.md new file mode 100644 index 0000000..9c15071 --- /dev/null +++ b/dev/committers-guide.md @@ -0,0 +1,118 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. + + Original imported from Apache Aurora. +--> + +Committer's Guide +================= + +Information for official Apache Parquet committers. + +Setting up your email account +----------------------------- +Once your Apache ID has been set up you can configure your account and add ssh keys and setup an +email forwarding address at + + http://id.apache.org + +Additional instructions for setting up your new committer email can be found at + + http://www.apache.org/dev/user-email.html + +The recommended setup is to configure all services (mailing lists, JIRA) to send +emails to your @apache.org email address. + + +Creating a gpg key for releases +------------------------------- +In order to create a release candidate you will need a gpg key published to an external key server +and that key will need to be added to our KEYS file as well. + +1. Create a key: + + gpg --gen-key + +2. Add your gpg key to the Apache Parquet {format,mr,cpp} KEYS file: + + git clone https://git-wip-us.apache.org/repos/asf/parquet-{format,mr,cpp}.git + (gpg --list-sigs <KEY ID> && gpg --armor --export <KEY ID>) >> KEYS + git add KEYS && git commit -m "Adding gpg key for <APACHE ID>" + ./rbt post -o -g + +3. Publish the key to an external key server: + + gpg --keyserver pgp.mit.edu --send-keys <KEY ID> + +4. Update the changes to the KEYS file to the Apache Parquet svn dist locations listed below: + + https://dist.apache.org/repos/dist/dev/parquet/KEYS + https://dist.apache.org/repos/dist/release/parquet/KEYS + +5. Add your key to git config for use with the release scripts: + + git config --global user.signingkey <KEY ID> + + +Creating a release +------------------ +The following will guide you through the steps to create a release candidate, vote, and finally an +official Apache Parquet C++ release. Before starting your gpg key should be in the KEYS file and you +must have access to commit to the dist.a.o repositories. + +1. Ensure that all issues resolved for this release candidate are tagged with the correct Fix +Version in JIRA, the changelog script will use this to generate the CHANGELOG in step #2. +To assign the fix version: + + * Look up the [previous release date](https://issues.apache.org/jira/browse/parquet/?selectedTab=com.atlassian.jira.jira-projects-plugin:versions-panel). + * Query all issues resolved after that release date: `project = PARQUET AND status in (resolved, Closed) and fixVersion is empty and resolutiondate >= "YYYY/MM/DD"` + * In the upper right corner of the query result, select Tools > Bulk Edit. + * Select all issues > edit issue > set 'Change Fix Version/s' to the release version. + * Make sure to uncheck 'Send mail for this update' at the bottom. + +2. Prepare RELEASE-NOTES.md for the release. This just boils down to removing the "(Not yet +released)" suffix from the impending release. + +2. Create a release candidate. This will automatically update the CHANGELOG and commit it, create a +branch and update the current version within the trunk. To create a minor version update and publish +it run + + ./dev/release/release-candidate -l m -p + +3. Update, if necessary, the draft email created from the `release-candidate` script in step #2 and +send the [VOTE] email to the dev@ mailing list. You can verify the release signature and checksums +by running + + ./dev/release/verify-release-candidate + +4. Wait for the vote to complete. If the vote fails close the vote by replying to the initial [VOTE] +email sent in step #3 by editing the subject to [RESULT][VOTE] ... and noting the failure reason +(example [here](http://markmail.org/message/d4d6xtvj7vgwi76f)). You'll also need to manually revert +the commits generated by the release candidate script that incremented the snapshot version and +updated the changelog. Once that is done, now address any issues and go back to step #1 and run +again, this time you will use the -r flag to increment the release candidate version. This will +automatically clean up the release candidate rc0 branch and source distribution. + + ./dev/release/release-candidate -l m -r 1 -p + +5. Once the vote has successfully passed create the release + +**IMPORTANT: make sure to use the correct release at this final step (e.g.: `-r 1` if rc1 candidate +has been voted for). Once the release tag is pushed it will be very hard to undo due to remote +git pre-receive hook explicitly forbidding release tag manipulations.** + + ./dev/release/release + +6. Update the draft email created fom the `release` script in step #5 to include the Apache ID's for +all binding votes and send the [RESULT][VOTE] email to the dev@ mailing list. + http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/changelog ---------------------------------------------------------------------- diff --git a/dev/release/changelog b/dev/release/changelog new file mode 100755 index 0000000..d6ed9f2 --- /dev/null +++ b/dev/release/changelog @@ -0,0 +1,155 @@ +#!/usr/bin/env ruby +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Updates the CHANGELOG for the specified Apache Parquet C++ release, if no +# version override is specified then .parquetcppversion will be read and used. +# The version string will need to match the Apache Parquet C++ jira tickets +# fixVersion to generate this list. +# +require 'rubygems' +require 'json' +require 'uri' +require 'net/http' +require 'net/https' +require 'openssl' +require 'tempfile' + +def http_post(uri, params={}, headers={}, body=nil, debug=false) + http = Net::HTTP.new(uri.host, uri.port) + http.set_debug_output($stdout) if debug + if uri.scheme == "https" || uri.port == 443 + http.use_ssl = true + http.verify_mode = OpenSSL::SSL::VERIFY_NONE + end + response = nil + + begin + request = Net::HTTP::Post.new(uri.request_uri) + request.set_form_data(params) if (params && params.size > 1) + request.basic_auth uri.user, uri.password if uri.user + request.body = body if body + + headers.each_pair do |name, value| + request[name] = value + end + + response = http.request(request) + rescue Exception => e + raise "Error posting to #{uri.to_s}. #{e}" + end + + response +end + +def get_jira_issues_for_query(url, jql, startAt=0, maxResults=100) + puts(jql) + begin + request_body = { :jql => jql, :startAt => startAt, :maxResults => maxResults} + response = http_post( + URI(url), + {}, + {"content-type" => 'application/json', "accept" => "application/json"}, + request_body.to_json + ) + rescue StandardError => e + raise "Error executing jql query: #{jql}. #{e}" + end + + begin + results = JSON.parse(response.body) + rescue StandardError => e + raise "Error parsing json result" + end + + results +end + +def get_all_jira_issues_for_query(url, jql) + startAt = 0 + maxResults = 100 + + results = [] + begin + res = get_jira_issues_for_query(url, jql, startAt, maxResults) + results.concat(res['issues']) if res && res.has_key?('issues') + startAt = startAt + maxResults + end until (results.size() == res['total'] ) + + results +end + +base_dir = `git rev-parse --show-toplevel`.strip + +# Get the current version from the .parquetcppversion file if no version override is provided +version = nil +if ARGV[0].nil? + version = nil + Dir.chdir(base_dir) do + version = File.read('.parquetcppversion').strip + end +else + version = ARGV[0] +end + +raise "Unable to read .parquetcppversion" if version.nil? + +jira_base_url = "https://issues.apache.org/jira"; +jira_search_url = "/rest/api/2/search" +jira_url = jira_base_url + jira_search_url + +jql="project=PARQUET " \ + "AND fixVersion='cpp-#{version}' " \ + "AND status = Resolved " \ + "AND resolution in (Fixed, Done) " \ + "ORDER BY issuetype DESC" + +# Fetch all the issues available for the given jql query +results = get_all_jira_issues_for_query(jira_url, jql) + +changelog = {} +# Loop through and add all results +results.each do |issue| + key = issue['key'] + summary = issue['fields']['summary'] + type = issue['fields']['issuetype']['name'] + + changelog_entry = "[#{key}] - #{summary}" + + if !changelog[type].nil? + changelog[type] << changelog_entry + else + changelog[type] = [changelog_entry] + end +end + +# Merge the new updates and the existing changelog +tmpfile = Tempfile.open('parquet-cpp.changelog') +begin + tmpfile.puts "Parquet C++ #{version}", "-" * 80 + changelog.keys.sort.each do |type| + tmpfile.puts "## #{type}" + changelog[type].each { |entry| tmpfile.puts " * #{entry}" } + tmpfile.puts "" + end + # Append all the existing CHANGELOG entries and write the new CHANGELOG file + tmpfile.puts "" + changelog_file = File.join(base_dir, 'CHANGELOG') + tmpfile.write File.read(changelog_file) if File.exist?(changelog_file) + tmpfile.rewind + tmpfile.flush + File.open(changelog_file,"w+") {|f| f.write(tmpfile.read) } +ensure + tmpfile.close + tmpfile.unlink +end http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/release ---------------------------------------------------------------------- diff --git a/dev/release/release b/dev/release/release new file mode 100755 index 0000000..61c294f --- /dev/null +++ b/dev/release/release @@ -0,0 +1,250 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# +# This script is used to publish the official release after a successful +# vote of a release-candidate. + +set -e +set -o nounset + +parquetcpp_git_web_url='https://git-wip-us.apache.org/repos/asf?p=parquet-cpp.git' +parquet_svn_dist_url='https://dist.apache.org/repos/dist/release/parquet' +parquet_svn_dev_dist_url='https://dist.apache.org/repos/dist/dev/parquet' + +function print_help_and_exit { +cat <<EOF +Apache Parquet C++ release tool. + +Usage: $0 [-h] [-r #] [-p | publish] + + -h Print this help message and exit + -r Release candidate number (default: 0) + -p Publish (default: dry-run (does not publish anything)) +EOF +exit 0 +} + +publish=0 +rc_tag_version=0 +while getopts ":hl:r:p" opt; do + case $opt in + r) + rc_tag_version=${OPTARG} + ;; + p) + publish=1 + ;; + h) + print_help_and_exit + ;; + * ) + echo "Unknown option: -$OPTARG" + print_help_and_exit + ;; + esac +done + +shift $(($OPTIND - 1)) +if [[ "${1:-dry-run}" == "publish" ]]; then + publish=1 +fi + +# Update local repository +git fetch --all -q +git fetch --tags -q + +# Ensure that a signing key is available +if [[ -z "`git config user.signingkey`" ]]; then + cat <<EOF +Error: No GPG signing key can be found within gitconfig. + +To configure one, find your code signing key's ID with + + gpg --list-secret-keys + +Then configure it as the signing key for this repository with + + git config --global user.signingkey YOUR_KEY_ID +EOF + exit 1 +fi + +# Set the base dir for the script to be the top level of the repository +base_dir=$(git rev-parse --show-toplevel) +# Verify that this is a clean repository +if [[ -n "`git status --porcelain`" ]]; then + echo "ERROR: Please run from a clean master." + exit 1 +elif [[ "`git rev-parse --abbrev-ref HEAD`" == "master" ]]; then + echo "ERROR: This script must be run from the released branch." + exit 1 +fi + +if [[ "$base_dir" != "$PWD" ]]; then + echo "Warrning: This script must be run from the root of the repository ${base_dir}" + cd $base_dir +fi + +# Make sure that this is not on a snapshot release +current_version=$(cat .parquetcppversion | tr '[a-z]' '[A-Z]') +if [[ $current_version =~ .*-SNAPSHOT ]]; then + echo "ERROR: .parquetcppversion can not be a 'SNAPSHOT', it is ${current_version}" + exit 1 +else + major=`echo $current_version | cut -d. -f1` + minor=`echo $current_version | cut -d. -f2` + patch=`echo $current_version | cut -d. -f3 | cut -d- -f1` + + current_version="${major}.${minor}.${patch}" +fi + +previous_version_tag="${current_version}-rc${rc_tag_version}" +current_version_tag="rel/${current_version}" + +# Make sure the tag does not exist +if [[ $(git ls-remote --exit-code --tags origin refs/tags/${current_version} >/dev/null 2>&1) == 0 ]]; then + echo "ERROR: ${current_version} tag exists." + exit 1 +fi + +# All check are now complete, before we start alert if we are in dry-run +if [[ $publish == 0 ]]; then + echo "Performing dry-run" +fi + +dist_name="apache-parquet-cpp-${current_version}" + +if [[ $publish == 1 ]]; then + # Create release branch and tag and push them to the origin + echo "Creating ${current_version_tag} staging branch" + git checkout -b "stage_${current_version_tag}" + + # Increment the version and create a branch + echo ${current_version} > .parquetcppversion + git add .parquetcppversion + git commit -m "Updating .parquetcppversion to release version ${current_version}." + git tag -s "${current_version_tag}" \ + -m "Apache Parquet C++ ${current_version} release" HEAD + git push origin "${current_version_tag}" +fi + +dist_dir=${base_dir}/dist +rc_dir=${dist_dir}/rc +release_dir=${dist_dir}/${current_version} +mkdir -p ${rc_dir} +cd ${dist_dir} + +# Checkout the release candidate +svn co ${parquet_svn_dev_dist_url}/${previous_version_tag} ${rc_dir} >/dev/null 2>&1 + +if [[ $publish == 1 ]]; then + echo "Publishing the release" + # Make the release dist directory and check it out + svn mkdir ${parquet_svn_dist_url}/${current_version} -m "parquet-cpp-${current_version} release" + svn co --depth=empty ${parquet_svn_dist_url}/${current_version} ${release_dir} +else + mkdir ${release_dir} +fi + +cd ${release_dir} + +# Rename the .parquetcppversion from -RC[:digit:] to release current_version and repackage the release +tar -xzf ${rc_dir}/apache-parquet-cpp-*.tar.gz +mv apache-parquet-cpp-* ${dist_name} +echo ${current_version} > ${dist_name}/.parquetcppversion +tar -czf ${dist_name}.tar.gz ${dist_name} +rm -rf ${dist_name} + +# Sign the tarball. +echo "Signing the distribution" +gpg --armor --output ${dist_name}.tar.gz.asc --detach-sig ${dist_name}.tar.gz + +# Create the checksums +echo "Creating checksums" +gpg --print-md MD5 ${dist_name}.tar.gz > ${dist_name}.tar.gz.md5 +shasum ${dist_name}.tar.gz > ${dist_name}.tar.gz.sha + +if [[ $publish == 1 ]]; then + # Commit the release + svn add ${dist_name}* + svn ci -m "parquet-cpp-${current_version} release" +fi + +cd ${base_dir} + +echo +echo "Done creating the release. The following draft email has been created" +echo "to send to the [email protected] mailing list." +echo + +# Create the email template for the release to be sent to the mailing lists. +MESSAGE=$(cat <<__EOF__ +To: [email protected] +Subject: [RESULT][VOTE] Release Apache Parquet C++ ${current_version} RC${rc_tag_version} + +All, +The vote to accept Apache Parquet C++ ${current_version} RC${rc_tag_version} +as the official Apache Parquet C++ ${current_version} release has passed. + + ++1 (Binding) +------------------------------ + + ++1 (Non-binding) +------------------------------ + + +There were no 0 or -1 votes. Thank you to all who helped make this release. + + +Parquet C++ ${current_version} includes the following: +--- +The CHANGELOG for the release is available at: +${parquetcpp_git_web_url}&f=CHANGELOG&hb=${current_version_tag} + +The tag used to create the release with is ${current_version_tag}: +${parquetcpp_git_web_url}&a=shortlog&h=refs/tags/${current_version_tag} + +The release is available at: +${parquet_svn_dist_url}/${current_version}/${dist_name}.tar.gz + +The MD5 checksum of the release can be found at: +${parquet_svn_dist_url}/${current_version}/${dist_name}.tar.gz.md5 + +The signature of the release can be found at: +${parquet_svn_dist_url}/${current_version}/${dist_name}.tar.gz.asc + +The GPG key used to sign the release are available at: +${parquet_svn_dist_url}/KEYS + +__EOF__ +) +echo "--------------------------------------------------------------------------------" +echo +echo "${MESSAGE}" +echo +echo "--------------------------------------------------------------------------------" +echo + +# Print reset instructions if this was a dry-run +if [[ $publish == 0 ]]; then + echo + echo "This is a dry run, nothing has been published." + echo + echo "To clean up run: rm -rf ${dist_dir}" +fi + +exit 0 http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/release-candidate ---------------------------------------------------------------------- diff --git a/dev/release/release-candidate b/dev/release/release-candidate new file mode 100755 index 0000000..9fe6027 --- /dev/null +++ b/dev/release/release-candidate @@ -0,0 +1,308 @@ +#!/bin/bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# +# This script is used to create a release candidate. It will update the current +# .parquetcppversion as well as creates a tag for the new release candidate and +# publishes the source distribution and signatures to be voted on. +# +# master~1 (0.5.0-snapshot) ----- master (0.6.0-snapshot) +# \---- 0.5.0 (0.5.0) +# +# A email template will be generated after successfully generating a release +# candidate which will need to be sent to the dev@ and private@ mailing lists. +# +set -o errexit +set -o nounset + +rc_number=0 +override_version='' +parquetcpp_git_web_url='https://git-wip-us.apache.org/repos/asf?p=parquet-cpp.git' +parquet_svn_dist_url='https://dist.apache.org/repos/dist/dev/parquet' + +function print_help_and_exit { +cat <<EOF +Apache Parquet C++ release candidate tool. + +Usage: $0 [-h] [-l p|m|M] [-r #] [-p | publish] + + -h Print this help message and exit + -l Increment level, must be one of: + p, patch (default) + m, minor + M, major + -v Override the existing version in .parquetcppversion + -r Release candidate number (default: 0) + -p Publish the release candidate (default: dry-run, does not publish anything) +EOF +exit 0 +} + +publish=0 +increment_level="patch" +rc_number=0 +while getopts ":hl:v:r:p" opt; do + case $opt in + l) + case ${OPTARG} in + 'p' | 'patch') increment_level='patch' ;; + 'm' | 'minor') increment_level='minor' ;; + 'M' | 'major') increment_level='major' ;; + *) echo 'Unknown increment level'; exit 1 ;; + esac + ;; + r) + rc_number=${OPTARG} + ;; + p) + publish=1 + ;; + h) + print_help_and_exit + ;; + v) + override_version=${OPTARG} + ;; + * ) + echo "Unknown option: -$OPTARG" + print_help_and_exit + ;; + esac +done + +shift $(($OPTIND - 1)) +if [[ "${1:-dry-run}" == "publish" ]]; then + publish=1 +fi + +# Update local repository +git fetch --all -q +git fetch --tags -q + +# Verify that this is a clean repository +if [[ -n "`git status --porcelain`" ]]; then + echo "ERROR: Please run from a clean git repository." + exit 1 +elif [[ "`git rev-parse --abbrev-ref HEAD`" != "master" ]]; then + echo "ERROR: This script must be run from master." + exit 1 +fi + +if [[ ! -f .parquetcppversion ]]; then + echo "Warrning: This script must be run from the root of the repository" + exit 1 +fi + +# Calculate the new version string +current_version=$(cat .parquetcppversion | tr '[a-z]' '[A-Z]') +if ! [[ $current_version =~ .*-SNAPSHOT ]]; then + echo "ERROR: .parquetcppversion is required to contain 'SNAPSHOT', it is ${current_version}" + exit 1 +else + if [[ $override_version != "" ]]; then + current_version=$override_version + fi + + major=`echo $current_version | cut -d. -f1` + minor=`echo $current_version | cut -d. -f2` + patch=`echo $current_version | cut -d. -f3 | cut -d- -f1` + + current_version="${major}.${minor}.${patch}" + + if [[ $increment_level == "patch" ]]; then + new_master_version="${major}.${minor}.$((patch + 1))" + elif [[ $increment_level == "minor" ]]; then + new_master_version="${major}.$((minor + 1)).0" + elif [[ $increment_level == "major" ]]; then + new_master_version="$((major + 1)).0.0" + else + echo "Unknown release increment ${increment_level}" + exit 1 + fi + + new_snapshot_version="${new_master_version}-SNAPSHOT" +fi + +# Add the rc tag to the current version +rc_version="${current_version}-rc${rc_number}" +rc_version_tag="rel/${rc_version}" + +echo +echo "Generating release candidate ${rc_version}" +echo + +# Make sure the tag does not exist +if git rev-parse ${rc_version_tag} >/dev/null 2>&1; then + echo "ERROR: tag ${rc_version_tag} exists." + exit 1 +fi + +# Reset instructions +current_git_rev=$(git rev-parse HEAD) +function print_reset_instructions { +cat <<EOF +To roll back your local repo you will need to run: + + git checkout master + git reset --hard ${current_git_rev} + git tag -D ${rc_version_tag} +EOF +} + +# If anything goes wrong from here then print roll back instructions before exiting. +function print_rollback_instructions { + echo "ERROR: Looks like something has failed while creating the release candidate." + print_reset_instructions +} +trap print_rollback_instructions EXIT + +# All check are now complete, before we start alert if we are in dry-run +if [[ $publish == 0 ]]; then + echo "Performing dry-run, run with '-p' when you are ready to run and publish a release candidate" +fi + +# This should be a clean repo we are working against. Run clean just to ensure it is. +git clean -fdxq + +echo "Generating changelog" +./dev/release/changelog $current_version +git add CHANGELOG +git commit -m "Updating CHANGELOG for ${current_version} release." + +echo "Committing updated .parquetcppversion on master" +echo $new_snapshot_version > .parquetcppversion +git add .parquetcppversion +git commit -m "Incrementing snapshot version to ${new_snapshot_version}." + +echo "Creating ${rc_version} staging branch" +git checkout -b "stage_${rc_version_tag}" + +echo "Updating .parquetcppversion on staging branch" +# Increment the version and create a branch +echo ${rc_version} > .parquetcppversion +git add .parquetcppversion +git commit -m "Updating .parquetcppversion to ${rc_version}." + +# Build the source distribution from the new branch +echo "Building the source distribution" +dist_dir=dist +dist_name="apache-parquet-cpp-${rc_version}" + +mkdir -p ${dist_dir} +git archive --prefix=${dist_name}/ -o ${dist_dir}/${dist_name}.tar.gz HEAD +pushd ${dist_dir} + # Sign the tarball. + echo "Signing the distribution" + gpg --armor --output ${dist_name}.tar.gz.asc --detach-sig ${dist_name}.tar.gz + + # Create the checksums + echo "Creating checksums" + gpg --print-md MD5 ${dist_name}.tar.gz > ${dist_name}.tar.gz.md5 + shasum ${dist_name}.tar.gz > ${dist_name}.tar.gz.sha +popd + +parquet_svn_rc_url="${parquet_svn_dist_url}/${rc_version}" + +# Publish release candidate to svn and commit and push the new git tag +if [[ $publish == 1 ]]; then + echo "Publishing release candidate to ${parquet_svn_rc_url}" + svn mkdir ${parquet_svn_rc_url} -m "parquet-cpp-${current_version} release candidate ${rc_version_tag}" + svn co --depth=empty ${parquet_svn_rc_url} ${dist_dir} + pushd ${dist_dir} + svn add ${dist_name}* + svn ci -m "parquet-cpp-${current_version} release candidate ${rc_version_tag}" + popd + + echo "Creating tag ${rc_version_tag}" + git tag -s ${rc_version_tag} \ + -m "Apache Parquet C++ ${current_version} release candidate ${rc_version_tag}" + git push origin "${rc_version_tag}" + + echo "Pushing updated .parquetcppversion to master" + git checkout master + git push origin master +fi + +echo "Done creating the release candidate. The following draft email has been created" +echo "to send to the [email protected] mailing list" +echo + +# Create the email template for the release candidate to be sent to the mailing lists. +if [[ $(uname) == Darwin ]]; then + vote_end=$(date -v+3d) +else + vote_end=$(date -d+3days) +fi + +MESSAGE=$(cat <<__EOF__ +To: [email protected] +Subject: [VOTE] Release Apache Parquet C++ ${current_version} RC${rc_number} + +All, + +I propose that we accept the following release candidate as the official +Apache Parquet C++ ${current_version} release. + +Parquet C++ ${rc_version} includes the following: +--- +The RELEASE NOTES for the release are available at: +${parquetcpp_git_web_url}&f=RELEASE-NOTES.md&hb=${rc_version_tag} + +The CHANGELOG for the release is available at: +${parquetcpp_git_web_url}&f=CHANGELOG&hb=${rc_version_tag} + +The tag used to create the release candidate is: +${parquetcpp_git_web_url};a=shortlog;h=refs/tags/${rc_version_tag} + +The release candidate is available at: +${parquet_svn_rc_url}/${dist_name}.tar.gz + +The MD5 checksum of the release candidate can be found at: +${parquet_svn_rc_url}/${dist_name}.tar.gz.md5 + +The signature of the release candidate can be found at: +${parquet_svn_rc_url}/${dist_name}.tar.gz.asc + +The GPG key used to sign the release are available at: +${parquet_svn_dist_url}/KEYS + +Please download, verify, and test. + +The vote will close on ${vote_end} + +[ ] +1 Release this as Apache Parquet C++ ${current_version} +[ ] +0 +[ ] -1 Do not release this as Apache Parquet C++ ${current_version} because... + +__EOF__ +) + +echo "--------------------------------------------------------------------------------" +echo +echo "${MESSAGE}" +echo +echo "--------------------------------------------------------------------------------" +echo + +# Print reset instructions if this was a dry-run +if [[ $publish == 0 ]]; then + echo + echo "This was a dry run, nothing has been published." + echo + print_reset_instructions +fi + +# Unset error message handler and exit +trap '' EXIT +exit 0 http://git-wip-us.apache.org/repos/asf/parquet-cpp/blob/a1517582/dev/release/verify-release-candidate ---------------------------------------------------------------------- diff --git a/dev/release/verify-release-candidate b/dev/release/verify-release-candidate new file mode 100755 index 0000000..efcc111 --- /dev/null +++ b/dev/release/verify-release-candidate @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# +# Downloads a release candidate and verifies that it passes binary +# verification (signature and checksums) and test suites. +# +set -ex + +HERE=$(cd `dirname "${BASH_SOURCE[0]:-$0}"` && pwd) + +parquet_svn_dev_dist_url='https://dist.apache.org/repos/dist/dev/parquet' + +download_dist_file() { + curl -f -O ${parquet_svn_dev_dist_url}/$1 +} + +download_rc_file() { + download_dist_file ${verify_version}/$1 +} + +import_gpg_keys() { + download_dist_file KEYS + gpg --import KEYS +} + +fetch_archive() { + local dist_name=$1 + download_rc_file ${dist_name}.tar.gz + download_rc_file ${dist_name}.tar.gz.asc + download_rc_file ${dist_name}.tar.gz.md5 + download_rc_file ${dist_name}.tar.gz.sha + gpg --verify ${dist_name}.tar.gz.asc ${dist_name}.tar.gz + gpg --print-md MD5 ${dist_name}.tar.gz | diff - ${dist_name}.tar.gz.md5 + shasum ${dist_name}.tar.gz | diff - ${dist_name}.tar.gz.sha +} + +run_tests() { + # Build + source ./setup_build_env.sh + mkdir -p build + cd build + cmake -DPARQUET_ARROW=ON .. + make -j5 + + # Test + export PARQUET_TEST_DATA=$(dirname $(pwd))/data + make unittest +} + +setup_tempdir() { + cleanup() { + rm -fr "$TMPDIR" + } + trap cleanup EXIT + TMPDIR=$(mktemp -d -t "$1.XXXXX") +} + +case $# in + 1) verify_version="$1" + ;; + + *) echo "Usage: $0 RC_VERSION" + exit 1 + ;; +esac + +setup_tempdir "parquet-cpp-$verify_version" +echo "Working in sandbox $TMPDIR" +cd $TMPDIR + +import_gpg_keys + +dist_name="apache-parquet-cpp-${verify_version}" +fetch_archive $dist_name +tar xvzf ${dist_name}.tar.gz +cd ${dist_name} +run_tests + +echo 'Release candidate looks good!' +exit 0
