ashb commented on a change in pull request #14690: URL: https://github.com/apache/airflow/pull/14690#discussion_r594513656
########## File path: dev/README_RELEASE_AIRFLOW_UPGRADE_CHECK.md ########## @@ -0,0 +1,642 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> +<!-- START doctoc generated TOC please keep comment here to allow auto update --> +<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> +**Table of contents** + +- [Prepare the Apache Airflow Upgrade Check Package RC](#prepare-the-apache-airflow-upgrade-check-package-rc) + - [Pre-requisites](#pre-requisites) + - [Build RC artifacts](#build-rc-artifacts) + - [Prepare PyPI convenience "snapshot" packages](#prepare-pypi-convenience-snapshot-packages) + - [Prepare Vote email on the Apache Airflow release candidate](#prepare-vote-email-on-the-apache-airflow-release-candidate) +- [Verify the release candidate by PMCs](#verify-the-release-candidate-by-pmcs) + - [SVN check](#svn-check) + - [Licence check](#licence-check) + - [Signature check](#signature-check) + - [SHA512 sum check](#sha512-sum-check) +- [Verify release candidates by Contributors](#verify-release-candidates-by-contributors) +- [Publish the final release](#publish-the-final-release) + - [Summarize the voting for the release](#summarize-the-voting-for-the-release) + - [Publish release to SVN](#publish-release-to-svn) + - [Prepare PyPI "release" packages](#prepare-pypi-release-packages) + - [Notify developers of release](#notify-developers-of-release) + - [Update Announcements page](#update-announcements-page) + +<!-- END doctoc generated TOC please keep comment here to allow auto update --> + +You can find the prerequisites to release Apache Airflow in [README.md](README.md). This document +details the steps for releasing apache-airflow-upgrade-check package. + +# Prepare the Apache Airflow Upgrade Check Package RC + +## Pre-requisites + +- Install `setuptools-build-subpackage` v0.1.0 by running: + + ```shell + pip install https://github.com/ashb/setuptools-build-subpackage/archive/master.zip + ``` + +## Build RC artifacts + +The Release Candidate artifacts we vote upon should be the exact ones we vote against, +without any modification than renaming – i.e. the contents of the files must be +the same between voted release candidate and final release. +Because of this the version in the built artifacts that will become the +official Apache releases must not include the rcN suffix. + +- Set environment variables + + ```shell script + # Set Version + export VERSION=1.3.0rc1 + + + # Set AIRFLOW_REPO_ROOT to the path of your git repo + export AIRFLOW_REPO_ROOT=$(pwd) + + + # Example after cloning + git clone https://github.com/apache/airflow.git airflow + cd airflow + export AIRFLOW_REPO_ROOT=$(pwd) + ``` + +- Checkout `v1-10-stable` branch: + + ```shell + git checkout v1-10-stable + ``` + +- Set your version to 1.3.0 in `airflow/upgrade/version.py` (without the RC tag) +- Commit the version change. + +- Tag your release + + ```shell script + git tag -s upgrade-check/${VERSION} + ``` + +- Clean the checkout: the sdist step below will + + ```shell script + git clean -fxd + ``` + +- Tarball the repo + + ```shell script + git archive upgrade-check/${VERSION} --prefix=apache-airflow-upgrade-check-${VERSION%rc?}/ \ + -o apache-airflow-upgrade-check-${VERSION}-source.tar.gz \ + airflow/upgrade NOTICE LICENSE licenses/ + ``` + + +- Generate sdist + + NOTE: Make sure your checkout is clean at this stage - any untracked or changed files will otherwise be included + in the file produced. + + ```shell script + python -m setuptools_build_subpackage --subpackage-folder airflow/upgrade \ + egg_info sdist --license-template ./LICENSE bdist_wheel + ``` + +- Rename the sdist + + ```shell script + mv dist/apache-airflow-upgrade-check-${VERSION%rc?}.tar.gz apache-airflow-upgrade-check-${VERSION}-bin.tar.gz + mv dist/apache_airflow_upgrade_check-${VERSION%rc?}-py2.py3-none-any.whl apache_airflow_upgrade_check-${VERSION}-py2.py3-none-any.whl + ``` + +- Generate SHA512/ASC (If you have not generated a key yet, generate it by following instructions on + http://www.apache.org/dev/openpgp.html#key-gen-generate-key) + + ```shell script + ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-upgrade-check-${VERSION}-source.tar.gz + ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache-airflow-upgrade-check-${VERSION}-bin.tar.gz + ${AIRFLOW_REPO_ROOT}/dev/sign.sh apache_airflow_upgrade_check-${VERSION}-py2.py3-none-any.whl + ``` + +- Push the artifacts to ASF dev dist repo + + ```shell + # First clone the repo + svn checkout https://dist.apache.org/repos/dist/dev/airflow airflow-dev + + # Create new folder for the release + cd airflow-dev/upgrade-check + svn mkdir ${VERSION} + + # Move the artifacts to svn folder & commit + mv ${AIRFLOW_REPO_ROOT}/apache{-,_}airflow*-${VERSION}* ${VERSION}/ + cd ${VERSION} + svn add * + svn commit -m "Add artifacts for Upgrade Check ${VERSION}" + ``` + +## Prepare PyPI convenience "snapshot" packages + +At this point we have the artefact that we vote on, but as a convenience to developers we also want to +publish "snapshots" of the RC builds to pypi for installing via pip. Also those packages +are used to build the production docker image in DockerHub, so we need to upload the packages +before we push the tag to GitHub. Pushing the tag to GitHub automatically triggers image building in +DockerHub. + +To do this we need to + +- Build the package: + + ```shell script + cd ${AIRFLOW_REPO_ROOT} + python -m setuptools_build_subpackage --subpackage-folder airflow/upgrade \ + egg_info --tag-build "$(sed -e "s/^[0-9.]*//" <<<"$VERSION")" sdist \ + --license-template ./LICENSE bdist_wheel + ``` + +- Verify the artifacts that would be uploaded: + + ```shell script + twine check dist/* + ``` + +- Upload the package to PyPi's test environment: + + ```shell script + twine upload -r pypitest dist/* + ``` + +- Verify that the test package looks good by downloading it and installing it into a virtual environment. + The package download link is available at: https://test.pypi.org/project/apache-airflow-upgrade-check/#history + +- Upload the package to PyPi's production environment: + + ```shell Review comment: ```suggestion ```shell script ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
