potiuk commented on a change in pull request #11310: URL: https://github.com/apache/airflow/pull/11310#discussion_r501536269
########## File path: dev/README.md ########## @@ -575,8 +706,645 @@ https://airflow.apache.org/changelog.html#airflow-1-10-2-2019-01-19 Cheers, <your name> -</p> -</details> +EOF +``` + +### Update Announcements page + +Update "Announcements" page at the [Official Airflow website](https://airflow.apache.org/announcements/) + + +----------------------------------------------------------------------------------------------------------- + + +# Backport Provider Packages + +You can read more about the command line tools used to generate backport packages in +[Backport Providers README](../backport_packages/README.md). + +## Decide when to release + +You can release backport packages separately on an ad-hoc basis, whenever we find that a given provider needs +to be released - due to new features or due to bug fixes. You can release each backport package +separately. + +We are using the [CALVER](https://calver.org/) versioning scheme for the backport packages. We also have an +automated way to prepare and build the packages, so it should be very easy to release the packages often and +separately. + +## Prepare the Backport Provider Packages RC + +### Generate release notes + +Prepare release notes for all the packages you plan to release. Where YYYY.MM.DD is the CALVER +date for the packages. + +``` +./breeze prepare-backport-readme YYYY.MM.DD [packages] +``` + +If you iterate with merges and release candidates you can update the release date without providing +the date (to update the existing release notes) + +``` +./breeze prepare-backport-readme google +``` + +Generated readme files should be eventually committed to the repository. + +### Build an RC release for SVN apache upload + +The Release Candidate artifacts we vote upon should be the exact ones we vote against, without any +modification than renaming i.e. the contents of the files must be the same between voted +release candidate and final release. Because of this the version in the built artifacts +that will become the official Apache releases must not include the rcN suffix. They also need +to be signed and have checksum files. You can generate the checksum/signature files by running +the "dev/sign.sh" script (assuming you have the right PGP key set-up for signing). The script +generates corresponding .asc and .sha512 files for each file to sign. + +#### Build and sign the source and convenience packages + +* Set environment variables (version and root of airflow repo) + +```shell script +export VERSION=2020.5.20rc2 +export AIRFLOW_REPO_ROOT=$(pwd) + +``` + +* Build the source package: + +``` +./backport_packages/build_source_package.sh + +``` + +It will generate `apache-airflow-backport-providers-${VERSION}-source.tar.gz` + +* Generate the packages - since we are preparing packages for SVN repo, we should use the right switch. Note + that this will clean up dist folder before generating the packages, so it will only contain the packages + you intended to build. + +```shell script +./breeze prepare-backport-packages --version-suffix-for-svn rc1 +``` + +if you ony build few packages, run: + +```shell script +./breeze prepare-backport-packages --version-suffix-for-svn rc1 PACKAGE PACKAGE .... +``` + +* Move the source tarball to dist folder + +```shell script +mv apache-airflow-backport-providers-${VERSION}-source.tar.gz dist +``` + +* Sign all your packages + +```shell script +pushd dist +../dev/sign.sh * +popd +``` + +* Push tags to Apache repository (assuming that you have apache remote pointing to apache/airflow repo)] + +```shell script +git push apache backport-providers-${VERSION} +``` + +#### Commit the source packages to Apache SVN repo + +* Push the artifacts to ASF dev dist repo + +```shell script +# First clone the repo if you do not have it +svn checkout https://dist.apache.org/repos/dist/dev/airflow airflow-dev + +# update the repo in case you have it already +cd airflow-dev +svn update + +# Create a new folder for the release. +cd airflow-dev/backport-providers +svn mkdir ${VERSION} + +# Move the artifacts to svn folder +mv ${AIRFLOW_REPO_ROOT}/dist/* ${VERSION}/ + +# Add and commit +svn add ${VERSION}/* +svn commit -m "Add artifacts for Airflow ${VERSION}" + +cd ${AIRFLOW_REPO_ROOT} +``` + +Verify that the files are available at +[backport-providers](https://dist.apache.org/repos/dist/dev/airflow/backport-providers/) + +### Publish the RC convenience package to PyPI + +In order to publish to PyPI you just need to build and release packages. The packages should however +contain the rcN suffix in the version name as well, so you need to use `--version-suffix-for-pypi` switch +to prepare those packages. Note that these are different packages than the ones used for SVN upload +though they should be generated from the same sources. + +* Generate the packages with the right RC version (specify the version suffix with PyPI switch). Note that +this will clean up dist folder before generating the packages, so you will only have the right packages there. + +```shell script +./breeze prepare-backport-packages --version-suffix-for-pypi rc1 +``` + +if you ony build few packages, run: + +```shell script +./breeze prepare-backport-packages --version-suffix-for-pypi rc1 PACKAGE PACKAGE .... +``` + +* Verify the artifacts that would be uploaded: + +```shell script +twine check dist/* +``` + +* Upload the package to PyPi's test environment: + +```shell script +twine upload -r pypitest dist/* +``` + +* Verify that the test packages look good by downloading it and installing them into a virtual environment. +Twine prints the package links as output - separately for each package. + +* Upload the package to PyPi's production environment: + +```shell script +twine upload -r pypi dist/* +``` + +* Copy the list of links to the uploaded packages - they will be useful in preparing VOTE email. + +* Again, confirm that the packages are available under the links printed. + +## Vote and verify the Backport Providers release candidate + +### Prepare voting email for Backport Providers release candidate + +Make sure the packages are in https://dist.apache.org/repos/dist/dev/airflow/backport-providers/ + +Send out a vote to the [email protected] mailing list. Here you can prepare text of the +email using the ${VERSION} variable you already set in the command line. + +subject: + + +```shell script +cat <<EOF +[VOTE] Airflow Backport Providers ${VERSION} +EOF +``` + +```shell script +cat <<EOF +Hey all, + +I have cut Airflow Backport Providers ${VERSION}. This email is calling a vote on the release, +which will last for 72 hours - which means that it will end on $(date -d '+3 days'). + +Consider this my (binding) +1. + +Airflow Backport Providers ${VERSION} are available at: +https://dist.apache.org/repos/dist/dev/airflow/backport-providers/${VERSION}/ + +*apache-airflow-backport-providers-${VERSION}-source.tar.gz* is a source release that comes + with INSTALL instructions. + +*apache-airflow-backport-providers-<PROVIDER>-${VERSION}-bin.tar.gz* are the binary + Python "sdist" release. + +The test procedure for PMCs and Contributors who would like to test the RC candidates are described in +https://github.com/apache/airflow/blob/master/dev/README.md#vote-and-verify-the-backport-providers-release-candidate Review comment: @ashb @kaxil @mik-laj I've added links to instructions on how to test the RC in the email template. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
