This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion-python.git
The following commit(s) were added to refs/heads/main by this push:
new e263eb7 Implement new release process (#149)
e263eb7 is described below
commit e263eb7c68fc258f59ce5e34439a3f80ecda3587
Author: Andy Grove <[email protected]>
AuthorDate: Sat Feb 25 07:58:22 2023 -0700
Implement new release process (#149)
* document the new release process
* Update release notes
---
.github/workflows/build.yml | 2 +-
dev/release/README.md | 75 +++++++++++++++++++++++++++++----------------
2 files changed, 50 insertions(+), 27 deletions(-)
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index 54fdcfe..b9b382f 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -21,7 +21,7 @@ on:
branches: ["main"]
push:
tags: ["*-rc*"]
- branches: ["main"]
+ branches: ["branch-*"]
jobs:
generate-license:
diff --git a/dev/release/README.md b/dev/release/README.md
index aa2a10e..aed0e29 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -19,22 +19,51 @@
# DataFusion Python Release Process
-## Update Version
+Development happens on the `main` branch, and most of the time, we depend on
DataFusion using GitHub dependencies
+rather than using an official release from crates.io. This allows us to pick
up new features and bug fixes frequently
+by creating PRs to move to a later revision of the code. It also means we can
incrementally make updates that are
+required due to changes in DataFusion rather than having a large amount of
work to do when the next official release
+is available.
-The version number in Cargo.toml should be increased, according to semver.
+When there is a new official release of DataFusion, we update the `main`
branch to point to that, update the version
+number, and create a new release branch, such as `branch-0.8`. Once this
branch is created, we switch the `main` branch
+back to using GitHub dependencies. The release activity (such as generating
the changelog) can then happen on the
+release branch without blocking ongoing development in the `main` branch.
-## Update CHANGELOG.md
+We can cherry-pick commits from the `main` branch into `branch-0.8` as needed
and then create new patch releases
+from that branch.
-Define release branch (e.g. `main`), base version tag (e.g. `0.6.0`) and
future version tag (e.g. `0.7.0`). Commits
-between the base version tag and the release branch will be used to populate
the changelog content.
+## Detailed Guide
+
+### Pre-requisites
-You will need a GitHub Personal Access Token for the following steps. Follow
+Releases can currently only be created by PMC members due to the permissions
needed.
+
+You will need a GitHub Personal Access Token. Follow
[these
instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
to generate one if you do not already have one.
+You will need a PyPI API token. Create one at
https://test.pypi.org/manage/account/#api-tokens, setting the “Scope” to
+“Entire account”.
+
+You will also need access to the
[datafusion](https://test.pypi.org/project/datafusion/) project on testpypi.
+
+### Preparing the `main` Branch
+
+Before creating a new release:
+
+- We need to ensure that the main branch does not have any GitHub dependencies
+- a PR should be created and merged to update the major version number of the
project
+- A new release branch should be created, such as `branch-0.8`
+
+### Update CHANGELOG.md
+
+Define release branch (e.g. `branch-0.8`), base version tag (e.g. `0.7.0`) and
future version tag (e.g. `0.9.0`). Commits
+between the base version tag and the release branch will be used to populate
the changelog content.
+
```bash
# create the changelog
-CHANGELOG_GITHUB_TOKEN=<TOKEN>
./dev/release/update_change_log-datafusion-python.sh main 0.7.0 0.6.0
+CHANGELOG_GITHUB_TOKEN=<TOKEN>
./dev/release/update_change_log-datafusion-python.sh main 0.8.0 0.7.0
# review change log / edit issues and labels if needed, rerun until you are
happy with the result
git commit -a -m 'Create changelog for release'
```
@@ -46,24 +75,23 @@ value of the `--cpus` argument in the
`update_change_log.sh` script._
You can add `invalid` or `development-process` label to exclude items from
release notes.
-Send a PR to get these changes merged into `main` branch. If new commits that
-could change the change log content landed in the `main` branch before you
-could merge the PR, you need to rerun the changelog update script to regenerate
-the changelog and update the PR accordingly.
+Send a PR to get these changes merged into the release branch (e.g.
`branch-0.8`). If new commits that could change the
+change log content landed in the release branch before you could merge the PR,
you need to rerun the changelog update
+script to regenerate the changelog and update the PR accordingly.
-## Preparing a Release Candidate
+### Preparing a Release Candidate
### Tag the Repository
```bash
-git tag 0.7.0-rc1
-git push apache 0.7.0-rc1
+git tag 0.8.0-rc1
+git push apache 0.8.0-rc1
```
### Create a source release
```bash
-./dev/release/create_tarball 0.7.0 1
+./dev/release/create_tarball 0.8.0 1
```
This will also create the email template to send to the mailing list. Here is
an example:
@@ -107,17 +135,12 @@ Create a draft email using this content, but do not send
until after completing
### Publish Python Artifacts to testpypi
-To securely upload your project, you’ll need a PyPI API token. Create one at
-https://test.pypi.org/manage/account/#api-tokens, setting the “Scope” to
“Entire account”.
-
-You will also need access to the
[datafusion](https://test.pypi.org/project/datafusion/) project on testpypi.
-
-This section assumes some familiary with publishing Python packages to PyPi.
For more information, refer to \
+This section assumes some familiarity with publishing Python packages to PyPi.
For more information, refer to \
[this
tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/#uploading-the-distribution-archives).
#### Publish Python Wheels to testpypi
-Pushing an `rc` tag to main will cause a GitHub Workflow to run that will
build the Python wheels.
+Pushing an `rc` tag to the release branch will cause a GitHub Workflow to run
that will build the Python wheels.
Go to https://github.com/apache/arrow-datafusion-python/actions and look for
an action named "Python Release Build"
that has run against the pushed tag.
@@ -195,7 +218,7 @@ Once the vote passes, we can publish the release.
Create the source release tarball:
```bash
-./dev/release/release-tarball.sh 0.7.0 1
+./dev/release/release-tarball.sh 0.8.0 1
```
### Publishing Python Artifacts
@@ -211,7 +234,7 @@ twine upload --repository pypi dist-release/*
### Push the Release Tag
```bash
-git checkout 0.7.0-rc1
-git tag 0.7.0
-git push apache 0.7.0
+git checkout 0.8.0-rc1
+git tag 0.8.0
+git push apache 0.8.0
```
