This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new 2dd17b9e6c chore: Add script to protect RC branches during the release
(#18660)
2dd17b9e6c is described below
commit 2dd17b9e6c0f2d55f6ad0a188709e48a5426b155
Author: Oleks V <[email protected]>
AuthorDate: Tue Nov 18 17:01:53 2025 -0800
chore: Add script to protect RC branches during the release (#18660)
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax. For example
`Closes #123` indicates that this PR will close issue #123.
-->
- Closes https://github.com/apache/datafusion/issues/17134
## Rationale for this change
ASF Infra works in tricky way, to apply a protection for the new branch
its needed to get:
- branch created in repo
- add rules to `.asf.yaml` in `main`
- tricky part ASF reapply policies ONLY if `.asf.yaml` changed in PR,
means that "future" branches won't be covered until the file changed and
pushed.
Adding a script to add automatically a block with new RC branch
protection rules and updated documentation
<!--
Why are you proposing this change? If this is already explained clearly
in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand
your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it
is sometimes worth providing a summary of the individual changes in this
PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example,
are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
---------
Co-authored-by: Martin Grigorov <[email protected]>
---
.asf.yaml | 62 +-------------
dev/release/README.md | 48 ++++++-----
dev/release/add-branch-protection.sh | 160 +++++++++++++++++++++++++++++++++++
3 files changed, 190 insertions(+), 80 deletions(-)
diff --git a/.asf.yaml b/.asf.yaml
index 2f77331adf..0588a300a5 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -54,72 +54,14 @@ github:
# needs to be updated as part of the release process
# .asf.yaml doesn't support wildcard branch protection rules, only exact
branch names
#
https://github.com/apache/infrastructure-asfyaml?tab=readme-ov-file#branch-protection
- # Keeping set of protected branches for future releases
- # Meanwhile creating a prerelease script that will update the branch
protection names
- # automatically. Keep track on it
https://github.com/apache/datafusion/issues/17134
+ # these branches protection blocks autogenerated during release process
which is described in
+ #
https://github.com/apache/datafusion/tree/main/dev/release#2-add-a-protection-to-release-candidate-branch
branch-50:
required_pull_request_reviews:
required_approving_review_count: 1
branch-51:
required_pull_request_reviews:
required_approving_review_count: 1
-# branch-52:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-53:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-54:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-55:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-56:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-57:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-58:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-59:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-60:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-61:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-62:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-63:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-64:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-65:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-66:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-67:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-68:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-69:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-70:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
pull_requests:
# enable updating head branches of pull requests
allow_update_branch: true
diff --git a/dev/release/README.md b/dev/release/README.md
index d9bab521fa..898bceb6f7 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -26,11 +26,11 @@ Patch releases are made on an adhoc basis, but we try and
avoid them given the f
## Release Process Overview
New development happens on the `main` branch.
-Releases are made from branches, e.g. `branch-37` for the `37.x.y` release
series.
+Releases are made from branches, e.g. `branch-50` for the `50.x.y` release
series.
To prepare for a new release series, we:
-- Create a new branch from `main`, such as `branch-37` in the Apache
repository (not in a fork)
+- Create a new branch from `main`, such as `branch-50` in the Apache
repository (not in a fork)
- Continue merging new features changes to `main` branch
- Prepare the release branch for release:
- Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`
@@ -55,7 +55,7 @@ these steps:
1. Find (or create) the issue for the incremental release ([example release
issue]) and discuss the proposed change there with the maintainers.
2. Follow normal workflow to create PR to `main` branch and wait for its
approval and merge.
-3. After PR is squash merged to `main`, branch from most recent release branch
(e.g. `branch-37`), cherry-pick the commit and create a PR targeting the
release branch [example backport PR].
+3. After PR is squash merged to `main`, branch from most recent release branch
(e.g. `branch-50`), cherry-pick the commit and create a PR targeting the
release branch [example backport PR].
For example, to backport commit `12345` from `main` to `branch-50`:
@@ -127,16 +127,35 @@ We then publish the code in the approved artifacts to
crates.io.
First create a new release branch from `main` in the apache repository.
-For example, to create the `branch-51` branch for the `51.x.y` release series:
+For example, to create the `branch-50` branch for the `50.x.y` release series:
```shell
git fetch apache # make sure we are up to date
git checkout apache/main # checkout current latest development branch
-git checkout -b branch-51 # create local branch
-git push -u apache branch-51 # push branch to apache remote
+git checkout -b branch-50 # create local branch
+git push -u apache branch-50 # push branch to apache remote
```
-### 2. Prepare PR to Update Changelog and the Release Version
+### 2. Add a protection to release candidate branch
+
+To protect a release candidate branch from accidental merges, run:
+
+```shell
+./dev/release/add-branch-protection.sh 50
+```
+
+The script will modify `.asf.yaml` and add following block:
+
+```yaml
+branch-50:
+ required_pull_request_reviews:
+ required_approving_review_count: 1
+```
+
+- Create a PR.
+- Merge to `main`.
+
+### 3. Prepare PR to Update Changelog and the Release Version
First, prepare a PR to update the changelog and versions to reflect the planned
release. See [#18173](https://github.com/apache/datafusion/pull/18173) for an
example.
@@ -202,17 +221,6 @@ git commit -a -m 'Update version'
Remember to merge any fixes back to `main` branch as well.
-### 3. Prepare a PR to Modify `asf.yaml`
-
-Modify `asf.yaml` to protect future release candidate branch to prevent
accidental merges:
-
-```yaml
-# needs to be updated as part of the release process
-branch-50:
- required_pull_request_reviews:
- required_approving_review_count: 1
-```
-
### 4. Prepare Release Candidate Artifacts
After the PR gets merged, you are ready to create release artifacts based off
the
@@ -425,7 +433,7 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusion
Delete a release candidate:
```shell
-svn delete -m "delete old DataFusion RC"
https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-38.0.0-rc1/
+svn delete -m "delete old DataFusion RC"
https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/
```
#### Deleting old releases from `release` svn
@@ -441,5 +449,5 @@ svn ls https://dist.apache.org/repos/dist/release/datafusion
Delete a release:
```shell
-svn delete -m "delete old DataFusion release"
https://dist.apache.org/repos/dist/release/datafusion/datafusion-37.0.0
+svn delete -m "delete old DataFusion release"
https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0
```
diff --git a/dev/release/add-branch-protection.sh
b/dev/release/add-branch-protection.sh
new file mode 100755
index 0000000000..735bae7f90
--- /dev/null
+++ b/dev/release/add-branch-protection.sh
@@ -0,0 +1,160 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eu
+
+# Script to add branch protection for a new release branch in .asf.yaml
+#
+# This script automates the process of adding branch protection rules to
.asf.yaml
+# for new release branches. It ensures the branch protection block doesn't
already
+# exist before adding it.
+#
+# Usage:
+# ./dev/release/add-branch-protection.sh <release_number>
+#
+# Examples:
+# ./dev/release/add-branch-protection.sh 52
+# ./dev/release/add-branch-protection.sh 53
+#
+# The script will:
+# 1. Validate the release number is a positive integer
+# 2. Check if branch protection already exists for branch-<release_number>
+# 3. Add the branch protection block to .asf.yaml if it doesn't exist
+# 4. Error out if the block already exists
+
+# Check if release number is provided
+if [ $# -eq 0 ]; then
+ echo "Error: Release number is required"
+ echo "Usage: $0 <release_number>"
+ echo "Example: $0 52"
+ exit 1
+fi
+
+RELEASE_NUM=$1
+BRANCH_NAME="branch-${RELEASE_NUM}"
+ASF_YAML_FILE=".asf.yaml"
+
+# Validate release number is a positive integer
+if ! [[ "$RELEASE_NUM" =~ ^[0-9]+$ ]]; then
+ echo "Error: Release number must be a positive integer"
+ echo "Provided: $RELEASE_NUM"
+ echo "Example: ./dev/release/add-branch-protection.sh 52"
+ exit 1
+fi
+
+# Check if .asf.yaml exists
+if [ ! -f "$ASF_YAML_FILE" ]; then
+ echo "Error: $ASF_YAML_FILE not found in current directory"
+ echo "Please run this script from the repository root"
+ exit 1
+fi
+
+# Check if the branch exists in the official Apache DataFusion repository
+GITHUB_API_URL="https://api.github.com/repos/apache/datafusion/branches/${BRANCH_NAME}";
+HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$GITHUB_API_URL")
+
+if [ "$HTTP_STATUS" != "200" ]; then
+ echo "Error: Branch ${BRANCH_NAME} does not exist in the official Apache
DataFusion repository"
+ echo "Please create the branch '${BRANCH_NAME}' first before adding branch
protection"
+ echo ""
+ echo "To check existing branches, visit:"
+ echo " https://github.com/apache/datafusion/branches";
+ exit 1
+fi
+
+# Check if branch protection already exists for this release
+if grep -q "^[[:space:]]*${BRANCH_NAME}:" "$ASF_YAML_FILE"; then
+ echo "Error: Branch protection for ${BRANCH_NAME} already exists in
$ASF_YAML_FILE"
+ exit 1
+fi
+
+# Create a temporary file
+TEMP_FILE=$(mktemp)
+
+# Read the file and insert the new branch protection block
+# We'll insert it after the last branch-XX block
+awk -v branch="$BRANCH_NAME" '
+/^[[:space:]]*branch-[0-9]+:/ {
+ last_branch_line = NR
+ last_branch_content = $0
+}
+{
+ lines[NR] = $0
+}
+END {
+ if (last_branch_line == 0) {
+ print "Error: No existing branch protection blocks found" >
"/dev/stderr"
+ exit 1
+ }
+
+ # Print all lines up to and including the last branch block
+ for (i = 1; i <= last_branch_line; i++) {
+ print lines[i]
+ }
+
+ # Print the required_pull_request_reviews lines after the last branch
+ for (i = last_branch_line + 1; i <= NR; i++) {
+ print lines[i]
+ # After printing the required_approving_review_count line, insert new
branch
+ if (lines[i] ~ /required_approving_review_count:/) {
+ # Check if this belongs to the last branch block by looking ahead
+ next_non_empty = i + 1
+ while (next_non_empty <= NR && lines[next_non_empty] ~
/^[[:space:]]*$/) {
+ next_non_empty++
+ }
+ # If next non-empty line is not indented more than branch level,
we found the end
+ if (next_non_empty > NR || lines[next_non_empty] !~
/^[[:space:]]{6,}/) {
+ print " " branch ":"
+ print " required_pull_request_reviews:"
+ print " required_approving_review_count: 1"
+ # Skip to next iteration to avoid double printing
+ for (j = i + 1; j <= NR; j++) {
+ i = j
+ if (j <= NR) print lines[j]
+ }
+ break
+ }
+ }
+ }
+}
+' "$ASF_YAML_FILE" > "$TEMP_FILE"
+
+# Check if awk succeeded
+if [ $? -ne 0 ]; then
+ rm -f "$TEMP_FILE"
+ exit 1
+fi
+
+# Verify the new content was added
+if ! grep -q "^[[:space:]]*${BRANCH_NAME}:" "$TEMP_FILE"; then
+ echo "Error: Failed to add branch protection block"
+ rm -f "$TEMP_FILE"
+ exit 1
+fi
+
+# Replace the original file with the modified version
+mv "$TEMP_FILE" "$ASF_YAML_FILE"
+
+echo "Successfully added branch protection for ${BRANCH_NAME} to
$ASF_YAML_FILE"
+echo ""
+echo "Added block:"
+echo " ${BRANCH_NAME}:"
+echo " required_pull_request_reviews:"
+echo " required_approving_review_count: 1"
+echo ""
+echo "Please review the changes and commit them."
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
