This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v2-10-test by this push:
new 26d9857c4a Update release command for Airflow 2 (#41907)
26d9857c4a is described below
commit 26d9857c4af8618843ed32fef4b9e3250bf3dd25
Author: Ephraim Anierobi <[email protected]>
AuthorDate: Sun Sep 1 08:10:01 2024 +0100
Update release command for Airflow 2 (#41907)
* Update release command for Airflow 2
For airflow 2, we create a branch out of v2-10-test and sync that branch
with v2-10-stable to make a release.
* Update CI check
---
.github/workflows/basic-tests.yml | 3 +-
dev/README_RELEASE_AIRFLOW.md | 49 ++++++++++------------
.../output_release-management_start-rc-process.svg | 26 +++++++-----
.../output_release-management_start-rc-process.txt | 2 +-
.../commands/release_candidate_command.py | 10 +++--
.../commands/release_management_commands_config.py | 1 +
6 files changed, 46 insertions(+), 45 deletions(-)
diff --git a/.github/workflows/basic-tests.yml
b/.github/workflows/basic-tests.yml
index 9828a14993..ae968103db 100644
--- a/.github/workflows/basic-tests.yml
+++ b/.github/workflows/basic-tests.yml
@@ -372,7 +372,8 @@ jobs:
run: breeze release-management create-minor-branch --version-branch
2-8 --answer yes
- name: "Check Airflow RC process command"
run: >
- breeze release-management start-rc-process --version 2.8.3rc1
--previous-version 2.8.0 --answer yes
+ breeze release-management start-rc-process --version 2.8.3rc1
--previous-version 2.8.0
+ --sync-branch sync_v2_8_test --answer yes
- name: "Check Airflow release process command"
run: >
breeze release-management start-release --release-candidate 2.8.3rc1
--previous-release 2.8.0
diff --git a/dev/README_RELEASE_AIRFLOW.md b/dev/README_RELEASE_AIRFLOW.md
index 19d2bcec04..2d63edf301 100644
--- a/dev/README_RELEASE_AIRFLOW.md
+++ b/dev/README_RELEASE_AIRFLOW.md
@@ -22,8 +22,8 @@
- [Selecting what to put into the
release](#selecting-what-to-put-into-the-release)
- [Selecting what to cherry-pick](#selecting-what-to-cherry-pick)
- - [Making the cherry picking](#making-the-cherry-picking)
- - [Reviewing cherry-picked PRs and assigning
labels](#reviewing-cherry-picked-prs-and-assigning-labels)
+ - [Backporting the PRs](#backporting-the-prs)
+ - [Reviewing Backported PRs and assigning
labels](#reviewing-backported-prs-and-assigning-labels)
- [Prepare the Apache Airflow Package
RC](#prepare-the-apache-airflow-package-rc)
- [Update the milestone](#update-the-milestone)
- [Build RC artifacts](#build-rc-artifacts)
@@ -73,9 +73,6 @@ The first step of a release is to work out what is being
included. This differs
## Selecting what to cherry-pick
-For obvious reasons, you can't cherry-pick every change from `main` into the
release branch -
-some are incompatible without a large set of other changes, some are brand-new
features, and some just don't need to be in a release.
-
In general only security fixes, data-loss bugs and regression fixes are
essential to bring into a patch release;
also changes in dependencies (pyproject.toml) resulting from releasing newer
versions of packages that Airflow depends on.
Other bugfixes can be added on a best-effort basis, but if something is going
to be very difficult to backport
@@ -93,7 +90,7 @@ and mark those as well. You can accomplish this by running
the following command
./dev/airflow-github needs-categorization 2.3.2 HEAD
```
-Often you also want to cherry-pick changes related to CI and development
tools, to include the latest
+Often you also want to backport changes related to CI and development tools,
to include the latest
stability fixes in CI and improvements in development tools. Usually you can
see the list of such
changes via (this will exclude already merged changes):
@@ -105,9 +102,9 @@ git log --oneline --decorate
apache/v2-2-stable..apache/main -- Dockerfile* scri
Most of those PRs should be marked with `changelog:skip` label, so that they
are excluded from the
user-facing changelog as they only matter for developers of Airflow. We have a
tool
-that allows to easily review the cherry-picked PRs and mark them with the
right label - see below.
+that allows to easily review the backported PRs and mark them with the right
label - see below.
-You also likely want to cherry-pick some of the latest doc changes in order to
bring clarification and
+You also likely want to backport some of the latest doc changes in order to
bring clarification and
explanations added to the documentation. Usually you can see the list of such
changes via:
```shell
@@ -119,29 +116,24 @@ git log --oneline --decorate
apache/v2-2-stable..apache/main -- docs/apache-airf
Those changes that are "doc-only" changes should be marked with
`type:doc-only` label so that they
land in documentation part of the changelog. The tool to review and assign the
labels is described below.
-## Making the cherry picking
-
-It is recommended to clone Airflow upstream (not your fork) and run the
commands on
-the relevant test branch in this clone. That way origin points to the upstream
repo.
+## Backporting the PRs
-To see cherry picking candidates (unmerged PR with the appropriate milestone),
from the test
-branch you can run:
+If a PR needs to be backported, checkout v2-10-test and make a new branch for
the backport:
```shell
-./dev/airflow-github compare 2.1.2 --unmerged
+git checkout v2-10-test
+git pull && git checkout -b <branchname>
```
-You can start cherry picking from the bottom of the list. (older commits first)
-
-When you cherry-pick, pick in chronological order onto the `vX-Y-test` release
branch.
-You'll move them over to be on `vX-Y-stable` once the release is cut. Use the
`-x` option
-to keep a reference to the original commit we cherry picked from. ("cherry
picked from commit ...")
+Then cherry-pick the commit from main:
```shell
git cherry-pick <hash-commit> -x
```
-## Reviewing cherry-picked PRs and assigning labels
+Make your PR and wait for reviews and approval
+
+## Reviewing Backported PRs and assigning labels
We have the tool that allows to review cherry-picked PRs and assign the labels
[./assign_cherry_picked_prs_with_milestone.py](./assign_cherry_picked_prs_with_milestone.py)
@@ -152,7 +144,7 @@ It allows to manually review and assign milestones and
labels to cherry-picked P
./dev/assign_cherry_picked_prs_with_milestone.py assign-prs --previous-release
v2-2-stable --current-release apache/v2-2-test --milestone-number 48
```
-It summarises the state of each cherry-picked PR including information whether
it is going to be
+It summarises the state of each Backported PR including information whether it
is going to be
excluded or included in changelog or included in doc-only part of it. It also
allows to re-assign
the PRs to the target milestone and apply the `changelog:skip` or
`type:doc-only` label.
@@ -160,7 +152,7 @@ You can also add `--skip-assigned` flag if you want to
automatically skip the qu
for the PRs that are already correctly assigned to the milestone. You can also
avoid the "Are you OK?"
question with `--assume-yes` flag.
-You can review the list of PRs cherry-picked and produce a nice summary with
`--print-summary` (this flag
+You can review the list of PRs backported and produce a nice summary with
`--print-summary` (this flag
assumes the `--skip-assigned` flag, so that the summary can be produced
without questions:
```shell
@@ -169,7 +161,7 @@ assumes the `--skip-assigned` flag, so that the summary can
be produced without
--output-folder /tmp
```
-This will produce summary output with nice links that you can use to review
the cherry-picked changes,
+This will produce summary output with nice links that you can use to review
the backported changes,
but it also produces files with list of commits separated by type in the
folder specified. In the case
above, it will produce three files that you can use in the next step:
@@ -225,6 +217,7 @@ The Release Candidate artifacts we vote upon should be the
exact ones we vote ag
export VERSION_SUFFIX=rc3
export VERSION_BRANCH=2-1
export VERSION_WITHOUT_RC=${VERSION/rc?/}
+ export SYNC_BRANCH=sync_v2_10_test
# Set AIRFLOW_REPO_ROOT to the path of your git repo
export AIRFLOW_REPO_ROOT=$(pwd)
@@ -253,8 +246,8 @@ The Release Candidate artifacts we vote upon should be the
exact ones we vote ag
- Check out the 'test' branch
```shell script
- git checkout v${VERSION_BRANCH}-test
- git reset --hard origin/v${VERSION_BRANCH}-test
+ git checkout ${SYNC_BRANCH}
+ git reset --hard origin/${SYNC_BRANCH}
```
- Set your version in `airflow/__init__.py`,
`airflow/api_connexion/openapi/v1.yaml` (without the RC tag).
@@ -284,7 +277,7 @@ The Release Candidate artifacts we vote upon should be the
exact ones we vote ag
create a fragment to document its change, to generate the body of the
release note based on the cherry picked commits:
```
- ./dev/airflow-github changelog v2-3-stable v2-3-test
+ ./dev/airflow-github changelog v2-3-stable ${SYNC_BRANCH}
```
- Commit the release note change.
@@ -310,7 +303,7 @@ The Release Candidate artifacts we vote upon should be the
exact ones we vote ag
```shell script
git checkout main
git pull # Ensure that the script is up-to-date
- breeze release-management start-rc-process --version ${VERSION}
--previous-version <PREVIOUS_VERSION>
+ breeze release-management start-rc-process --version ${VERSION}
--previous-version <PREVIOUS_VERSION> --sync-branch ${SYNC_BRANCH}
```
- Create issue in github for testing the release using this subject:
diff --git
a/dev/breeze/doc/images/output_release-management_start-rc-process.svg
b/dev/breeze/doc/images/output_release-management_start-rc-process.svg
index a91c45583c..7d8bd37259 100644
--- a/dev/breeze/doc/images/output_release-management_start-rc-process.svg
+++ b/dev/breeze/doc/images/output_release-management_start-rc-process.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 391.59999999999997"
xmlns="http://www.w3.org/2000/svg";>
+<svg class="rich-terminal" viewBox="0 0 1482 416.0"
xmlns="http://www.w3.org/2000/svg";>
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -45,7 +45,7 @@
<defs>
<clipPath id="breeze-release-management-start-rc-process-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="340.59999999999997" />
+ <rect x="0" y="0" width="1463.0" height="365.0" />
</clipPath>
<clipPath id="breeze-release-management-start-rc-process-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -86,9 +86,12 @@
<clipPath id="breeze-release-management-start-rc-process-line-12">
<rect x="0" y="294.3" width="1464" height="24.65"/>
</clipPath>
+<clipPath id="breeze-release-management-start-rc-process-line-13">
+ <rect x="0" y="318.7" width="1464" height="24.65"/>
+ </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="389.6" rx="8"/><text
class="breeze-release-management-start-rc-process-title" fill="#c5c8c6"
text-anchor="middle" x="740"
y="27">Command: release-management start-rc-process</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="414" rx="8"/><text
class="breeze-release-management-start-rc-process-title" fill="#c5c8c6"
text-anchor="middle" x="740"
y="27">Command: release-management start-rc-process</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -104,14 +107,15 @@
</text><text class="breeze-release-management-start-rc-process-r1" x="12.2"
y="93.2" textLength="500.2"
clip-path="url(#breeze-release-management-start-rc-process-line-3)">Start the process for releasing a new RC.</text><text
class="breeze-release-management-start-rc-process-r1" x="1464" y="93.2"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-3)">
</text><text class="breeze-release-management-start-rc-process-r1" x="1464"
y="117.6" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-4)">
</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="142" textLength="24.4"
clip-path="url(#breeze-release-management-start-rc-process-line-5)">╭─</text><text
class="breeze-release-management-start-rc-process-r5" x="24.4" y="142"
textLength="292.8"
clip-path="url(#breeze-release-management-start-rc-process-line-5)"> Start RC process flags </text><text
class="breeze-release-management-start-rc-process-r5" x="317.2" y="142"
textLength="1122.4 [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="166.4" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-6)">│</text><text
class="breeze-release-management-start-rc-process-r6" x="24.4" y="166.4"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-6)">*</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="166.4"
textLength="12.2" clip-path="url(#breeze-release-management-start [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="190.8" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-7)">│</text><text
class="breeze-release-management-start-rc-process-r6" x="24.4" y="190.8"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-7)">*</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="190.8"
textLength="12.2" clip-path="url(#breeze-release-management-start [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="215.2" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-8)">│</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="215.2"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-8)">-</text><text
class="breeze-release-management-start-rc-process-r4" x="73.2" y="215.2"
textLength="85.4" clip-path="url(#breeze-release-management-start [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="239.6" textLength="1464"
clip-path="url(#breeze-release-management-start-rc-process-line-9)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-start-rc-process-r1" x="1464" y="239.6"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-9)">
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="264" textLength="24.4"
clip-path="url(#breeze-release-management-start-rc-process-line-10)">╭─</text><text
class="breeze-release-management-start-rc-process-r5" x="24.4" y="264"
textLength="195.2"
clip-path="url(#breeze-release-management-start-rc-process-line-10)"> Common options </text><text
class="breeze-release-management-start-rc-process-r5" x="219.6" y="264"
textLength="1220" clip-path="url(# [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="288.4" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-11)">│</text><text
class="breeze-release-management-start-rc-process-r4" x="24.4" y="288.4"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-11)">-</text><text
class="breeze-release-management-start-rc-process-r4" x="36.6" y="288.4"
textLength="85.4" clip-path="url(#breeze-release-management-s [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="312.8" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-12)">│</text><text
class="breeze-release-management-start-rc-process-r4" x="24.4" y="312.8"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-12)">-</text><text
class="breeze-release-management-start-rc-process-r4" x="36.6" y="312.8"
textLength="61" clip-path="url(#breeze-release-management-sta [...]
-</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="337.2" textLength="1464"
clip-path="url(#breeze-release-management-start-rc-process-line-13)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-start-rc-process-r1" x="1464" y="337.2"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-13)">
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="166.4" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-6)">│</text><text
class="breeze-release-management-start-rc-process-r6" x="24.4" y="166.4"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-6)">*</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="166.4"
textLength="109.8" clip-path="url(#breeze-release-management-star [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="190.8" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-7)">│</text><text
class="breeze-release-management-start-rc-process-r6" x="24.4" y="190.8"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-7)">*</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="190.8"
textLength="219.6" clip-path="url(#breeze-release-management-star [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="215.2" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-8)">│</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="215.2"
textLength="170.8"
clip-path="url(#breeze-release-management-start-rc-process-line-8)">--github-token</text><text
class="breeze-release-management-start-rc-process-r1" x="329.4" y="215.2"
textLength="878.4" clip-path="url(#breeze-release- [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="239.6" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-9)">│</text><text
class="breeze-release-management-start-rc-process-r6" x="24.4" y="239.6"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-9)">*</text><text
class="breeze-release-management-start-rc-process-r4" x="61" y="239.6"
textLength="158.6" clip-path="url(#breeze-release-management-star [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="264" textLength="1464"
clip-path="url(#breeze-release-management-start-rc-process-line-10)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-start-rc-process-r1" x="1464" y="264"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-10)">
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="288.4" textLength="24.4"
clip-path="url(#breeze-release-management-start-rc-process-line-11)">╭─</text><text
class="breeze-release-management-start-rc-process-r5" x="24.4" y="288.4"
textLength="195.2"
clip-path="url(#breeze-release-management-start-rc-process-line-11)"> Common options </text><text
class="breeze-release-management-start-rc-process-r5" x="219.6" y="288.4"
textLength="1220" clip-path= [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="312.8" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-12)">│</text><text
class="breeze-release-management-start-rc-process-r4" x="24.4" y="312.8"
textLength="97.6"
clip-path="url(#breeze-release-management-start-rc-process-line-12)">--answer</text><text
class="breeze-release-management-start-rc-process-r9" x="146.4" y="312.8"
textLength="24.4" clip-path="url(#breeze-release-mana [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="337.2" textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-13)">│</text><text
class="breeze-release-management-start-rc-process-r4" x="24.4" y="337.2"
textLength="73.2"
clip-path="url(#breeze-release-management-start-rc-process-line-13)">--help</text><text
class="breeze-release-management-start-rc-process-r9" x="146.4" y="337.2"
textLength="24.4" clip-path="url(#breeze-release-manage [...]
+</text><text class="breeze-release-management-start-rc-process-r5" x="0"
y="361.6" textLength="1464"
clip-path="url(#breeze-release-management-start-rc-process-line-14)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-start-rc-process-r1" x="1464" y="361.6"
textLength="12.2"
clip-path="url(#breeze-release-management-start-rc-process-line-14)">
</text>
</g>
</g>
diff --git
a/dev/breeze/doc/images/output_release-management_start-rc-process.txt
b/dev/breeze/doc/images/output_release-management_start-rc-process.txt
index a17d1176a8..4d18780d55 100644
--- a/dev/breeze/doc/images/output_release-management_start-rc-process.txt
+++ b/dev/breeze/doc/images/output_release-management_start-rc-process.txt
@@ -1 +1 @@
-b27bd524dd3c89f50a747b60a7e892c1
+38af817973e7aac6a21f10c7ab9b2f4d
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_candidate_command.py
b/dev/breeze/src/airflow_breeze/commands/release_candidate_command.py
index 422804b79a..8c5c449ed7 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_candidate_command.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_candidate_command.py
@@ -35,7 +35,7 @@ CI = os.environ.get("CI")
RUNNING_IN_CI = True if CI else False
-def merge_pr(version_branch):
+def merge_pr(version_branch, sync_branch):
if confirm_action("Do you want to merge the Sync PR?"):
run_command(
[
@@ -52,7 +52,7 @@ def merge_pr(version_branch):
check=True,
)
run_command(
- ["git", "merge", "--ff-only", f"v{version_branch}-test"],
+ ["git", "merge", "--ff-only", f"{sync_branch}"],
dry_run_override=RUNNING_IN_CI,
check=True,
)
@@ -365,8 +365,9 @@ def prepare_airflow_tarball(version: str):
@click.option(
"--github-token", help="GitHub token to use in generating issue for
testing of release candidate"
)
[email protected]("--sync-branch", required=True, help="The branch to sync to the
stable branch")
@option_answer
-def publish_release_candidate(version, previous_version, github_token):
+def publish_release_candidate(version, previous_version, github_token,
sync_branch):
check_python_version()
from packaging.version import Version
@@ -395,6 +396,7 @@ def publish_release_candidate(version, previous_version,
github_token):
console_print(f"version_branch: {version_branch}")
console_print(f"version_without_rc: {version_without_rc}")
console_print(f"airflow_repo_root: {airflow_repo_root}")
+ console_print(f"sync_branch: {sync_branch}")
console_print()
console_print("Below are your git remotes. We will push to origin:")
run_command(["git", "remote", "-v"], dry_run_override=RUNNING_IN_CI)
@@ -403,7 +405,7 @@ def publish_release_candidate(version, previous_version,
github_token):
# Final confirmation
confirm_action("Pushes will be made to origin. Do you want to continue?",
abort=True)
# Merge the sync PR
- merge_pr(version_branch)
+ merge_pr(version_branch, sync_branch)
#
# # Tag & clean the repo
git_tag(version)
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
index 4013476737..ab14922c05 100644
---
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
+++
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
@@ -348,6 +348,7 @@ RELEASE_MANAGEMENT_PARAMETERS: dict[str, list[dict[str, str
| list[str]]]] = {
"--version",
"--previous-version",
"--github-token",
+ "--sync-branch",
],
}
],
