This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 86c165ba54f Improve tagging of providers when preparing a release
(#57628)
86c165ba54f is described below
commit 86c165ba54f949ab5a8a92bfbc3aa0ccc852da57
Author: Jarek Potiuk <[email protected]>
AuthorDate: Fri Oct 31 17:54:10 2025 +0100
Improve tagging of providers when preparing a release (#57628)
Tagging providers after release after #57337 happened after preparing
the tarball - which was wrong :) as tarball uses the tag to build
the -sources package. Also it cannot be done too early becasue
tags are created based on the provider packages prepared in dist.
So - tagging providers *SHOULD* happen between creating provider
distributions and creating tarball.
This PR moves creation of the tarball to the right place, it also
improves a bit the "tag-providers" step:
* it uses PACKAGE_DATE environment variable if it is defined
* it does not react on existing tags - it simply overwrites the
tags if such tags exist.
---
dev/README_RELEASE_PROVIDERS.md | 33 ++++++------
.../output_release-management_tag-providers.svg | 28 ++++++-----
.../output_release-management_tag-providers.txt | 2 +-
.../commands/release_management_commands.py | 58 ++++++++--------------
.../commands/release_management_commands_config.py | 2 +-
5 files changed, 56 insertions(+), 67 deletions(-)
diff --git a/dev/README_RELEASE_PROVIDERS.md b/dev/README_RELEASE_PROVIDERS.md
index edec1b3e5cc..47b35e771ec 100644
--- a/dev/README_RELEASE_PROVIDERS.md
+++ b/dev/README_RELEASE_PROVIDERS.md
@@ -37,7 +37,6 @@
- [Build and sign the source and convenience
packages](#build-and-sign-the-source-and-convenience-packages)
- [Commit the source packages to Apache SVN
repo](#commit-the-source-packages-to-apache-svn-repo)
- [Publish the Regular distributions to PyPI (release
candidates)](#publish-the-regular-distributions-to-pypi-release-candidates)
- - [Add tags in git](#add-tags-in-git)
- [Prepare documentation in Staging](#prepare-documentation-in-staging)
- [Prepare issue in GitHub to keep status of
testing](#prepare-issue-in-github-to-keep-status-of-testing)
- [Prepare voting email for Providers release
candidate](#prepare-voting-email-for-providers-release-candidate)
@@ -47,7 +46,7 @@
- [Summarize the voting for the Apache Airflow
release](#summarize-the-voting-for-the-apache-airflow-release)
- [Publish release to SVN](#publish-release-to-svn)
- [Publish the packages to PyPI](#publish-the-packages-to-pypi)
- - [Add tags in git](#add-tags-in-git-1)
+ - [Add tags in git](#add-tags-in-git)
- [Publish documentation](#publish-documentation)
- [Update providers metadata](#update-providers-metadata)
- [Notify developers of release](#notify-developers-of-release)
@@ -368,6 +367,7 @@ rm -rf ${AIRFLOW_REPO_ROOT}/dist/*
```shell script
breeze release-management prepare-provider-distributions
--include-removed-providers --distribution-format both
+breeze release-management tag-providers
breeze release-management prepare-airflow-tarball --version ${PACKAGE_DATE}
--distribution-name apache_airflow_providers
```
@@ -376,6 +376,7 @@ if you only build few packages, run:
```shell script
breeze release-management prepare-provider-distributions
--include-removed-providers \
--distribution-format both PACKAGE PACKAGE ....
+breeze release-management tag-providers
breeze release-management prepare-airflow-tarball --version ${PACKAGE_DATE}
--distribution-name apache_airflow_providers
```
@@ -383,6 +384,19 @@ In case you want to also release a pre-installed provider
that is in ``not-ready
you want to release it before you switch their state to ``ready``), you need
to pass
``--include-not-ready-providers`` flag to the command above.
+.. note:
+
+ The "tag-providers" step assumes that your remote for apache repository is
called `apache`.
+
+ Sometimes in cases when there is a connectivity issue to GitHub, it might be
possible that local
+ tags get created and lead to annoying errors. The default behaviour would
be to clean such local tags up,
+ so you can re-run tag-providers command without issues, and it will force
tag creation properly, overriding
+ existing tags.
+
+ If you want to disable this behaviour, use --no-clean-tags.
+
+
+
* Sign all your packages
@@ -489,21 +503,6 @@ twine upload -r pypi ${AIRFLOW_REPO_ROOT}/dist/*
* Confirm that the packages are available under the links printed and look
good.
-
-## Add tags in git
-
-Assume that your remote for apache repository is called `apache` you should now
-set tags for the providers in the repo.
-
-Sometimes in cases when there is a connectivity issue to GitHub, it might be
possible that local tags get created
-and lead to annoying errors. The default behaviour would be to clean such
local tags up.
-
-If you want to disable this behaviour, set the env **CLEAN_LOCAL_TAGS** to
false.
-
-```shell script
-breeze release-management tag-providers
-```
-
## Prepare documentation in Staging
Documentation is an essential part of the product and should be made available
to users.
diff --git a/dev/breeze/doc/images/output_release-management_tag-providers.svg
b/dev/breeze/doc/images/output_release-management_tag-providers.svg
index 00ac6b5bee3..7a9e511f910 100644
--- a/dev/breeze/doc/images/output_release-management_tag-providers.svg
+++ b/dev/breeze/doc/images/output_release-management_tag-providers.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 416.0"
xmlns="http://www.w3.org/2000/svg";>
+<svg class="rich-terminal" viewBox="0 0 1482 440.4"
xmlns="http://www.w3.org/2000/svg";>
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -42,7 +42,7 @@
<defs>
<clipPath id="breeze-release-management-tag-providers-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="365.0" />
+ <rect x="0" y="0" width="1463.0" height="389.4" />
</clipPath>
<clipPath id="breeze-release-management-tag-providers-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -86,9 +86,12 @@
<clipPath id="breeze-release-management-tag-providers-line-13">
<rect x="0" y="318.7" width="1464" height="24.65"/>
</clipPath>
+<clipPath id="breeze-release-management-tag-providers-line-14">
+ <rect x="0" y="343.1" width="1464" height="24.65"/>
+ </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="414" rx="8"/><text
class="breeze-release-management-tag-providers-title" fill="#c5c8c6"
text-anchor="middle" x="740"
y="27">Command: release-management tag-providers</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="438.4" rx="8"/><text
class="breeze-release-management-tag-providers-title" fill="#c5c8c6"
text-anchor="middle" x="740"
y="27">Command: release-management tag-providers</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -104,15 +107,16 @@
</text><text class="breeze-release-management-tag-providers-r1" x="12.2"
y="93.2" textLength="549"
clip-path="url(#breeze-release-management-tag-providers-line-3)">Generates tags for airflow provider releases.</text><text
class="breeze-release-management-tag-providers-r1" x="1464" y="93.2"
textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-3)">
</text><text class="breeze-release-management-tag-providers-r1" x="1464"
y="117.6" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-4)">
</text><text class="breeze-release-management-tag-providers-r5" x="0" y="142"
textLength="24.4"
clip-path="url(#breeze-release-management-tag-providers-line-5)">╭─</text><text
class="breeze-release-management-tag-providers-r5" x="24.4" y="142"
textLength="280.6"
clip-path="url(#breeze-release-management-tag-providers-line-5)"> Add tags to providers </text><text
class="breeze-release-management-tag-providers-r5" x="305" y="142"
textLength="1134.6" clip-path="url(# [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="166.4" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-6)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="166.4"
textLength="219.6"
clip-path="url(#breeze-release-management-tag-providers-line-6)">--clean-local-tags</text><text
class="breeze-release-management-tag-providers-r1" x="244" y="166.4"
textLength="12.2" clip-path="url(#breeze-release-management-t [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="190.8" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-7)">│</text><text
class="breeze-release-management-tag-providers-r1" x="561.2" y="190.8"
textLength="878.4"
clip-path="url(#breeze-release-management-tag-providers-line-7)">avoid errors. The default behaviour would be to clean such local tags.  </text><text
class="bre [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="215.2" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-8)">│</text><text
class="breeze-release-management-tag-providers-r5" x="561.2" y="215.2"
textLength="878.4"
clip-path="url(#breeze-release-management-tag-providers-line-8)">[default: clean-local-tags]                     &#
[...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="239.6" textLength="1464"
clip-path="url(#breeze-release-management-tag-providers-line-9)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-tag-providers-r1" x="1464" y="239.6"
textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-9)">
-</text><text class="breeze-release-management-tag-providers-r5" x="0" y="264"
textLength="24.4"
clip-path="url(#breeze-release-management-tag-providers-line-10)">╭─</text><text
class="breeze-release-management-tag-providers-r5" x="24.4" y="264"
textLength="195.2"
clip-path="url(#breeze-release-management-tag-providers-line-10)"> Common options </text><text
class="breeze-release-management-tag-providers-r5" x="219.6" y="264"
textLength="1220" clip-path="url(#breeze-release- [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="288.4" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-11)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="288.4"
textLength="109.8"
clip-path="url(#breeze-release-management-tag-providers-line-11)">--dry-run</text><text
class="breeze-release-management-tag-providers-r6" x="158.6" y="288.4"
textLength="24.4" clip-path="url(#breeze-release-management-tag-pr [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="312.8" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-12)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="312.8"
textLength="109.8"
clip-path="url(#breeze-release-management-tag-providers-line-12)">--verbose</text><text
class="breeze-release-management-tag-providers-r6" x="158.6" y="312.8"
textLength="24.4" clip-path="url(#breeze-release-management-tag-pr [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="337.2" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-13)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="337.2"
textLength="73.2"
clip-path="url(#breeze-release-management-tag-providers-line-13)">--help</text><text
class="breeze-release-management-tag-providers-r6" x="158.6" y="337.2"
textLength="24.4" clip-path="url(#breeze-release-management-tag-provid [...]
-</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="361.6" textLength="1464"
clip-path="url(#breeze-release-management-tag-providers-line-14)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-tag-providers-r1" x="1464" y="361.6"
textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-14)">
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="166.4" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-6)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="166.4"
textLength="146.4"
clip-path="url(#breeze-release-management-tag-providers-line-6)">--clean-tags</text><text
class="breeze-release-management-tag-providers-r1" x="170.8" y="166.4"
textLength="12.2" clip-path="url(#breeze-release-management-tag-p [...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="190.8" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-7)">│</text><text
class="breeze-release-management-tag-providers-r1" x="414.8" y="190.8"
textLength="1024.8"
clip-path="url(#breeze-release-management-tag-providers-line-7)">issues to avoid errors. The default behaviour would be to clean both local and  
[...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="215.2" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-8)">│</text><text
class="breeze-release-management-tag-providers-r1" x="414.8" y="215.2"
textLength="1024.8"
clip-path="url(#breeze-release-management-tag-providers-line-8)">remote tags.                       
[...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="239.6" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-9)">│</text><text
class="breeze-release-management-tag-providers-r5" x="414.8" y="239.6"
textLength="1024.8"
clip-path="url(#breeze-release-management-tag-providers-line-9)">[default: clean-tags]                      &
[...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0" y="264"
textLength="1464"
clip-path="url(#breeze-release-management-tag-providers-line-10)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-tag-providers-r1" x="1464" y="264"
textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-10)">
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="288.4" textLength="24.4"
clip-path="url(#breeze-release-management-tag-providers-line-11)">╭─</text><text
class="breeze-release-management-tag-providers-r5" x="24.4" y="288.4"
textLength="195.2"
clip-path="url(#breeze-release-management-tag-providers-line-11)"> Common options </text><text
class="breeze-release-management-tag-providers-r5" x="219.6" y="288.4"
textLength="1220" clip-path="url(#breeze-re [...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="312.8" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-12)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="312.8"
textLength="109.8"
clip-path="url(#breeze-release-management-tag-providers-line-12)">--dry-run</text><text
class="breeze-release-management-tag-providers-r6" x="158.6" y="312.8"
textLength="24.4" clip-path="url(#breeze-release-management-tag-pr [...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="337.2" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-13)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="337.2"
textLength="109.8"
clip-path="url(#breeze-release-management-tag-providers-line-13)">--verbose</text><text
class="breeze-release-management-tag-providers-r6" x="158.6" y="337.2"
textLength="24.4" clip-path="url(#breeze-release-management-tag-pr [...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0"
y="361.6" textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-14)">│</text><text
class="breeze-release-management-tag-providers-r4" x="24.4" y="361.6"
textLength="73.2"
clip-path="url(#breeze-release-management-tag-providers-line-14)">--help</text><text
class="breeze-release-management-tag-providers-r6" x="158.6" y="361.6"
textLength="24.4" clip-path="url(#breeze-release-management-tag-provid [...]
+</text><text class="breeze-release-management-tag-providers-r5" x="0" y="386"
textLength="1464"
clip-path="url(#breeze-release-management-tag-providers-line-15)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-tag-providers-r1" x="1464" y="386"
textLength="12.2"
clip-path="url(#breeze-release-management-tag-providers-line-15)">
</text>
</g>
</g>
diff --git a/dev/breeze/doc/images/output_release-management_tag-providers.txt
b/dev/breeze/doc/images/output_release-management_tag-providers.txt
index eb987987e51..6e77b6d91d0 100644
--- a/dev/breeze/doc/images/output_release-management_tag-providers.txt
+++ b/dev/breeze/doc/images/output_release-management_tag-providers.txt
@@ -1 +1 @@
-403dae670f839e37c83baa15ac65af9a
+cd7abd0560cc4829b9567a4f03880fd5
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
index 843efb83bb4..9a225af754e 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
@@ -17,7 +17,6 @@
from __future__ import annotations
import ast
-import contextlib
import glob
import operator
import os
@@ -1288,18 +1287,18 @@ def run_generate_constraints_in_parallel(
help="Generates tags for airflow provider releases.",
)
@click.option(
- "--clean-local-tags/--no-clean-local-tags",
+ "--clean-tags/--no-clean-tags",
default=True,
is_flag=True,
- envvar="CLEAN_LOCAL_TAGS",
- help="Delete local tags that are created due to github connectivity issues
to avoid errors. "
- "The default behaviour would be to clean such local tags.",
+ envvar="CLEAN_TAGS",
+ help="Delete tags (both local and remote) that are created due to github
connectivity "
+ "issues to avoid errors. The default behaviour would be to clean both
local and remote tags.",
show_default=True,
)
@option_dry_run
@option_verbose
def tag_providers(
- clean_local_tags: bool,
+ clean_tags: bool,
):
found_remote = None
remotes = ["origin", "apache"]
@@ -1317,32 +1316,31 @@ def tag_providers(
except subprocess.CalledProcessError:
pass
- release_date = datetime.now().strftime("%Y-%m-%d")
+ release_date = os.environ.get("PACKAGE_DATE",
datetime.now().strftime("%Y-%m-%d"))
if found_remote is None:
raise ValueError("Could not find remote configured to push to
apache/airflow")
+ extra_flags = []
tags = []
+ if clean_tags:
+ extra_flags.append("--force")
for file in os.listdir(os.path.join(SOURCE_DIR_PATH, "dist")):
if file.endswith(".whl"):
match = re.match(r".*airflow_providers_(.*)-(.*)-py3.*", file)
if match:
provider = f"providers-{match.group(1).replace('_', '-')}"
tag = f"{provider}/{match.group(2)}"
- try:
- get_console().print(f"[info]Creating tag: {tag}")
- run_command(
- ["git", "tag", tag, "-m", f"Release {release_date} of
providers"],
- check=True,
- )
- tags.append(tag)
- except subprocess.CalledProcessError as e:
- get_console().print(f"[warning]Failed to create {tag}:
{e}")
- pass
+ get_console().print(f"[info]Creating tag: {tag}")
+ run_command(
+ ["git", "tag", tag, *extra_flags, "-m", f"Release
{release_date} of providers"],
+ check=True,
+ )
+ tags.append(tag)
providers_date_tag = f"providers/{release_date}"
if tags:
run_command(
- ["git", "tag", providers_date_tag, "-m", f"Release {release_date}
of providers"],
+ ["git", "tag", providers_date_tag, *extra_flags, "-m", f"Release
{release_date} of providers"],
check=True,
)
get_console().print()
@@ -1350,24 +1348,12 @@ def tag_providers(
get_console().print(providers_date_tag)
get_console().print()
tags.append(providers_date_tag)
- try:
- push_result = run_command(
- ["git", "push", found_remote, *tags],
- check=False,
- )
- if push_result.returncode == 0:
- get_console().print("\n[success]Tags pushed successfully.[/]")
- except subprocess.CalledProcessError:
- get_console().print("\n[error]Failed to push tags, probably a
connectivity issue to GitHub.[/]")
- if clean_local_tags:
- for tag in tags:
- with contextlib.suppress(subprocess.CalledProcessError):
- run_command(["git", "tag", "-d", tag], check=True)
- get_console().print("\n[success]Cleaning up local tags...[/]")
- else:
- get_console().print(
- "\n[success]Local tags are not cleaned up, unset
CLEAN_LOCAL_TAGS or set to true.[/]"
- )
+ push_result = run_command(
+ ["git", "push", found_remote, *extra_flags, *tags],
+ check=True,
+ )
+ if push_result.returncode == 0:
+ get_console().print("\n[success]Tags pushed successfully.[/]")
@release_management.command(
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
index 2e843a04af6..2ea715946d5 100644
---
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
+++
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
@@ -243,7 +243,7 @@ RELEASE_MANAGEMENT_PARAMETERS: dict[str, list[dict[str, str
| list[str]]]] = {
{
"name": "Add tags to providers",
"options": [
- "--clean-local-tags",
+ "--clean-tags",
],
},
],
