This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 5d88f6f9a4 Improve generation of "Status of testing" issue (#36470)
5d88f6f9a4 is described below
commit 5d88f6f9a4c7140c1da9db47aab7caf2d4c5f453
Author: Jarek Potiuk <[email protected]>
AuthorDate: Thu Dec 28 21:02:27 2023 +0100
Improve generation of "Status of testing" issue (#36470)
This PR improves/simplifies the process of issue generation when
provider package rc candidates are prepared for voting.
It improves the commmand to generate the issue and makes it simpler
(less copy&paste) to create such issue, the issue also does not use
the "Meta" template and gets the right labels assigned automatically.
Recent changes that automatically derive the suffix from PyPI packages
prepared, removed the need of passing `--suffix` as parameter. In all
cases the right rc* suffix will be automatically added during issue
generation based on the version of package being prepared. The process
has been updated and command simplified by removing the `--suffix` flag.
When the issue is prepared, we display the issue in terminal and asked
the release manager to create the issue by copy&pasting the issue
content and title to a new issue, but that required a few copy&pastes
and opening new Issue via "Meta" task type. This PR simplifies it a
bit by not only displaying the content but also generating a URL that
can be either copy&pasted to browser URL field or just Cmd+clicked
if your terminal allows that. Issue created this way does not have
the "Body" field header and has the labels properly assigned including
a dedicated "testing status" label that is used to gether stats for
past "status" issues.
The advice for release manager has been improved (the comment generated
had some missing end of sentence and it should be now clearer on how
to iterate during issue generation if you want to remove some PRs from
the generated issue content.
---
dev/README_RELEASE_PROVIDER_PACKAGES.md | 50 ++++++++++++++++++----
.../commands/release_management_commands.py | 47 +++++++++++++++-----
.../commands/release_management_commands_config.py | 1 -
.../provider_issue_TEMPLATE.md.jinja2 | 8 +---
...management_generate-issue-content-providers.svg | 18 +++-----
...management_generate-issue-content-providers.txt | 2 +-
6 files changed, 88 insertions(+), 38 deletions(-)
diff --git a/dev/README_RELEASE_PROVIDER_PACKAGES.md
b/dev/README_RELEASE_PROVIDER_PACKAGES.md
index ba0657f507..618a6b7d68 100644
--- a/dev/README_RELEASE_PROVIDER_PACKAGES.md
+++ b/dev/README_RELEASE_PROVIDER_PACKAGES.md
@@ -530,28 +530,60 @@ git push --set-upstream origin "${branch}"
## Prepare issue in GitHub to keep status of testing
-Create a GitHub issue with the content generated via manual
-execution of the script below. You will use link to that issue in the next
step. You need a GITHUB_TOKEN
-set as your environment variable.
+Create a GitHub issue with the content generated via manual execution of the
command below. You will use
+link to that issue in the next step.
-You can also pass the token as `--github-token` option in the script.
-You can also pass list of PR to be excluded from the issue with
`--excluded-pr-list`.
+```shell script
+cd "${AIRFLOW_REPO_ROOT}"
+
+breeze release-management generate-issue-content-providers
--only-available-in-dist
+```
+
+GitHub API uses rate limiting that is based on the public IP address of client
if you do not authenticate
+with GitHub, so when you retrieve bigger number of PRs or when you are behind
NAT and share your public
+IP address with many other Anonymous GitHub API users, issue retrieval will be
halted and your API calls
+might slow down to a crawl, you will need then a GITHUB_TOKEN set as your
+environment variable or pass the token as `--github-token` option in the
script.
```shell script
cd "${AIRFLOW_REPO_ROOT}"
+breeze release-management generate-issue-content-providers
--only-available-in-dist --github-token TOKEN
+```
+
+or
+
+```shell script
+cd "${AIRFLOW_REPO_ROOT}"
+export GITHUB_TOKEN=TOKEN
breeze release-management generate-issue-content-providers
--only-available-in-dist
```
-You can also generate the token by following
-[this
link](https://github.com/settings/tokens/new?description=Read%20sssues&scopes=repo:status)
+You can generate the token by following
+[this
link](https://github.com/settings/tokens/new?description=Read%20issues&scopes=repo:status).
Since it is easy to generate such token, by following the link, it is
recommended to
+generate a new token for each release and delete it once you've generated the
issue.
+
+If you see in the output that some of the PRs are just "noise" (i.e. there is
no need to verify them
+as they are misc/documentation kind of changes that have no impact on the
actual installation of
+the provider or the code of the provider, can optionally pass list of PR to be
excluded from
+the issue with `--excluded-pr-list`. This might limit the scope of
verification. Some providers
+might disappear from the list and list of authors that will be pinged in the
generated issue.
-If you are preparing release for RC2/RC3 candidates, you should add `--suffix`
parameter:
+You can repeat that and regenerate the issue content until you are happy with
the generated issue.
```shell script
-breeze release-management generate-issue-content-providers
--only-available-in-dist --suffix rc2
+cd "${AIRFLOW_REPO_ROOT}"
+
+breeze release-management generate-issue-content-providers
--only-available-in-dist --github-token TOKEN \
+ --excluded-pr-list PR_NUMBER1,PR_NUMBER2
```
+It's also OK to manually modify the content of such generated issue before
actually creating the
+issue. There is a comment generated with NOTE TO RELEASE MANAGER about this in
the issue content.
+Hit Preview button on "create issue" screen before creating it to verify how
it will look like
+for the contributors.
+
+
## Prepare voting email for Providers release candidate
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
index 88dd04bb4a..1f21b6937b 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
@@ -29,7 +29,7 @@ from copy import deepcopy
from datetime import datetime
from pathlib import Path
from subprocess import DEVNULL
-from typing import IO, TYPE_CHECKING, Any, Generator, NamedTuple
+from typing import IO, TYPE_CHECKING, Any, Generator, Iterable, NamedTuple
import click
from rich.progress import Progress
@@ -1572,6 +1572,25 @@ def get_prs_for_package(provider_id: str) -> list[int]:
return prs
+def create_github_issue_url(title: str, body: str, labels: Iterable[str]) ->
str:
+ """
+ Creates URL to create the issue with title, body and labels.
+ :param title: issue title
+ :param body: issue body
+ :param labels: labels for the issue
+ :return: URL to use to create the issue
+ """
+ from urllib.parse import quote
+
+ quoted_labels = quote(",".join(labels))
+ quoted_title = quote(title)
+ quoted_body = quote(body)
+ return (
+ f"https://github.com/apache/airflow/issues/new?labels={quoted_labels}&";
+ f"title={quoted_title}&body={quoted_body}"
+ )
+
+
@release_management.command(
name="generate-issue-content-providers", help="Generates content for issue
to test the release."
)
@@ -1593,14 +1612,12 @@ def get_prs_for_package(provider_id: str) -> list[int]:
is_flag=True,
help="Only consider package ids with packages prepared in the dist folder",
)
[email protected]("--suffix", default="rc1", help="Suffix to add to the version
prepared")
@argument_provider_packages
def generate_issue_content_providers(
disable_progress: bool,
excluded_pr_list: str,
github_token: str,
only_available_in_dist: bool,
- suffix: str,
provider_packages: list[str],
):
import jinja2
@@ -1675,7 +1692,7 @@ def generate_issue_content_providers(
provider_package_id=provider_id,
pypi_package_name=provider_yaml_dict["package-name"],
pr_list=pull_request_list,
- suffix=package_suffix if package_suffix else suffix,
+ suffix=package_suffix if package_suffix else "",
)
template = jinja2.Template(
(Path(__file__).parents[1] /
"provider_issue_TEMPLATE.md.jinja2").read_text()
@@ -1689,19 +1706,29 @@ def generate_issue_content_providers(
get_console().print()
get_console().print()
get_console().print(
- "Issue title: [yellow]Status of testing Providers that were "
+ "Issue title: [warning]Status of testing Providers that were "
f"prepared on {datetime.now():%B %d, %Y}[/]"
)
get_console().print()
- syntax = Syntax(issue_content, "markdown", theme="ansi_dark")
- get_console().print(syntax)
- get_console().print()
+ issue_content += "\n"
users: set[str] = set()
for provider_info in providers.values():
for pr in provider_info.pr_list:
users.add("@" + pr.user.login)
- get_console().print("All users involved in the PRs:")
- get_console().print(" ".join(users))
+ issue_content += f"All users involved in the PRs:\n{' '.join(users)}"
+ syntax = Syntax(issue_content, "markdown", theme="ansi_dark")
+ get_console().print(syntax)
+ url_to_create_the_issue = create_github_issue_url(
+ title=f"Status of testing Providers that were prepared on
{datetime.now():%B %d, %Y}",
+ body=issue_content,
+ labels=["testing status", "kind:meta"],
+ )
+ get_console().print()
+ get_console().print(
+ "[info]You can prefill the issue by copy&pasting this link to
browser "
+ "(or Cmd+Click if your terminal supports it):\n"
+ )
+ print(url_to_create_the_issue)
def get_all_constraint_files(refresh_constraints: bool, python_version: str)
-> None:
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
index 1dfa78f3aa..77fc83650c 100644
---
a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
+++
b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py
@@ -234,7 +234,6 @@ RELEASE_MANAGEMENT_PARAMETERS: dict[str, list[dict[str, str
| list[str]]]] = {
"--excluded-pr-list",
"--github-token",
"--only-available-in-dist",
- "--suffix",
],
}
],
diff --git a/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2
b/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2
index 8009549772..c519048723 100644
--- a/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2
+++ b/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2
@@ -20,11 +20,7 @@ Those are providers that require testing as there were some
substantial changes
NOTE TO RELEASE MANAGER:
-Please move here the providers that have doc-only changes or for which changes
are trivial, and
-you could assess that they are OK. In case
-
-The providers are automatically installed on Airflow 2.3 and latest `main`
during the CI, so we know they
-are installable. Also, all classes within the providers are imported during
the CI run so we know all
-providers can be imported.
+You can move here the providers that have doc-only changes or for which
changes are trivial, and
+you could assess that they are OK.
-->
diff --git
a/images/breeze/output_release-management_generate-issue-content-providers.svg
b/images/breeze/output_release-management_generate-issue-content-providers.svg
index 46fce9329b..e8e193e1d1 100644
---
a/images/breeze/output_release-management_generate-issue-content-providers.svg
+++
b/images/breeze/output_release-management_generate-issue-content-providers.svg
@@ -1,4 +1,4 @@
-<svg class="rich-terminal" viewBox="0 0 1482 733.1999999999999"
xmlns="http://www.w3.org/2000/svg";>
+<svg class="rich-terminal" viewBox="0 0 1482 708.8"
xmlns="http://www.w3.org/2000/svg";>
<!-- Generated with Rich https://www.textualize.io -->
<style>
@@ -43,7 +43,7 @@
<defs>
<clipPath
id="breeze-release-management-generate-issue-content-providers-clip-terminal">
- <rect x="0" y="0" width="1463.0" height="682.1999999999999" />
+ <rect x="0" y="0" width="1463.0" height="657.8" />
</clipPath>
<clipPath
id="breeze-release-management-generate-issue-content-providers-line-0">
<rect x="0" y="1.5" width="1464" height="24.65"/>
@@ -123,12 +123,9 @@
<clipPath
id="breeze-release-management-generate-issue-content-providers-line-25">
<rect x="0" y="611.5" width="1464" height="24.65"/>
</clipPath>
-<clipPath
id="breeze-release-management-generate-issue-content-providers-line-26">
- <rect x="0" y="635.9" width="1464" height="24.65"/>
- </clipPath>
</defs>
- <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="731.2" rx="8"/><text
class="breeze-release-management-generate-issue-content-providers-title"
fill="#c5c8c6" text-anchor="middle" x="740"
y="27">Command: release-management generate-issue-content-providers</text>
+ <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1"
x="1" y="1" width="1480" height="706.8" rx="8"/><text
class="breeze-release-management-generate-issue-content-providers-title"
fill="#c5c8c6" text-anchor="middle" x="740"
y="27">Command: release-management generate-issue-content-providers</text>
<g transform="translate(26,22)">
<circle cx="0" cy="0" r="7" fill="#ff5f57"/>
<circle cx="22" cy="0" r="7" fill="#febc2e"/>
@@ -161,11 +158,10 @@
</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="508" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-20)">│</text><text
class="breeze-release-management-generate-issue-content-providers-r1" x="366"
y="508" textLength="1073.6"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-20)">https://github.com/settings/tokens/new?description=Read%20sssues&scopes=repo:status
[...]
</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="532.4" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-21)">│</text><text
class="breeze-release-management-generate-issue-content-providers-r6" x="366"
y="532.4" textLength="1073.6"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-21)">(TEXT)             
[...]
</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="556.8" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-22)">│</text><text
class="breeze-release-management-generate-issue-content-providers-r4" x="24.4"
y="556.8" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-22)">-</text><text
class="breeze-release-management-generate-issue-content-providers-r4" [...]
-</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="581.2" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-23)">│</text><text
class="breeze-release-management-generate-issue-content-providers-r4" x="24.4"
y="581.2" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-23)">-</text><text
class="breeze-release-management-generate-issue-content-providers-r4" [...]
-</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="605.6" textLength="1464"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-24)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-generate-issue-content-providers-r1" x="1464"
y="605.6" textLength="12.2" clip-path="url(#breeze-release-management-generate
[...]
-</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="630" textLength="24.4"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)">╭─</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="24.4"
y="630" textLength="195.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)"> Common options </text><text
class="breeze-release-management-generate- [...]
-</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="654.4" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-26)">│</text><text
class="breeze-release-management-generate-issue-content-providers-r4" x="24.4"
y="654.4" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-26)">-</text><text
class="breeze-release-management-generate-issue-content-providers-r4" [...]
-</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="678.8" textLength="1464"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-27)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-generate-issue-content-providers-r1" x="1464"
y="678.8" textLength="12.2" clip-path="url(#breeze-release-management-generate
[...]
+</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="581.2" textLength="1464"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-23)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-generate-issue-content-providers-r1" x="1464"
y="581.2" textLength="12.2" clip-path="url(#breeze-release-management-generate
[...]
+</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="605.6" textLength="24.4"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-24)">╭─</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="24.4"
y="605.6" textLength="195.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-24)"> Common options </text><text
class="breeze-release-management-gener [...]
+</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="630" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)">│</text><text
class="breeze-release-management-generate-issue-content-providers-r4" x="24.4"
y="630" textLength="12.2"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)">-</text><text
class="breeze-release-management-generate-issue-content-providers-r4" x="3
[...]
+</text><text
class="breeze-release-management-generate-issue-content-providers-r5" x="0"
y="654.4" textLength="1464"
clip-path="url(#breeze-release-management-generate-issue-content-providers-line-26)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text
class="breeze-release-management-generate-issue-content-providers-r1" x="1464"
y="654.4" textLength="12.2" clip-path="url(#breeze-release-management-generate
[...]
</text>
</g>
</g>
diff --git
a/images/breeze/output_release-management_generate-issue-content-providers.txt
b/images/breeze/output_release-management_generate-issue-content-providers.txt
index 896b36c77a..aa42535eac 100644
---
a/images/breeze/output_release-management_generate-issue-content-providers.txt
+++
b/images/breeze/output_release-management_generate-issue-content-providers.txt
@@ -1 +1 @@
-7361238411e5097d70e0e359b5246a2a
+8109ff3d327a5e6921b15402f9c15bb6
