This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-8-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit b9ef5a089837e4d911c7a80d4ab5449946fbb6d8 Author: Jarek Potiuk <[email protected]> AuthorDate: Thu Dec 28 21:02:27 2023 +0100 Improve generation of "Status of testing" issue (#36470) This PR improves/simplifies the process of issue generation when provider package rc candidates are prepared for voting. It improves the commmand to generate the issue and makes it simpler (less copy&paste) to create such issue, the issue also does not use the "Meta" template and gets the right labels assigned automatically. Recent changes that automatically derive the suffix from PyPI packages prepared, removed the need of passing `--suffix` as parameter. In all cases the right rc* suffix will be automatically added during issue generation based on the version of package being prepared. The process has been updated and command simplified by removing the `--suffix` flag. When the issue is prepared, we display the issue in terminal and asked the release manager to create the issue by copy&pasting the issue content and title to a new issue, but that required a few copy&pastes and opening new Issue via "Meta" task type. This PR simplifies it a bit by not only displaying the content but also generating a URL that can be either copy&pasted to browser URL field or just Cmd+clicked if your terminal allows that. Issue created this way does not have the "Body" field header and has the labels properly assigned including a dedicated "testing status" label that is used to gether stats for past "status" issues. The advice for release manager has been improved (the comment generated had some missing end of sentence and it should be now clearer on how to iterate during issue generation if you want to remove some PRs from the generated issue content. (cherry picked from commit 5d88f6f9a4c7140c1da9db47aab7caf2d4c5f453) --- dev/README_RELEASE_PROVIDER_PACKAGES.md | 50 ++++++++++++++++++---- .../commands/release_management_commands.py | 47 +++++++++++++++----- .../commands/release_management_commands_config.py | 1 - .../provider_issue_TEMPLATE.md.jinja2 | 8 +--- ...management_generate-issue-content-providers.svg | 18 +++----- ...management_generate-issue-content-providers.txt | 2 +- 6 files changed, 88 insertions(+), 38 deletions(-) diff --git a/dev/README_RELEASE_PROVIDER_PACKAGES.md b/dev/README_RELEASE_PROVIDER_PACKAGES.md index cbf645d649..9609c20873 100644 --- a/dev/README_RELEASE_PROVIDER_PACKAGES.md +++ b/dev/README_RELEASE_PROVIDER_PACKAGES.md @@ -529,28 +529,60 @@ git push --set-upstream origin "${branch}" ## Prepare issue in GitHub to keep status of testing -Create a GitHub issue with the content generated via manual -execution of the script below. You will use link to that issue in the next step. You need a GITHUB_TOKEN -set as your environment variable. +Create a GitHub issue with the content generated via manual execution of the command below. You will use +link to that issue in the next step. -You can also pass the token as `--github-token` option in the script. -You can also pass list of PR to be excluded from the issue with `--excluded-pr-list`. +```shell script +cd "${AIRFLOW_REPO_ROOT}" + +breeze release-management generate-issue-content-providers --only-available-in-dist +``` + +GitHub API uses rate limiting that is based on the public IP address of client if you do not authenticate +with GitHub, so when you retrieve bigger number of PRs or when you are behind NAT and share your public +IP address with many other Anonymous GitHub API users, issue retrieval will be halted and your API calls +might slow down to a crawl, you will need then a GITHUB_TOKEN set as your +environment variable or pass the token as `--github-token` option in the script. ```shell script cd "${AIRFLOW_REPO_ROOT}" +breeze release-management generate-issue-content-providers --only-available-in-dist --github-token TOKEN +``` + +or + +```shell script +cd "${AIRFLOW_REPO_ROOT}" +export GITHUB_TOKEN=TOKEN breeze release-management generate-issue-content-providers --only-available-in-dist ``` -You can also generate the token by following -[this link](https://github.com/settings/tokens/new?description=Read%20sssues&scopes=repo:status) +You can generate the token by following +[this link](https://github.com/settings/tokens/new?description=Read%20issues&scopes=repo:status). Since it is easy to generate such token, by following the link, it is recommended to +generate a new token for each release and delete it once you've generated the issue. + +If you see in the output that some of the PRs are just "noise" (i.e. there is no need to verify them +as they are misc/documentation kind of changes that have no impact on the actual installation of +the provider or the code of the provider, can optionally pass list of PR to be excluded from +the issue with `--excluded-pr-list`. This might limit the scope of verification. Some providers +might disappear from the list and list of authors that will be pinged in the generated issue. -If you are preparing release for RC2/RC3 candidates, you should add `--suffix` parameter: +You can repeat that and regenerate the issue content until you are happy with the generated issue. ```shell script -breeze release-management generate-issue-content-providers --only-available-in-dist --suffix rc2 +cd "${AIRFLOW_REPO_ROOT}" + +breeze release-management generate-issue-content-providers --only-available-in-dist --github-token TOKEN \ + --excluded-pr-list PR_NUMBER1,PR_NUMBER2 ``` +It's also OK to manually modify the content of such generated issue before actually creating the +issue. There is a comment generated with NOTE TO RELEASE MANAGER about this in the issue content. +Hit Preview button on "create issue" screen before creating it to verify how it will look like +for the contributors. + + ## Prepare voting email for Providers release candidate diff --git a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py index 8d1ee6eaa6..7525a353ff 100644 --- a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py +++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py @@ -29,7 +29,7 @@ from copy import deepcopy from datetime import datetime from pathlib import Path from subprocess import DEVNULL -from typing import IO, TYPE_CHECKING, Any, Generator, NamedTuple +from typing import IO, TYPE_CHECKING, Any, Generator, Iterable, NamedTuple import click from rich.progress import Progress @@ -1571,6 +1571,25 @@ def get_prs_for_package(provider_id: str) -> list[int]: return prs +def create_github_issue_url(title: str, body: str, labels: Iterable[str]) -> str: + """ + Creates URL to create the issue with title, body and labels. + :param title: issue title + :param body: issue body + :param labels: labels for the issue + :return: URL to use to create the issue + """ + from urllib.parse import quote + + quoted_labels = quote(",".join(labels)) + quoted_title = quote(title) + quoted_body = quote(body) + return ( + f"https://github.com/apache/airflow/issues/new?labels={quoted_labels}&"; + f"title={quoted_title}&body={quoted_body}" + ) + + @release_management.command( name="generate-issue-content-providers", help="Generates content for issue to test the release." ) @@ -1592,14 +1611,12 @@ def get_prs_for_package(provider_id: str) -> list[int]: is_flag=True, help="Only consider package ids with packages prepared in the dist folder", ) [email protected]("--suffix", default="rc1", help="Suffix to add to the version prepared") @argument_provider_packages def generate_issue_content_providers( disable_progress: bool, excluded_pr_list: str, github_token: str, only_available_in_dist: bool, - suffix: str, provider_packages: list[str], ): import jinja2 @@ -1674,7 +1691,7 @@ def generate_issue_content_providers( provider_package_id=provider_id, pypi_package_name=provider_yaml_dict["package-name"], pr_list=pull_request_list, - suffix=package_suffix if package_suffix else suffix, + suffix=package_suffix if package_suffix else "", ) template = jinja2.Template( (Path(__file__).parents[1] / "provider_issue_TEMPLATE.md.jinja2").read_text() @@ -1688,19 +1705,29 @@ def generate_issue_content_providers( get_console().print() get_console().print() get_console().print( - "Issue title: [yellow]Status of testing Providers that were " + "Issue title: [warning]Status of testing Providers that were " f"prepared on {datetime.now():%B %d, %Y}[/]" ) get_console().print() - syntax = Syntax(issue_content, "markdown", theme="ansi_dark") - get_console().print(syntax) - get_console().print() + issue_content += "\n" users: set[str] = set() for provider_info in providers.values(): for pr in provider_info.pr_list: users.add("@" + pr.user.login) - get_console().print("All users involved in the PRs:") - get_console().print(" ".join(users)) + issue_content += f"All users involved in the PRs:\n{' '.join(users)}" + syntax = Syntax(issue_content, "markdown", theme="ansi_dark") + get_console().print(syntax) + url_to_create_the_issue = create_github_issue_url( + title=f"Status of testing Providers that were prepared on {datetime.now():%B %d, %Y}", + body=issue_content, + labels=["testing status", "kind:meta"], + ) + get_console().print() + get_console().print( + "[info]You can prefill the issue by copy&pasting this link to browser " + "(or Cmd+Click if your terminal supports it):\n" + ) + print(url_to_create_the_issue) def get_all_constraint_files(refresh_constraints: bool, python_version: str) -> None: diff --git a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py index 1dfa78f3aa..77fc83650c 100644 --- a/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py +++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands_config.py @@ -234,7 +234,6 @@ RELEASE_MANAGEMENT_PARAMETERS: dict[str, list[dict[str, str | list[str]]]] = { "--excluded-pr-list", "--github-token", "--only-available-in-dist", - "--suffix", ], } ], diff --git a/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2 b/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2 index 8009549772..c519048723 100644 --- a/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2 +++ b/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2 @@ -20,11 +20,7 @@ Those are providers that require testing as there were some substantial changes NOTE TO RELEASE MANAGER: -Please move here the providers that have doc-only changes or for which changes are trivial, and -you could assess that they are OK. In case - -The providers are automatically installed on Airflow 2.3 and latest `main` during the CI, so we know they -are installable. Also, all classes within the providers are imported during the CI run so we know all -providers can be imported. +You can move here the providers that have doc-only changes or for which changes are trivial, and +you could assess that they are OK. --> diff --git a/images/breeze/output_release-management_generate-issue-content-providers.svg b/images/breeze/output_release-management_generate-issue-content-providers.svg index 46fce9329b..e8e193e1d1 100644 --- a/images/breeze/output_release-management_generate-issue-content-providers.svg +++ b/images/breeze/output_release-management_generate-issue-content-providers.svg @@ -1,4 +1,4 @@ -<svg class="rich-terminal" viewBox="0 0 1482 733.1999999999999" xmlns="http://www.w3.org/2000/svg";> +<svg class="rich-terminal" viewBox="0 0 1482 708.8" xmlns="http://www.w3.org/2000/svg";> <!-- Generated with Rich https://www.textualize.io --> <style> @@ -43,7 +43,7 @@ <defs> <clipPath id="breeze-release-management-generate-issue-content-providers-clip-terminal"> - <rect x="0" y="0" width="1463.0" height="682.1999999999999" /> + <rect x="0" y="0" width="1463.0" height="657.8" /> </clipPath> <clipPath id="breeze-release-management-generate-issue-content-providers-line-0"> <rect x="0" y="1.5" width="1464" height="24.65"/> @@ -123,12 +123,9 @@ <clipPath id="breeze-release-management-generate-issue-content-providers-line-25"> <rect x="0" y="611.5" width="1464" height="24.65"/> </clipPath> -<clipPath id="breeze-release-management-generate-issue-content-providers-line-26"> - <rect x="0" y="635.9" width="1464" height="24.65"/> - </clipPath> </defs> - <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="731.2" rx="8"/><text class="breeze-release-management-generate-issue-content-providers-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management generate-issue-content-providers</text> + <rect fill="#292929" stroke="rgba(255,255,255,0.35)" stroke-width="1" x="1" y="1" width="1480" height="706.8" rx="8"/><text class="breeze-release-management-generate-issue-content-providers-title" fill="#c5c8c6" text-anchor="middle" x="740" y="27">Command: release-management generate-issue-content-providers</text> <g transform="translate(26,22)"> <circle cx="0" cy="0" r="7" fill="#ff5f57"/> <circle cx="22" cy="0" r="7" fill="#febc2e"/> @@ -161,11 +158,10 @@ </text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="508" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-20)">│</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="366" y="508" textLength="1073.6" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-20)">https://github.com/settings/tokens/new?description=Read%20sssues&scopes=repo:status [...] </text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="532.4" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-21)">│</text><text class="breeze-release-management-generate-issue-content-providers-r6" x="366" y="532.4" textLength="1073.6" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-21)">(TEXT)              [...] </text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="556.8" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-22)">│</text><text class="breeze-release-management-generate-issue-content-providers-r4" x="24.4" y="556.8" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-22)">-</text><text class="breeze-release-management-generate-issue-content-providers-r4" [...] -</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="581.2" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-23)">│</text><text class="breeze-release-management-generate-issue-content-providers-r4" x="24.4" y="581.2" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-23)">-</text><text class="breeze-release-management-generate-issue-content-providers-r4" [...] -</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="605.6" textLength="1464" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-24)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="1464" y="605.6" textLength="12.2" clip-path="url(#breeze-release-management-generate [...] -</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="630" textLength="24.4" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)">╭─</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="24.4" y="630" textLength="195.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)"> Common options </text><text class="breeze-release-management-generate- [...] -</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="654.4" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-26)">│</text><text class="breeze-release-management-generate-issue-content-providers-r4" x="24.4" y="654.4" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-26)">-</text><text class="breeze-release-management-generate-issue-content-providers-r4" [...] -</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="678.8" textLength="1464" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-27)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="1464" y="678.8" textLength="12.2" clip-path="url(#breeze-release-management-generate [...] +</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="581.2" textLength="1464" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-23)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="1464" y="581.2" textLength="12.2" clip-path="url(#breeze-release-management-generate [...] +</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="605.6" textLength="24.4" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-24)">╭─</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="24.4" y="605.6" textLength="195.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-24)"> Common options </text><text class="breeze-release-management-gener [...] +</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="630" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)">│</text><text class="breeze-release-management-generate-issue-content-providers-r4" x="24.4" y="630" textLength="12.2" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-25)">-</text><text class="breeze-release-management-generate-issue-content-providers-r4" x="3 [...] +</text><text class="breeze-release-management-generate-issue-content-providers-r5" x="0" y="654.4" textLength="1464" clip-path="url(#breeze-release-management-generate-issue-content-providers-line-26)">╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯</text><text class="breeze-release-management-generate-issue-content-providers-r1" x="1464" y="654.4" textLength="12.2" clip-path="url(#breeze-release-management-generate [...] </text> </g> </g> diff --git a/images/breeze/output_release-management_generate-issue-content-providers.txt b/images/breeze/output_release-management_generate-issue-content-providers.txt index 896b36c77a..aa42535eac 100644 --- a/images/breeze/output_release-management_generate-issue-content-providers.txt +++ b/images/breeze/output_release-management_generate-issue-content-providers.txt @@ -1 +1 @@ -7361238411e5097d70e0e359b5246a2a +8109ff3d327a5e6921b15402f9c15bb6
