cmcfarlen commented on code in PR #13063: URL: https://github.com/apache/trafficserver/pull/13063#discussion_r3972129803
########## tools/changelog/changelog.py: ########## @@ -0,0 +1,509 @@ +#!/usr/bin/env python3 +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Generate a changelog from merged PRs in a GitHub milestone. + +Usage: + uv run --project tools/changelog python tools/changelog/changelog.py \ + -o apache -r trafficserver -m 10.2.0 + +Output is written to stdout in the format used by CHANGELOG-* files: + + Changes with Apache Traffic Server 10.2.0 + #11945 - Make directory operations methods on `Directory` + #12026 - Static link opentelemetry-cpp libraries to otel_tracer plugin + ... + +To generate a changelog file for a release: + + uv run --project tools/changelog python tools/changelog/changelog.py \ + -o apache -r trafficserver -m 10.2.0 > CHANGELOG-10.2.0 + +Use --doc to include extra metadata (merge commit SHA, PR body, labels) for each +PR, useful for generating release documentation. With --from-git the body is the +commit message body rather than a PR body: + + uv run --project tools/changelog python tools/changelog/changelog.py \ + -o apache -r trafficserver -m 10.2.0 --doc --format yaml > changelog.yaml + +For security releases whose fixes land directly on a release branch without +public pull requests (and therefore never appear in a milestone), use +--from-git to source the changelog from a git commit range. This is merged +with the milestone PRs (deduplicated by PR number) so the result includes both +the direct-to-branch commits and any milestone PRs. The -m value is still used +as the version label in the output: + + uv run --project tools/changelog python tools/changelog/changelog.py \ + -o apache -r trafficserver -m 10.1.4 \ + --from-git 10.1.3..upstream/10.1.x + +Requires a GitHub token via GH_TOKEN env var or -a flag to avoid rate limits. +""" + +import argparse +import json +import os +import re +import subprocess +import sys + +import httpx + +try: + import yaml +except ImportError: + yaml = None + +API_URL = "https://api.github.com"; + + +def gh_cli_available() -> bool: + try: + subprocess.run(["gh", "--version"], capture_output=True, check=True) + return True + except (FileNotFoundError, subprocess.CalledProcessError): + return False + + +def changelog_via_gh(owner: str, repo: str, milestone: str, verbose: bool, doc: bool) -> list[dict]: + """Use the gh CLI to fetch milestone PRs (avoids API rate limits).""" + milestone_id = None + result = subprocess.run( + ["gh", "api", f"/repos/{owner}/{repo}/milestones?state=all", "--paginate"], + capture_output=True, + text=True, + ) + if result.returncode != 0: + print(f"gh api error: {result.stderr}", file=sys.stderr) + sys.exit(1) + + milestones = json.loads(result.stdout) + for ms in milestones: + if ms["title"] == milestone: + milestone_id = ms["number"] + break + + if milestone_id is None: + print(f"Milestone not found: {milestone}", file=sys.stderr) + sys.exit(1) + + print(f"Looking for issues from Milestone {milestone}", file=sys.stderr) + + changelog = [] + page = 1 + while True: + print(f"Page {page}", file=sys.stderr) + result = subprocess.run( + [ + "gh", + "api", + f"/repos/{owner}/{repo}/issues?milestone={milestone_id}&state=closed&page={page}&per_page=100", + ], + capture_output=True, + text=True, + ) + if result.returncode != 0: + print(f"gh api error: {result.stderr}", file=sys.stderr) + sys.exit(1) + + issues = json.loads(result.stdout) + if not issues: + break + + for issue in issues: + number = issue["number"] + title = issue["title"] + if verbose: + print(f"Issue #{number} - {title} ", end="", file=sys.stderr) + + if "pull_request" not in issue: + if verbose: + print("not a PR.", file=sys.stderr) + continue + + if not _gh_is_merged(owner, repo, number): + if verbose: + print("not merged.", file=sys.stderr) + continue + + if verbose: + print("added.", file=sys.stderr) + + entry: dict = {"number": number, "title": title} + if doc: + labels = [label["name"] for label in issue.get("labels", [])] + entry["labels"] = labels + pr_detail = subprocess.run( + ["gh", "api", f"/repos/{owner}/{repo}/pulls/{number}"], + capture_output=True, + text=True, + ) + if pr_detail.returncode != 0: + print(f"gh api error fetching PR #{number}: {pr_detail.stderr.strip()}", file=sys.stderr) + sys.exit(1) + pr_data = json.loads(pr_detail.stdout) + entry["sha"] = pr_data.get("merge_commit_sha", "") + entry["body"] = pr_data.get("body", "") or "" + changelog.append(entry) + + page += 1 + + return changelog + + +def changelog_via_api( + owner: str, + repo: str, + milestone: str, + token: str | None, + verbose: bool, + doc: bool, +) -> list[dict]: + """Use httpx to call the GitHub REST API directly.""" + headers = { + "Accept": "application/vnd.github.v3+json", + "User-Agent": "ATS-Changelog-Tool", + } + if token: + headers["Authorization"] = f"Bearer {token}" + + with httpx.Client(base_url=API_URL, headers=headers, timeout=30) as client: + milestone_id = _lookup_milestone(client, owner, repo, milestone) + if milestone_id is None: + print(f"Milestone not found: {milestone}", file=sys.stderr) + sys.exit(1) + + print(f"Looking for issues from Milestone {milestone}", file=sys.stderr) + + changelog = [] + page = 1 + while True: + print(f"Page {page}", file=sys.stderr) + resp = client.get( + f"/repos/{owner}/{repo}/issues", + params={ + "milestone": milestone_id, + "state": "closed", + "page": page, + "per_page": 100, + }, + ) + _check_rate_limit(resp) + resp.raise_for_status() + issues = resp.json() + + if not issues: + break + + for issue in issues: + number = issue["number"] + title = issue["title"] + if verbose: + print(f"Issue #{number} - {title} ", end="", file=sys.stderr) + + if "pull_request" not in issue: + if verbose: + print("not a PR.", file=sys.stderr) + continue + + if not _is_merged(client, owner, repo, number): + if verbose: + print("not merged.", file=sys.stderr) + continue + + if verbose: + print("added.", file=sys.stderr) + + entry: dict = {"number": number, "title": title} + if doc: + labels = [label["name"] for label in issue.get("labels", [])] + entry["labels"] = labels + pr_resp = client.get(f"/repos/{owner}/{repo}/pulls/{number}") + _check_rate_limit(pr_resp) + pr_resp.raise_for_status() + pr_data = pr_resp.json() + entry["sha"] = pr_data.get("merge_commit_sha", "") + entry["body"] = pr_data.get("body", "") or "" + changelog.append(entry) Review Comment: Leaving this as-is. `/pulls/{n}` does carry the merged state, so the extra call is avoidable in `--doc` mode, but taking it out means merged-ness is determined one way with `--doc` and another without it, in both the `gh` and `httpx` paths — four combinations of a check that decides whether a PR appears in a release changelog. The rate-limit motivation does not apply at this scale either: 10.2.0 was generated with this tool at roughly twice the PR count of a normal release and did not come close to the authenticated limit. Not worth the extra path. ########## doc/developer-guide/release-process/index.en.rst: ########## @@ -67,8 +67,8 @@ Build #. Generate or update the CHANGELOG for the next release. :: - ./tools/git/changelog.pl -o apache -r trafficserver -m X.Y.Z > - CHANGELOG-X.Y.Z + uv run --project tools/changelog python tools/changelog/changelog.py \ Review Comment: That command does not work. `pyproject.toml` has no `[build-system]`, so uv treats this as a virtual project — `uv.lock` records `source = { virtual = "." }` — which installs the dependencies but never builds the package, so the `[project.scripts]` entry point is never created: ``` $ uv run --project tools/changelog changelog --help Installed 7 packages in 6ms error: Failed to spawn: `changelog` Caused by: No such file or directory (os error 2) ``` The documented `python tools/changelog/changelog.py` form does work, and it matches `tools/traffic_grapher`, which has the same shape — no `[build-system]`, with a `[project.scripts]` entry that is likewise inert. Making the console script real would mean adding a build backend to an in-tree script; I would rather keep it consistent with the sibling tool than diverge here. Happy to do it if the preference is otherwise. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
