villebro opened a new pull request, #45253: URL: https://github.com/apache/superset/pull/45253
### SUMMARY `RELEASING/changelog.py` looks up every PR on GitHub (title, labels, author login, and whether the commit touched migrations). For a large release this takes hours: the 7.0.0 changelog covers about 3,800 PRs, which is more requests than the hourly rate limit allows. Until now nothing was saved between runs, so any network error, or a single PR that GitHub no longer serves, crashed the run and the next attempt started over from the beginning. This PR makes the script resumable and stops it from crashing on missing PRs: - **Persistent cache.** Each PR's details are appended to a JSON-lines file as soon as they're fetched and loaded on startup, so rerunning the same command resumes where the previous run stopped. The file is set with `--cache_file`, defaults to the system temp directory, and an empty value turns caching off. A truncated last line left by a killed run is dropped on load. - **Fewer requests.** The repo object was fetched again for every PR, and twice for each new author. It's now fetched once, which cuts requests per PR from up to four to two. - **Missing PRs.** When GitHub returns 404 for a PR (for example, one hidden along with a suspended account), the entry is built from the commit instead: the subject as title, the git author name, and no labels. The script prints a warning so the line can be checked by hand. The existing code already expected this case (`pr_info.title if pr_info else git_log.message`), but nothing ever returned `None`. - **Auth deprecation.** The token is passed as `auth=Auth.Token(...)` instead of the positional argument PyGithub deprecated. The changelog section of `RELEASING/README.md` documents the cache and how to reuse a GitHub CLI token. Output is unchanged for PRs that exist. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A ### TESTING INSTRUCTIONS Tested against the live GitHub API on small ranges of the `7.0` branch, with PyGithub 2.10.0 and `-W error::DeprecationWarning`: - 4 PRs, original vs. new script: identical output. Requests went from 16 to 9 (counted from urllib3 debug logs). A second run with the cache warm made 0 requests and produced identical output. - A simulated network error on the 3rd PR: the 2 PRs fetched before it stayed in the cache, and the rerun fetched only the 2 missing PRs with identical output. This also held with a truncated line appended to the cache. - A range containing #38729, which GitHub returns 404 for: the run completes, warns about #38729, and lists it from its commit. A second run makes no requests. - `--risk` output matches the original script. To try it: ``` cd RELEASING export GITHUB_TOKEN=$(gh auth token) python changelog.py --previous_version 6.1.0 --current_version 7.0 changelog > /tmp/changelog.md ``` Interrupt it with Ctrl-C partway through, rerun the same command, and check that it prints `Loaded N cached pull requests from ...` and carries on from there. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
