potiuk opened a new pull request, #67669: URL: https://github.com/apache/airflow/pull/67669
## Summary Provider `README.rst` files are regenerated from `pyproject.toml` at distribution-build time (`dev/breeze/src/airflow_breeze/utils/packages.py:_prepare_readme_file`). Between releases, dependency bumps in `pyproject.toml` do not propagate to the committed README, leading to silent drift that has to be fixed manually — as in #67554 (celery, cncf/kubernetes, edge3, git, openlineage). This adds a prek hook (`sync-provider-readme`) that re-renders the `Requirements` table directly from the current `pyproject.toml` whenever the latter changes, catching the drift in pre-commit and CI instead of waiting for the next release build. Only the table block is touched — surrounding prose, the cross-provider table, and the rest of the README are left alone. It also brings 7 currently-stale provider READMEs into sync as the hook output: - `providers/apache/beam/README.rst` - `providers/apache/spark/README.rst` - `providers/fab/README.rst` - `providers/ibm/mq/README.rst` - `providers/microsoft/azure/README.rst` - `providers/mysql/README.rst` - `providers/tableau/README.rst` These are the same class of drift #67554 fixed manually for the other 5 providers. ## Why #67554 was a fix for a symptom — and the same drift kept recurring across providers because nothing in the CI / pre-commit pipeline catches a dependency bump that forgets to regenerate the matching README. The new hook closes that gap: every future `providers/*/pyproject.toml` change runs through `sync-provider-readme`, the Requirements table is re-rendered, and any diff fails the hook so the contributor stages it. ## Test plan - [x] Hook is registered (`prek run sync-provider-readme --all-files` → Passed once the 7 stale READMEs are synced). - [x] Idempotent on the synced state — running over all 102 provider `pyproject.toml` files after the fixes produces no diff. - [x] Catches drift end-to-end — tampering with a single version in `providers/celery/README.rst` makes the hook restore it cleanly. - [x] Skips providers with no `Requirements` table or no `[project].dependencies` (the message goes to stderr so it's visible without failing). - [ ] CI for this PR is green against the latest `main`. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 4.7, 1M context) Generated-by: Claude Code (Opus 4.7, 1M context) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
