jason810496 opened a new pull request, #73095: URL: https://github.com/apache/airflow/pull/73095
- depends-on: [ci/image-reuse/04-publisher](https://github.com/jason810496/airflow/compare/ci%2Fimage-reuse%2F03-artifact-selection...ci%2Fimage-reuse%2F04-publisher) - base-branch: `ci/image-reuse/04-publisher` Part 5/5 of the CI image reuse series. ## Why Avoid rebuilding compatible images while testing the current PR checkout. ## How Enable eligible PR lookups, restore selections through a dedicated script, and build a current-run fallback on misses — including when the selected image's own download is too slow to be worth finishing. Preserve full builds for canaries and dependency upgrades. ## Series impact (audit draft) [Sept 6–12, 2026 UTC](https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+merged%3A2026-09-06..2026-09-12) PR census and CI timing: | Metric | Figure | | --- | ---: | | Merged PRs / into main | 364 / 240 | | No dependency-manifest / CI-image / prod-image input changes | 218 (90.8%) / 141 (58.8%) / 47 (19.6%) | | Full AMD/ARM census execution | 17,175.6 h | | Image-prep latency, today → 100% hit rate (CI, Py3.10) | 19.74 → 5.77 min (−71%) | | Latency break-even hit rate | ~3.5% | | Compute ceiling, cost-free → cost-priced replacement download | 106.4 h → ~65.7 h/week | | Publisher budget proxy (daily-only cadence) | 8.0 h/run → 55.7 h/week | | Compute break-even hit rate (daily-only publisher) | ~85% | The purpose is to avoid rebuilding the CI image when a PR doesn't change package dependencies — currently, even a PR that only touches the API server or the UI still triggers a full CI image rebuild on every push — and real hit rate and end-to-end PR latency remain unmeasured pending a pilot. - **Happy path:** this PR is what actually removes the build-job wait on a hit, cutting image-prep latency by ~71% (19.74 → 5.77 min for the dominant Python 3.10 CI case). A miss only costs about 30 extra seconds, so even a low cache-hit rate (as little as ~3.5%) already makes this a net time saver overall. - **Smart fallback:** the consumer probes transfer speed over the first 30 seconds of a selected image's download; if that regresses below the upstream reuse floor (which `restore_selected_image.sh` inherits automatically), it falls back to a regular image build — today's existing behavior — instead of waiting out a degraded transfer. - **Daily image publish workflow:** a centralized periodic workflow (ci/image-reuse/04-publisher) publishes CI and production images, keyed by a fingerprint of build inputs, as GitHub Actions artifacts this PR's lookups consume. - **Retention:** a published artifact is only selected if produced within the last 48 hours, and is retained for 7 days. --- ##### Was generative AI tooling used to co-author this PR? - [x] Yes, with help of Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions). Generated-by: Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
