jedcunningham commented on code in PR #67988: URL: https://github.com/apache/airflow/pull/67988#discussion_r3359218253
########## dev/README_RELEASE_HELM_CHART.md: ########## @@ -986,14 +986,45 @@ The `index.yaml` file on the Airflow website allows users to easily add the Apac helm repo add apache-airflow https://airflow.apache.org ``` +The index points **every** version -- including the one just released -- at +`archive.apache.org`, never `downloads.apache.org`. `downloads.apache.org` keeps only the +latest release and is fronted by a CDN (Fastly) whose cache can serve a stale `200` that turns +into a real `404` once a version is removed -- exactly the failure that strands users on +removed-version URLs. `archive.apache.org` is append-only: once a version lands there it is +never removed, so those URLs resolve forever, no matter how stale a cached copy of `index.yaml` +is. + +This matters because `index.yaml` is itself served from `airflow.apache.org`, which is *also* +behind Fastly, so a freshly published index can be served stale for the cache TTL. That is +harmless **as long as every URL the index has ever contained stays resolvable** -- which the +archive-only policy guarantees for every superseded version. The one case it does not cover +automatically is the **just-released** version: `archive.apache.org` lags `downloads.apache.org` +(the archive sync is [documented](https://infra.apache.org/release-distribution.html) as +happening "about a day" after a release first appears on downloads -- but that is the worst +case; in practice the sync is usually within minutes, at most a few hours), so its archive URL +can 404 during that window. +**Do not publish the index entry for the new version until that version is live on archive** -- +the steps below block until it is. (Removing superseded versions from `dist/release`, by +contrast, needs no waiting -- they have been on archive since their own release.) + To update `index.yaml` for the newly released chart version, follow these steps: ```shell git clone https://github.com/apache/airflow-site.git airflow-site cd airflow-site + +# archive.apache.org lags the release (worst case ~1 day; usually minutes, at most hours). Block until the +# new version is actually downloadable from archive before referencing it in the index -- +# otherwise the index could advertise an archive URL that 404s until the sync catches up. +until curl -fsI "https://archive.apache.org/dist/airflow/helm-chart/${VERSION}/airflow-${VERSION}.tgz"; >/dev/null; do + echo "airflow-${VERSION}.tgz not on archive.apache.org yet; waiting 10 min..." + sleep 600 +done Review Comment: It might be nice to have a positive message when it does succeed, otherwise you come back and see it exited, and if you are lucky you show the exit code in your prompt to know it succeeded vs failed somehow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
