This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7bdb60e68cd6 [SPARK-58002][INFRA] Refresh CI build images on a schedule
7bdb60e68cd6 is described below
commit 7bdb60e68cd6107a212201327ec6b3ec20297345
Author: Nicholas Chammas <[email protected]>
AuthorDate: Fri Jul 17 08:22:38 2026 +0800
[SPARK-58002][INFRA] Refresh CI build images on a schedule
### What changes were proposed in this pull request?
Rebuild the various images used on CI on a schedule.
The schedule is set to once a day.
### Why are the changes needed?
All of our CI build images are based on `ubuntu:noble`. They are only
refreshed if we push changes to the Dockerfiles on `master`.
https://github.com/apache/spark/blob/d386172e690188b659cf89894e1eca4485c05c57/.github/workflows/build_infra_images_cache.yml#L31-L48
On the other hand, however, `ubuntu:noble` is updated roughly once a month:
```sh
$ curl -s
"https://hub.docker.com/v2/repositories/library/ubuntu/tags?name=noble&page_size=10";
\
| jq -r '.results[] | "\(.tag_last_pushed[:10]) \(.name)"'
2026-07-02 noble-20260610
2026-07-02 noble
2026-06-02 noble-20260509.1
2026-04-15 noble-20260410
2026-04-07 noble-20260324
2026-03-19 noble-20260217
2026-02-18 noble-20260210.1
2026-01-19 noble-20260113
2025-11-15 noble-20251013
2025-10-13 noble-20251001
```
This means that there are several updates to the base image that we have
not incorporated. So when CI runs, it cannot pull from an existing cache and
has to build the images from scratch. This takes about [50
minutes](https://github.com/nchammas/spark/actions/runs/28833909584/job/85513889893)!
With an up-to-date cache, this drops down to [~1
minute](https://github.com/nchammas/spark/actions/runs/28836719647/job/85522299879).
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
I tested this with a change to `build_and_test.yml` on one of my branches,
but I believe the correct fix is to ensure the that the base images are
refreshed on a schedule.
```yml
cache-from: |
type=registry,ref=ghcr.io/${{ github.repository_owner
}}/spark/apache-spark-github-action-image-cache:${{ inputs.branch }}
type=registry,ref=ghcr.io/apache/spark/apache-spark-github-action-image-cache:${{
inputs.branch }}
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner
}}/spark/apache-spark-github-action-image-cache:${{ inputs.branch }},mode=max
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57083 from nchammas/image-refresh-schedule.
Authored-by: Nicholas Chammas <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
---
.github/workflows/build_infra_images_cache.yml | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/.github/workflows/build_infra_images_cache.yml
b/.github/workflows/build_infra_images_cache.yml
index d195f07c31ff..0ad4e99a0a9d 100644
--- a/.github/workflows/build_infra_images_cache.yml
+++ b/.github/workflows/build_infra_images_cache.yml
@@ -20,6 +20,13 @@
name: Build / Cache base image
on:
+ # Rebuild cache on a schedule to keep it warm despite base image drift.
+ # The `ubuntu:noble` tag is re-pointed to a new image roughly monthly,
+ # which invalidates all cached layers built on top of it. Without a
+ # scheduled rebuild, the cache goes stale between Dockerfile changes
+ # and every commit on every PR spends ~50 minutes building base images!
+ schedule:
+ - cron: '0 4 * * *'
# Run jobs when a commit is merged
push:
branches:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
