This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c8e8f8642085 [SPARK-60867][INFRA] Run the GitHub Pages deployment
daily instead of on every master push
c8e8f8642085 is described below
commit c8e8f86420855405de636098e6f6b95c7659aa1d
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Nov 4 06:32:25 2026 -0800
[SPARK-60867][INFRA] Run the GitHub Pages deployment daily instead of on
every master push
### What changes were proposed in this pull request?
This PR changes the trigger of the `GitHub Pages deployment` workflow
(`.github/workflows/pages.yml`) from every push to `master` to a daily schedule
at 12:30 UTC. It also adds `workflow_dispatch` so that committers can run it on
demand.
The job itself, the `docs preview` concurrency group, and the
`github.repository == 'apache/spark'` guard are unchanged.
12:30 UTC was chosen for the following reasons.
- The job runs in the
`apache-spark-github-action-image-docs-cache:master-static` image, which
`build_infra_images_cache.yml` refreshes daily at 04:00 UTC. In the week of
2026-10-27 to 11-02, those refresh runs finished by 05:32 UTC at the latest.
- It is not at the start of an hour.
[GitHub](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#schedule)
notes that scheduled runs can be delayed, or even dropped, under the high load
at the start of every hour. That is also when most other scheduled workflows of
this repository start.
- It is one of the quietest times of `apache/spark` CI. In the same week,
an average of 20.8 jobs were running from 12:35 to 13:05 UTC, against 31.7 over
the whole week.
This is a `master`-only change. Scheduled workflows run only from the
default branch, and the `pages.yml` files in `branch-4.0` to `branch-4.3` and
`branch-4.x` only react to pushes to `master`, so they never run.
### Why are the changes needed?
To reduce the GitHub Actions usage of `apache/spark`. The [ASF GitHub
Actions policy](https://infra.apache.org/github-actions-policy.html) limits a
project to 250,000 runner minutes per week, and `apache/spark` used 315,939
minutes in the week of 2026-10-27 to 11-02 (UTC).
In that week, `master` had 113 pushes. This workflow ran for 89 of them,
and the other 24 were cancelled while waiting in the `docs preview` concurrency
group. Each run took about 20 minutes, 1,794 runner minutes in total. A daily
run takes about 150 minutes per week, so this saves about 1,650 minutes per
week.
https://apache.github.io/spark/ is still updated every day, and
`workflow_dispatch` can update it sooner when needed.
### Does this PR introduce _any_ user-facing change?
No. This is a CI-only change. The development documentation at
https://apache.github.io/spark/ is now updated once a day instead of after
every push to `master`.
### How was this patch tested?
Manual review, and the workflow file was parsed with PyYAML. The new
triggers take effect only in `apache/spark` after this is merged. After that,
the first run can be started with `workflow_dispatch` instead of waiting for
12:30 UTC.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #60082 from dongjoon-hyun/SPARK-60867.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/pages.yml | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml
index 3b7e31f4bbfa..d68515241442 100644
--- a/.github/workflows/pages.yml
+++ b/.github/workflows/pages.yml
@@ -20,9 +20,11 @@
name: GitHub Pages deployment
on:
- push:
- branches:
- - master
+ # Deploy once a day instead of on every push to master, to save CI resources.
+ # This runs after the daily docs image refresh in
build_infra_images_cache.yml (04:00 UTC).
+ schedule:
+ - cron: '30 12 * * *'
+ workflow_dispatch:
concurrency:
group: 'docs preview'
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]