Ruifeng Zheng created SPARK-60870:
-------------------------------------
Summary: Run master documentation deployments every ten commits
Key: SPARK-60870
URL: https://issues.apache.org/jira/browse/SPARK-60870
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
The GitHub Pages preview documentation workflow currently builds and deploys
the in-progress documentation after every push to master. For example, the
documentation job in run 41801950767 took 17 minutes 50 seconds.
SPARK-60834 reduced master post-merge CI to cumulative commit counts divisible
by ten. Apply the same sampling rule to .github/workflows/pages.yml so the
documentation build and deployment run on the same sampled pushes. Use a
lightweight commit-count job before the documentation container is started,
with full-history checkout and git rev-list --count HEAD, matching the existing
post-merge implementation.
Keep the existing documentation build and deployment steps. Move the
docs-preview concurrency group to the documentation job so unsampled pushes do
not occupy or replace a pending deployment in that group.
This reduces the frequency of the expensive documentation job by approximately
90% for ordinary single-commit pushes. It is a temporary resource-saving
measure toward the ASF GitHub Actions usage limits; preview documentation will
refresh roughly every ten master commits.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
