Ruifeng Zheng created SPARK-61159:
-------------------------------------
Summary: Stagger scheduled workflows on a three-day cadence
Key: SPARK-61159
URL: https://issues.apache.org/jira/browse/SPARK-61159
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
ASF Infrastructure has asked Spark to reduce GitHub-hosted Actions runner
consumption and concurrent job pressure. Many scheduled variants currently run
on the same odd dates, with several sharing an hour.
Move 66 scheduled workloads on master and maintenance branches to three
day-of-month phases, staggered by hour. Use minute 17 to avoid GitHub's busy
start-of-hour scheduling period; 17 is an arbitrary off-hour choice. The phases
reset monthly, so the interval is approximately three days rather than a strict
rolling 72 hours.
Keep Pages daily at 12:30 UTC and stale-PR cleanup daily at 00:30 UTC. Preserve
the original branch-3.5 scheduler, including its four-day cadence and shared
11:00 UTC trigger. Leave the build-status updater unchanged. Update all
matching branch-dispatch conditions and preserve manual dispatch, target
branches, and skip-if-unchanged checks.
The affected variants share a common cadence without ranking jobs by
importance. Their phase and hour assignments spread runner demand. An
individual interval can be shortened later if faster feedback is needed.
Estimated reduction in aggregate scheduled runner-minutes is 25-27%, or 14-15%
of total CI runner-minutes at the observed usage mix. These are projections
assuming similar job durations and branch activity, including the increases for
four- and six-day variants. Individual jobs do not execute faster because of
this change.
PR: https://github.com/apache/spark/pull/60216
Validation: Actionlint, before/after YAML comparisons, branch-dispatch mapping
checks, and calendar enumeration from 2026 through 2032.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
