Tian Gao created SPARK-61486:
--------------------------------
Summary: Add a generated visualization of the scheduled CI
workflows
Key: SPARK-61486
URL: https://issues.apache.org/jira/browse/SPARK-61486
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Tian Gao
Add a script, dev/visualize_ci_schedule.py, that renders the schedule of the
scheduled GitHub Actions workflows as an SVG image checked in at
.github/workflows/images/scheduled-ci.svg, so that we can see at a glance how
the scheduled CI is distributed and reschedule workflows accordingly.
The script reads the cron entries in .github/workflows, including the workflows
that the branch*_scheduler.yml workflows dispatch on maintenance branches, and
draws one timeline panel per cadence (every 3 days, weekly, daily) plus a
heatmap of the combined runner load over the cycle that covers every
combination of phase and weekday. Each run is drawn with an estimated duration
and number of runners, measured from recent apache/spark runs (the script can
refresh them with --sample), and the load is compared with the ASF GitHub
Actions limits.
The script only uses the Python standard library and its output is
deterministic. A new step in the lint job runs it with --check to fail if the
image is out of date with the workflow schedules.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]