Tian Gao created SPARK-61486:
--------------------------------

             Summary: Add a generated visualization of the scheduled CI 
workflows
                 Key: SPARK-61486
                 URL: https://issues.apache.org/jira/browse/SPARK-61486
             Project: Spark
          Issue Type: Improvement
          Components: Project Infra
    Affects Versions: 5.0.0
            Reporter: Tian Gao


Add a script, dev/visualize_ci_schedule.py, that renders the schedule of the 
scheduled GitHub Actions workflows as an SVG image checked in at 
.github/workflows/images/scheduled-ci.svg, so that we can see at a glance how 
the scheduled CI is distributed and reschedule workflows accordingly.

The script reads the cron entries in .github/workflows, including the workflows 
that the branch*_scheduler.yml workflows dispatch on maintenance branches, and 
draws one timeline panel per cadence (every 3 days, weekly, daily) plus a 
heatmap of the combined runner load over the cycle that covers every 
combination of phase and weekday. Each run is drawn with an estimated duration 
and number of runners, measured from recent apache/spark runs (the script can 
refresh them with --sample), and the load is compared with the ASF GitHub 
Actions limits.

The script only uses the Python standard library and its output is 
deterministic. A new step in the lint job runs it with --check to fail if the 
image is out of date with the workflow schedules.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to