zhengruifeng commented on PR #57461: URL: https://github.com/apache/spark/pull/57461#issuecomment-5087067643
### GitHub Actions capacity proposal The [ASF GitHub Actions policy](https://infra.apache.org/github-actions-policy.html) states: "The average number of minutes a project uses per calendar week MUST NOT exceed the equivalent of 25 full-time runners (250,000 minutes, or 4,200 hours)." The [Spark usage dashboard](https://github.com/apache/spark/actions/metrics/usage?dateRangeType=DATE_RANGE_TYPE_LAST_30_DAYS) reports 1,064,307 runner minutes in the last 30 days. Evenly distributed, that is about 248,338 minutes per seven days, which is very close to the 250,000-minute weekly limit. The policy is measured by calendar week, so the dashboard total is not itself a compliance determination, but it shows that we have little headroom. Proposal: - Run the existing full post-merge `build_main` workflow daily or after every N merged commits. - Run this lower-cost smart class-level selection workflow on every post-merge commit. The smart workflow preserves targeted feedback for each commit while keeping the expensive full build available as periodic broad coverage. It can also return no targets for documentation-only changes; otherwise it caps selection at 20 JVM suites and 20 PySpark modules, precompiles once, and runs only the selected targets. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
