Ruifeng Zheng created SPARK-60924:
-------------------------------------
Summary: Use GraphQL to count commits for post-merge CI sampling
Key: SPARK-60924
URL: https://issues.apache.org/jira/browse/SPARK-60924
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
The master post-merge CI sampling job downloads the entire Spark Git history
just to calculate git rev-list --count HEAD. This checkout runs on every master
push, including the nine out of ten pushes whose expensive build is skipped.
Replace the full-history checkout with a GitHub GraphQL query for
Commit.history.totalCount at the triggering GITHUB_SHA. Keep the existing
commit-count modulo ten rule, read-only contents permission, output, summary,
and downstream build conditions. Reject missing or invalid counts so an API
problem cannot select a build accidentally.
This follows Nicholas Chammas's review suggestion on SPARK-60834 / PR #60045:
https://github.com/apache/spark/pull/60045#discussion_r4455095349
The same query can be used when backporting the optimization to branch-4.x. No
Git checkout is required for the sampling job.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
