This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 4c6884291e8b [SPARK-48149][INFRA] Serialize `build_python.yml` to run a single Python version per cron schedule 4c6884291e8b is described below commit 4c6884291e8b97a7d64dd13530f7ecabe2839d16 Author: Dongjoon Hyun <dh...@apple.com> AuthorDate: Mon May 6 16:06:57 2024 -0700 [SPARK-48149][INFRA] Serialize `build_python.yml` to run a single Python version per cron schedule ### What changes were proposed in this pull request? This PR aims to serialize `build_python.yml` to run a single Python version per cron schedule to reduce the maximum concurrency per single GitHub Action job. ### Why are the changes needed? Currently, `build_python.yml` triggers 60 jobs. `30` of `60` jobs are running concurrently because 10 test pipelines are required per Python version. - https://github.com/apache/spark/actions/workflows/build_python.yml <img width="731" alt="Screenshot 2024-05-06 at 14 28 08" src="https://github.com/apache/spark/assets/9700541/e4f4e9d2-2b2e-43b9-a760-6b9943c7b5b7"> According to https://infra.apache.org/github-actions-policy.html, > All workflows SHOULD have a job concurrency level less than or equal to 15. After this PR, the maximum concurrently level will be 10. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manual review because this is a daily CI. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46407 from dongjoon-hyun/SPARK-48149. Authored-by: Dongjoon Hyun <dh...@apple.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .github/workflows/build_python.yml | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/.github/workflows/build_python.yml b/.github/workflows/build_python.yml index 3354fb726368..9195dc4af518 100644 --- a/.github/workflows/build_python.yml +++ b/.github/workflows/build_python.yml @@ -17,18 +17,26 @@ # under the License. # +# According to https://infra.apache.org/github-actions-policy.html, +# all workflows SHOULD have a job concurrency level less than or equal to 15. +# To do that, we run one python version per cron schedule name: "Build / Python-only (master, PyPy 3.9/Python 3.10/Python 3.12)" on: schedule: - cron: '0 15 * * *' + - cron: '0 17 * * *' + - cron: '0 19 * * *' jobs: run-build: strategy: fail-fast: false matrix: - pyversion: ["pypy3", "python3.10", "python3.12"] + include: + - pyversion: ${{ github.event.schedule == '0 15 * * *' && "pypy3" }} + - pyversion: ${{ github.event.schedule == '0 17 * * *' && "python3.10" }} + - pyversion: ${{ github.event.schedule == '0 19 * * *' && "python3.12" }} permissions: packages: write name: Run --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org