This is an automated email from the ASF dual-hosted git repository.
LuciferYang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4e0b430619e6 [SPARK-58833][INFRA] Make the build, pyspark and docker
job timeouts callable inputs
4e0b430619e6 is described below
commit 4e0b430619e67bf3880a9ecaed103df5eea0e531
Author: YangJie <[email protected]>
AuthorDate: Tue Aug 18 14:58:01 2026 +0800
[SPARK-58833][INFRA] Make the build, pyspark and docker job timeouts
callable inputs
### What changes were proposed in this pull request?
Turns three of `build_and_test.yml`'s hardcoded job timeouts into optional
`workflow_call` inputs:
| input | job | default |
| --- | --- | --- |
| `build_timeout_minutes` | `build` (whole matrix) | 150 |
| `pyspark_timeout_minutes` | `pyspark` (whole matrix) | 120 |
| `docker_integration_tests_timeout_minutes` | `docker-integration-tests` |
120 |
Each default is the value that was hardcoded, so every existing caller
behaves exactly as before. The other jobs keep their literals.
### Why are the changes needed?
`build_and_test.yml` is called by fifteen workflows, several of which run
the same test modules under a configuration that changes how long they take.
When a caller's configuration makes a job slower than the shared literal
allows, there is no way to ask for more time. The job gets cancelled, which
reports as a red run with no test results, the least useful failure a scheduled
build can produce.
The alternative available today is raising the shared literal, which buys
one caller room at the cost of slowing failure detection for the other fourteen.
This came up in #57925, a daily build on the javac codegen backend, where
three jobs no longer fit: `sql - other tests` at 171 minutes against a 150 cap,
and the `pyspark-sql` matrix entry and `docker-integration-tests` at 109 and
116 against 120. Those three are why exactly these three are parameterized
here. Split out per review feedback on that PR.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The defaults are mechanically identical to the literals they replace,
checked by parsing both versions of the file and comparing per job: `build`
150, `pyspark` 120, `docker-integration-tests` 120, three for three. The same
comparison confirms no job changed in any way other than its `timeout-minutes`,
and that the other eight jobs still carry their literals.
Every `inputs.*` reference in the file resolves to a declared input, and
all fifteen callers pass only `java`, `branch`, `hadoop`, `envs` and `jobs`
(`build_main.yml` passes nothing, `build_java17.yml` omits `branch`), none of
which this touches. The three new inputs are `required: false`, so no caller
becomes invalid.
This PR's own CI run exercises the default path, which is what every caller
except an explicit override uses.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
Closes #58061 from LuciferYang/ci-timeout-inputs.
Authored-by: YangJie <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.github/workflows/build_and_test.yml | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index f337f87b5d15..15b55bf9aa49 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -51,6 +51,23 @@ on:
required: false
type: string
default: ''
+ build_timeout_minutes:
+ description: >-
+ Timeout for each `build` matrix job. Raise it for a caller whose
configuration makes the
+ tests slower, e.g. a non-default codegen compiler.
+ required: false
+ type: number
+ default: 150
+ pyspark_timeout_minutes:
+ description: Timeout for each `pyspark` matrix job. Same reason as
above.
+ required: false
+ type: number
+ default: 120
+ docker_integration_tests_timeout_minutes:
+ description: Timeout for the `docker-integration-tests` job. Same
reason as above.
+ required: false
+ type: number
+ default: 120
secrets:
codecov_token:
description: The upload token of codecov.
@@ -261,7 +278,7 @@ jobs:
needs: [precondition, precompile]
if: (!cancelled()) && fromJson(needs.precondition.outputs.required).build
== 'true'
runs-on: ubuntu-latest
- timeout-minutes: 150
+ timeout-minutes: ${{ inputs.build_timeout_minutes }}
strategy:
fail-fast: false
max-parallel: 20
@@ -628,7 +645,7 @@ jobs:
fromJson(needs.precondition.outputs.required).pyspark-pandas == 'true'
name: "Build modules: ${{ matrix.modules }}"
runs-on: ubuntu-latest
- timeout-minutes: 120
+ timeout-minutes: ${{ inputs.pyspark_timeout_minutes }}
container:
image: ${{ needs.precondition.outputs.image_pyspark_url_link }}
options: >-
@@ -1379,7 +1396,7 @@ jobs:
if: (!cancelled()) &&
fromJson(needs.precondition.outputs.required).docker-integration-tests == 'true'
name: Run Docker integration tests
runs-on: ubuntu-latest
- timeout-minutes: 120
+ timeout-minutes: ${{ inputs.docker_integration_tests_timeout_minutes }}
env:
HADOOP_PROFILE: ${{ inputs.hadoop }}
HIVE_PROFILE: hive2.3
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
