This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new fceb1571d99d [SPARK-61114][INFRA] Run the full matrix in sampled
post-merge CI
fceb1571d99d is described below
commit fceb1571d99da32a59f4339ba853c116697328e6
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Sun Nov 8 10:30:42 2026 +0800
[SPARK-61114][INFRA] Run the full matrix in sampled post-merge CI
### What changes were proposed in this pull request?
Enable the complete existing job matrix for sampled upstream post-merge
pushes by passing an explicit all-enabled `jobs` selection from
`.github/workflows/build_main.yml` to the reusable build workflow.
Only `build_main.yml` changes. `build_and_test.yml`, its job definitions,
and its per-job module/test filtering are unchanged. Fork pushes keep
change-based job selection, and scheduled workflows keep their existing
configurations.
### Why are the changes needed?
Post-merge CI now runs on one in ten commits after #60177
([SPARK-61016](https://issues.apache.org/jira/browse/SPARK-61016)). The
selected runs should enable the full job matrix, including jobs currently
disabled for upstream pushes: pandas API on Spark, periodic Python and
old-client tests, SparkR, TPC-DS, Docker/Kubernetes/OIDC integration tests,
docs, UI, protobuf checks, and the Java 25 build.
The sampling frequency remains 10%. This change expands job selection while
preserving the shared workflow's existing test-selection behavior.
### Does this PR introduce _any_ user-facing change?
No. This changes CI job selection only.
### How was this patch tested?
- `actionlint` v1.7.11 passed for `.github/workflows/build_main.yml`; `git
diff --check` passed.
- Parsed the workflow in `spark-dev-313` and verified that the upstream
selection enables all 18 selection flags while preserving fork behavior and the
fork-master skip condition.
- Executed the existing precondition shell step with the full job selection
and verified its output.
- Confirmed that `build_and_test.yml` matches `master` exactly.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-6)
Closes #60203 from zhengruifeng/infra-full-postmerge-matrix-20261107.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
skip-checks: true
---
.github/workflows/build_main.yml | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/.github/workflows/build_main.yml b/.github/workflows/build_main.yml
index a93adae4dcbf..9f2ca2de1e0f 100644
--- a/.github/workflows/build_main.yml
+++ b/.github/workflows/build_main.yml
@@ -35,3 +35,27 @@ jobs:
# dev/merge_spark_pr.py through GitHub's workflow-skip commit trailer.
if: github.repository == 'apache/spark' || github.ref !=
'refs/heads/master'
uses: ./.github/workflows/build_and_test.yml
+ with:
+ # Sampled upstream pushes run the full job matrix.
+ # Fork pushes keep the reusable workflow's change-based selection.
+ jobs: >-
+ ${{ github.repository == 'apache/spark' && '{
+ "build": "true",
+ "build-core-utils": "true",
+ "pyspark": "true",
+ "pyspark-pandas": "true",
+ "pyspark-periodic": "true",
+ "pyspark-connect-old-client": "true",
+ "sparkr": "true",
+ "tpcds-1g": "true",
+ "docker-integration-tests": "true",
+ "lint": "true",
+ "java25": "true",
+ "docs": "true",
+ "yarn": "true",
+ "k8s-integration-tests": "true",
+ "oidc-e2e": "true",
+ "buf": "true",
+ "ui": "true",
+ "transpile": "true"
+ }' || '' }}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]