This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-connect-swift.git
The following commit(s) were added to refs/heads/main by this push:
new cc38900 [SPARK-57151] Run CI in forked repositories and report build
status to PRs
cc38900 is described below
commit cc389008d25b088fced35773a18ce5104409c71a
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri May 29 11:56:22 2026 -0700
[SPARK-57151] Run CI in forked repositories and report build status to PRs
### What changes were proposed in this pull request?
Make `build_and_test.yml` a reusable (`workflow_call`) workflow and add
Apache
Spark's fork-based CI status mechanism:
- `build_main.yml` — runs on `push` to all branches and calls
`build_and_test.yml`.
- `notify_test_workflow.yml` — on `pull_request_target`, creates a `Build`
check
on the PR that points to the fork's run.
- `update_build_status.yml` — every 15 minutes, syncs the `Build` check
from the
fork's run.
- Add `images/workflow-enable-button.png` used by the notify message, and
repoint
the README badge and `AGENTS.md` to `build_main.yml`.
Flow:
```mermaid
flowchart TD
subgraph fork["Forked repo (contributor)"]
P([push to any branch]) --> BM["build_main.yml — job: Run"]
BM -->|workflow_call| BAT["build_and_test.yml
(reusable):<br/>License Check, build, integration tests"]
BAT --> CR["check runs:<br/>Run / License Check, ..."]
end
subgraph up["Upstream repo (apache/spark-connect-swift)"]
PRT([pull_request_target]) --> N["notify_test_workflow.yml"]
SCH([schedule: every 15 min]) --> U["update_build_status.yml"]
N -->|create| B(["Build check on PR"])
U -->|"PATCH status / conclusion"| B
end
BM -.->|"① find run (id: build_main.yml)"| N
CR -.->|"② link check-run view"| N
BM -.->|"③ poll run status"| U
```
### Why are the changes needed?
Like Apache Spark main repository, run the heavy test matrix in
contributors' forks and mirror only a single `Build`
status check upstream, instead of running the matrix in the upstream repo
on every
PR. This matches the Apache Spark main repository.
### Does this PR introduce _any_ user-facing change?
No. CI and documentation only.
### How was this patch tested?
Static verification of the cross-workflow wiring: the `Run / License Check`
check-run name, the shared `Build` check name, and the `run_id` passed from
`notify` to `update`. Full behavior takes effect only after merge to
`main`, since
the `pull_request_target`/`schedule` workflows and `main`-pinned URLs
resolve from
the default branch.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
Closes #399 from dongjoon-hyun/SPARK-57151.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 5 +-
.github/workflows/build_main.yml | 29 ++++
.../workflows/images/workflow-enable-button.png | Bin 0 -> 79807 bytes
.github/workflows/notify_test_workflow.yml | 172 +++++++++++++++++++++
.github/workflows/update_build_status.yml | 111 +++++++++++++
AGENTS.md | 6 +-
README.md | 2 +-
7 files changed, 318 insertions(+), 7 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 0d3e283..5f4506f 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -20,10 +20,7 @@
name: Build and test
on:
- push:
- branches: [ "main" ]
- pull_request:
- branches: [ "main" ]
+ workflow_call:
jobs:
license-check:
diff --git a/.github/workflows/build_main.yml b/.github/workflows/build_main.yml
new file mode 100644
index 0000000..370da00
--- /dev/null
+++ b/.github/workflows/build_main.yml
@@ -0,0 +1,29 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: "Build"
+
+on:
+ push:
+ branches: [ "**" ]
+
+jobs:
+ call-build-and-test:
+ name: Run
+ uses: ./.github/workflows/build_and_test.yml
diff --git a/.github/workflows/images/workflow-enable-button.png
b/.github/workflows/images/workflow-enable-button.png
new file mode 100644
index 0000000..f7299f2
Binary files /dev/null and
b/.github/workflows/images/workflow-enable-button.png differ
diff --git a/.github/workflows/notify_test_workflow.yml
b/.github/workflows/notify_test_workflow.yml
new file mode 100644
index 0000000..beeed62
--- /dev/null
+++ b/.github/workflows/notify_test_workflow.yml
@@ -0,0 +1,172 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+# Intentionally has a general name.
+# because the test status check created in GitHub Actions
+# currently randomly picks any associated workflow.
+# So, the name was changed to make sense in that context too.
+# See also
https://github.community/t/specify-check-suite-when-creating-a-checkrun/118380/10
+name: On pull request update
+on:
+ pull_request_target:
+ types: [opened, reopened, synchronize]
+
+jobs:
+ notify:
+ name: Notify test workflow
+ runs-on: ubuntu-slim
+ permissions:
+ actions: read
+ checks: write
+ steps:
+ - name: "Notify test workflow"
+ uses: actions/github-script@v9
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET
/repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch'
+ const check_run_endpoint = 'GET
/repos/:owner/:repo/commits/:ref/check-runs?per_page=100'
+
+ // TODO: Should use pull_request.user and
pull_request.user.repos_url?
+ // If a different person creates a commit to another forked repo,
+ // it wouldn't be able to detect.
+ const params = {
+ owner: context.payload.pull_request.head.repo.owner.login,
+ repo: context.payload.pull_request.head.repo.name,
+ id: 'build_main.yml',
+ branch: context.payload.pull_request.head.ref,
+ }
+ const check_run_params = {
+ owner: context.payload.pull_request.head.repo.owner.login,
+ repo: context.payload.pull_request.head.repo.name,
+ ref: context.payload.pull_request.head.ref,
+ }
+
+ console.log('Ref: ' + context.payload.pull_request.head.ref)
+ console.log('SHA: ' + context.payload.pull_request.head.sha)
+
+ // Wait 3 seconds to make sure the fork repository triggered a
workflow.
+ await new Promise(r => setTimeout(r, 3000))
+
+ let runs
+ try {
+ runs = await github.request(endpoint, params)
+ } catch (error) {
+ console.error(error)
+ // Assume that runs were not found.
+ }
+
+ const name = 'Build'
+ const head_sha = context.payload.pull_request.head.sha
+ let status = 'queued'
+
+ if (!runs || runs.data.workflow_runs.length === 0) {
+ status = 'completed'
+ const conclusion = 'action_required'
+
+ github.rest.checks.create({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ name: name,
+ head_sha: head_sha,
+ status: status,
+ conclusion: conclusion,
+ output: {
+ title: 'Workflow run detection failed',
+ summary: `
+ Unable to detect the workflow run for testing the changes in your
PR.
+
+ 1. If you did not enable GitHub Actions in your forked repository,
please enable it by clicking the button as shown in the image below. See also
[Managing Github Actions Settings for a
repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository)
for more details.
+ 2. It is possible your branch is based on the old \`main\` branch
in Apache Spark Connect Client for Swift, please sync your branch to the latest
main branch. For example as below:
+ \`\`\`bash
+ git fetch upstream
+ git rebase upstream/main
+ git push origin YOUR_BRANCH --force
+ \`\`\``,
+ images: [
+ {
+ alt: 'enabling workflows button',
+ image_url:
'https://raw.githubusercontent.com/apache/spark-connect-swift/main/.github/workflows/images/workflow-enable-button.png'
+ }
+ ]
+ }
+ })
+ } else {
+ const run_id = runs.data.workflow_runs[0].id
+
+ if (runs.data.workflow_runs[0].head_sha !=
context.payload.pull_request.head.sha) {
+ throw new Error('There was a new unsynced commit pushed.
Please retrigger the workflow.');
+ }
+
+ // Here we get check run ID to provide Check run view instead of
Actions view, see also SPARK-37879.
+ let retryCount = 0;
+ let check_run_head;
+ while (retryCount < 3) {
+ const check_runs = await github.request(check_run_endpoint,
check_run_params);
+ check_run_head = check_runs.data.check_runs.find(r => r.name
=== "Run / License Check");
+ if (check_run_head) {
+ break;
+ }
+ retryCount++;
+ if (retryCount < 3) {
+ await new Promise(resolve => setTimeout(resolve, 3000));
+ }
+ }
+ if (!check_run_head) {
+ throw new Error('Failed to retrieve check_run_head after 3
attempts');
+ }
+
+ if (check_run_head.head_sha !=
context.payload.pull_request.head.sha) {
+ throw new Error('There was a new unsynced commit pushed.
Please retrigger the workflow.');
+ }
+
+ const check_run_url = 'https://github.com/'
+ + context.payload.pull_request.head.repo.full_name
+ + '/runs/'
+ + check_run_head.id
+ console.log('Check run URL: ' + check_run_url)
+
+ const actions_url = 'https://github.com/'
+ + context.payload.pull_request.head.repo.full_name
+ + '/actions/runs/'
+ + run_id
+ console.log('Actions URL: ' + actions_url)
+
+ github.rest.checks.create({
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ name: name,
+ head_sha: head_sha,
+ status: status,
+ output: {
+ title: 'Test results',
+ summary: '[See test results](' + check_run_url + ')\n\n'
+ + 'If the tests fail for reasons unrelated to this pull
request, '
+ + 'please rerun the workflow in your forked repository.\n'
+ + 'If the failures are related to this pull request, '
+ + 'please investigate them and push follow-up changes.',
+ text: JSON.stringify({
+ owner: context.payload.pull_request.head.repo.owner.login,
+ repo: context.payload.pull_request.head.repo.name,
+ run_id: run_id
+ })
+ },
+ details_url: actions_url,
+ })
+ }
diff --git a/.github/workflows/update_build_status.yml
b/.github/workflows/update_build_status.yml
new file mode 100644
index 0000000..26ab78f
--- /dev/null
+++ b/.github/workflows/update_build_status.yml
@@ -0,0 +1,111 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+name: Update build status workflow
+
+on:
+ schedule:
+ - cron: "*/15 * * * *"
+
+jobs:
+ update:
+ name: Update build status
+ runs-on: ubuntu-slim
+ permissions:
+ actions: read
+ checks: write
+ steps:
+ - name: "Update build status"
+ uses: actions/github-script@v9
+ with:
+ github-token: ${{ secrets.GITHUB_TOKEN }}
+ script: |
+ const endpoint = 'GET /repos/:owner/:repo/pulls?state=:state'
+ const params = {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ state: 'open'
+ }
+
+ // See
https://docs.github.com/en/graphql/reference/enums#mergestatestatus
+ const maybeReady = ['behind', 'clean', 'draft', 'has_hooks',
'unknown', 'unstable'];
+
+ // Iterate open PRs
+ for await (const prs of github.paginate.iterator(endpoint,params))
{
+ // Each page
+ for await (const pr of prs.data) {
+ console.log('SHA: ' + pr.head.sha)
+ console.log(' Mergeable status: ' + pr.mergeable_state)
+ if (pr.mergeable_state == null ||
maybeReady.includes(pr.mergeable_state)) {
+ const checkRuns = await github.request('GET
/repos/{owner}/{repo}/commits/{ref}/check-runs', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ ref: pr.head.sha
+ })
+
+ // Iterator GitHub Checks in the PR
+ for await (const cr of checkRuns.data.check_runs) {
+ if (cr.name == 'Build' && cr.conclusion !=
"action_required") {
+ // text contains parameters to make request in JSON.
+ const params = JSON.parse(cr.output.text)
+
+ // Get the workflow run in the forked repository
+ let run
+ try {
+ run = await github.request('GET
/repos/{owner}/{repo}/actions/runs/{run_id}', params)
+ } catch (error) {
+ console.error(error)
+ // Run not found. This can happen when the PR author
removes GitHub Actions runs or
+ // disables GitHub Actions.
+ continue
+ }
+
+ // Keep syncing the status of the checks
+ if (run.data.status == 'completed') {
+ console.log(' Run ' + cr.id + ': set status (' +
run.data.status + ') and conclusion (' + run.data.conclusion + ')')
+ const response = await github.request('PATCH
/repos/{owner}/{repo}/check-runs/{check_run_id}', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ check_run_id: cr.id,
+ output: cr.output,
+ status: run.data.status,
+ conclusion: run.data.conclusion,
+ details_url: run.data.details_url
+ })
+ } else {
+ // PATCH /check-runs accepts only queued | in_progress
| completed,
+ // but workflow_run may also report requested /
waiting / pending.
+ const status = run.data.status == 'in_progress' ?
'in_progress' : 'queued'
+ console.log(' Run ' + cr.id + ': set status (' +
run.data.status + ' -> ' + status + ')')
+ const response = await github.request('PATCH
/repos/{owner}/{repo}/check-runs/{check_run_id}', {
+ owner: context.repo.owner,
+ repo: context.repo.repo,
+ check_run_id: cr.id,
+ output: cr.output,
+ status: status,
+ details_url: run.data.details_url
+ })
+ }
+
+ break
+ }
+ }
+ }
+ }
+ }
diff --git a/AGENTS.md b/AGENTS.md
index 9ef64b4..955682e 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -46,8 +46,10 @@ catalog, ML) over gRPC, exchanging results in Apache Arrow
format.
- `Examples/` — runnable sample apps (`pi`, `spark-sql`, `stream`, `web`,
`app`,
`pyspark-connect`), each with its own `Package.swift` and `Dockerfile`.
- `dev/` — Python maintainer scripts (JIRA + PR merge tooling).
-- `.github/workflows/build_and_test.yml` — CI: license check, multi-platform
- build, and integration tests.
+- `.github/workflows/build_main.yml` — CI entry point: runs on push to all
+ branches and calls the reusable `build_and_test.yml` (license check,
+ multi-platform build, integration tests). `notify_test_workflow.yml` and
+ `update_build_status.yml` mirror forked-repo CI results onto PR checks.
## Build
diff --git a/README.md b/README.md
index 9833b0f..ea98bb8 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
# Apache Spark Connect Client for Swift
[](https://github.com/apache/spark-connect-swift/releases/latest)
-[](https://github.com/apache/spark-connect-swift/blob/main/.github/workflows/build_and_test.yml)
+[](https://github.com/apache/spark-connect-swift/blob/main/.github/workflows/build_main.yml)
[](https://swiftpackageindex.com/apache/spark-connect-swift)
[](https://swiftpackageindex.com/apache/spark-connect-swift)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
