This is an automated email from the ASF dual-hosted git repository.

nchammas pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new c6c90769867d [SPARK-60578][INFRA] Migrate build status workflow off of 
`pull_request_target` trigger
c6c90769867d is described below

commit c6c90769867dce1eca14643a60cbcbf89f8d2286
Author: Nicholas Chammas <[email protected]>
AuthorDate: Tue Nov 3 17:05:59 2026 -0500

    [SPARK-60578][INFRA] Migrate build status workflow off of 
`pull_request_target` trigger
    
    ### What changes were proposed in this pull request?
    
    Remove `notify_test_workflow.yml` and rely on the existing 
`update_build_status.yml` to report fork build status on PRs. Update the 
comments accordingly.
    
    This builds on 
[apache/spark#58579](https://github.com/apache/spark/pull/58579) (SPARK-59293), 
which fixes errors that abort the scheduled scan.
    
    ### Why are the changes needed?
    
    ASF is disabling the `pull_request_target` trigger in [November 2026][asf], 
ahead of GitHub's [December 2 deadline][github]. This is our only workflow 
using it.
    
    The scheduled updater already creates missing `Build` checks (SPARK-58660), 
so we don't need another workflow to replace the notifier. Fork builds still 
run on push, as usual; this PR only changes how their status gets reported.
    
    We should backport this PR to active maintenance branches that accept PRs, 
since `pull_request_target` uses the base branch's workflow.
    
    [asf]: https://infra.apache.org/github-actions-policy.html
    [github]: 
https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target
    
    I intentionally kept this PR simple as I _really_ don't like that we have 
to maintain so much GitHub Actions-specific JavaScript just to manage our 
fork-based contribution workflow.
    
    In other words, in the interest of simplicity I intentionally left out:
    - Any mechanism to update PRs "live".
    - Testing infrastructure for these workflows and the JavaScript they 
contain.
    
    This infrastructure is not in the Spark project's area of expertise. It 
really feels like something we should be able to import and use from somewhere. 
Until then, I think we should strive to keep these workflows as simple as 
possible.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No, nothing for end users.
    
    For contributors here on GitHub:
    
    - The "Notify test workflow" check goes away.
    - New `Build` checks appear as `queued` on the first scheduled pass and 
pick up the fork result on the next. The schedule is every 15 minutes, but 
GitHub can delay or drop runs.
    - If you push multiple updates to a PR between scans, only the latest 
commit gets a `Build` check.
    - The results link opens the fork's Actions run instead of its `Run / Check 
changes` check.
    
    ### How was this patch tested?
    
    Checked locally with actionlint, YAML parsing, and git diff --check.
    
    `update_build_status.yml` changes only comments, so the scheduled updater, 
which runs from master, already behaves as it will after this PR. In [run 
40517915808](https://github.com/apache/spark/actions/runs/40517915808) it 
created Build checks for PRs that had none (#54953, #59831), and later 
scheduled runs synced them (#54953 is now success).
    
    Additionally, an agent replayed the unmodified updater against 150 open PRs 
with their Build checks hidden and writes intercepted: all 150 resolve to the 
same fork run, and the 36 action_required PRs would get identical checks. All 
708 open master PRs currently have exactly one Build check, and 706 match their 
fork run.
    
    This PR can't fully test itself because `pull_request_target` runs the base 
branch's copy of the notifier. I will run additional tests post merge.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Copilot SDK in VS Code (version unavailable)
    
    Closes #59829 from nchammas/pr-target-actions-policy.
    
    Authored-by: Nicholas Chammas <[email protected]>
    Signed-off-by: Nicholas Chammas <[email protected]>
    (cherry picked from commit 6a02fc4e8e5c9c34cfdf01b31fefcf84589a2298)
    Signed-off-by: Nicholas Chammas <[email protected]>
---
 .github/workflows/notify_test_workflow.yml | 198 -----------------------------
 .github/workflows/update_build_status.yml  |  67 +++++-----
 2 files changed, 36 insertions(+), 229 deletions(-)

diff --git a/.github/workflows/notify_test_workflow.yml 
b/.github/workflows/notify_test_workflow.yml
deleted file mode 100644
index 1ca9e14c408b..000000000000
--- a/.github/workflows/notify_test_workflow.yml
+++ /dev/null
@@ -1,198 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements.  See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership.  The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License.  You may obtain a copy of the License at
-#
-#   http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied.  See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-# Intentionally has a general name.
-# because the test status check created in GitHub Actions
-# currently randomly picks any associated workflow.
-# So, the name was changed to make sense in that context too.
-# See also 
https://github.community/t/specify-check-suite-when-creating-a-checkrun/118380/10
-name: On pull request update
-on:
-  pull_request_target:
-    types: [opened, reopened, synchronize]
-
-jobs:
-  notify:
-    name: Notify test workflow
-    runs-on: ubuntu-latest
-    permissions:
-      actions: read
-      checks: write
-    steps:
-      - name: "Notify test workflow"
-        uses: actions/github-script@v8
-        with:
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          script: |
-            const endpoint = 'GET 
/repos/:owner/:repo/actions/workflows/:id/runs?&branch=:branch'
-            const check_run_endpoint = 'GET 
/repos/:owner/:repo/commits/:ref/check-runs?per_page=100'
-
-            // TODO: Should use pull_request.user and 
pull_request.user.repos_url?
-            // If a different person creates a commit to another forked repo,
-            // it wouldn't be able to detect.
-            const params = {
-              owner: context.payload.pull_request.head.repo.owner.login,
-              repo: context.payload.pull_request.head.repo.name,
-              id: 'build_main.yml',
-              branch: context.payload.pull_request.head.ref,
-            }
-            const check_run_params = {
-              owner: context.payload.pull_request.head.repo.owner.login,
-              repo: context.payload.pull_request.head.repo.name,
-              ref: context.payload.pull_request.head.ref,
-            }
-
-            console.log('Ref: ' + context.payload.pull_request.head.ref)
-            console.log('SHA: ' + context.payload.pull_request.head.sha)
-
-            const name = 'Build'
-            const head_sha = context.payload.pull_request.head.sha
-            let status = 'queued'
-
-            // Wait 3 seconds to make sure the fork repository triggered a 
workflow.
-            await new Promise(r => setTimeout(r, 3000))
-
-            // The workflow run for this exact SHA may not be registered yet, 
and the
-            // listing endpoint orders by most recent, so blindly taking the 
first run
-            // can return a stale run from a previous push. Re-query a few 
times looking
-            // for the run whose head_sha matches this PR's head SHA before 
giving up.
-            let matched_run
-            let any_runs = false
-            let retryCount = 0
-            while (retryCount < 3) {
-              let runs
-              try {
-                runs = await github.request(endpoint, params)
-              } catch (error) {
-                console.error(error)
-                // Assume that runs were not found.
-              }
-              if (runs && runs.data.workflow_runs.length > 0) {
-                any_runs = true
-                matched_run = runs.data.workflow_runs.find(r => r.head_sha == 
head_sha)
-                if (matched_run) {
-                  break
-                }
-              }
-              retryCount++
-              if (retryCount < 3) {
-                await new Promise(resolve => setTimeout(resolve, 3000))
-              }
-            }
-
-            // If we saw runs but none matched the PR head SHA, a newer commit 
was pushed
-            // and a fresh notify run will handle it; nothing useful to report 
for this SHA.
-            if (any_runs && !matched_run) {
-              throw new Error('There was a new unsynced commit pushed. Please 
retrigger the workflow.');
-            }
-
-            if (!matched_run) {
-              status = 'completed'
-              const conclusion = 'action_required'
-
-              github.rest.checks.create({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                name: name,
-                head_sha: head_sha,
-                status: status,
-                conclusion: conclusion,
-                output: {
-                  title: 'Workflow run detection failed',
-                  summary: `
-            Unable to detect the workflow run for testing the changes in your 
PR.
-
-            1. If you did not enable GitHub Actions in your forked repository, 
please enable it by clicking the button as shown in the image below. See also 
[Managing Github Actions Settings for a 
repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository)
 for more details.
-            2. It is possible your branch is based on the old \`master\` 
branch in Apache Spark, please sync your branch to the latest master branch. 
For example as below:
-                \`\`\`bash
-                git fetch upstream
-                git rebase upstream/master
-                git push origin YOUR_BRANCH --force
-                \`\`\``,
-                  images: [
-                    {
-                      alt: 'enabling workflows button',
-                      image_url: 
'https://raw.githubusercontent.com/apache/spark/master/.github/workflows/images/workflow-enable-button.png'
-                    }
-                  ]
-                }
-              })
-            } else {
-              const run_id = matched_run.id
-
-              const actions_url = 'https://github.com/'
-                + context.payload.pull_request.head.repo.full_name
-                + '/actions/runs/'
-                + run_id
-              console.log('Actions URL: ' + actions_url)
-
-              // Here we get the check run ID to provide a Check run view 
instead of the
-              // Actions view, see also SPARK-37879. The check run may not 
have materialized
-              // yet (it is created later than the workflow run, especially 
when the matrix
-              // is queued), so this is best-effort: if it cannot be found, we 
fall back to
-              // the Actions run URL rather than failing and leaving the PR 
with no Build
-              // check for the scheduled updater to sync.
-              let retryCount = 0;
-              let check_run_head;
-              while (retryCount < 3) {
-                const check_runs = await github.request(check_run_endpoint, 
check_run_params);
-                check_run_head = check_runs.data.check_runs.find(
-                  r => r.name === "Run / Check changes" && r.head_sha == 
head_sha);
-                if (check_run_head) {
-                  break;
-                }
-                retryCount++;
-                if (retryCount < 3) {
-                  await new Promise(resolve => setTimeout(resolve, 3000));
-                }
-              }
-
-              let summary_url = actions_url
-              if (check_run_head) {
-                summary_url = 'https://github.com/'
-                  + context.payload.pull_request.head.repo.full_name
-                  + '/runs/'
-                  + check_run_head.id
-                console.log('Check run URL: ' + summary_url)
-              } else {
-                console.log('Check run not found; falling back to Actions URL: 
' + actions_url)
-              }
-
-              github.rest.checks.create({
-                owner: context.repo.owner,
-                repo: context.repo.repo,
-                name: name,
-                head_sha: head_sha,
-                status: status,
-                output: {
-                  title: 'Test results',
-                  summary: '[See test results](' + summary_url + ')\n\n'
-                    + 'If the tests fail for reasons unrelated to this pull 
request, '
-                    + 'please rerun the workflow in your forked repository.\n'
-                    + 'If the failures are related to this pull request, '
-                    + 'please investigate them and push follow-up changes.',
-                  text: JSON.stringify({
-                    owner: context.payload.pull_request.head.repo.owner.login,
-                    repo: context.payload.pull_request.head.repo.name,
-                    run_id: run_id
-                  })
-                },
-                details_url: actions_url,
-              })
-            }
diff --git a/.github/workflows/update_build_status.yml 
b/.github/workflows/update_build_status.yml
index 1bb2775235b3..cc477ef08f5b 100644
--- a/.github/workflows/update_build_status.yml
+++ b/.github/workflows/update_build_status.yml
@@ -17,6 +17,17 @@
 # under the License.
 #
 
+# Publishes the status of each open PR's fork build as a `Build` check on the 
PR.
+# Fork builds still run in contributors' forks on push (build_main.yml); this
+# workflow only reports their results. Running on a schedule avoids
+# `pull_request_target` and needs no maintainer approval for fork PRs.
+#
+# Each pass synchronizes existing `Build` checks with their fork runs. A 
missing
+# check is created as `queued`, pointing at the fork run for the PR head 
commit,
+# and synchronized on a later pass. If no fork run is found, an 
`action_required`
+# check with guidance is created instead. GitHub may delay or drop scheduled
+# runs, so neither the initial check nor result updates are immediate.
+
 name: Update build status workflow
 
 on:
@@ -63,11 +74,11 @@ jobs:
               return false;
             };
 
-            // List all check-runs for a commit. per_page=100 (not the default 
30) matches
-            // notify_test_workflow.yml: a SHA can accumulate more check-runs 
than one page
-            // (CI matrix, external checks, duplicate Build checks from 
reopened PRs), which
-            // could otherwise push the target Build check off the first page 
and leave the PR
-            // stuck in 'queued' forever.
+            // List all check-runs for a commit. github.paginate follows every 
page, so the Build
+            // check is found even when a SHA accumulates many check-runs (CI 
matrix, external
+            // checks, duplicate Build checks from reopened PRs); missing it 
would leave the PR
+            // stuck in 'queued' forever. per_page=100 (not the default 30) 
only reduces the
+            // number of requests.
             const listCheckRuns = (ref) => github.paginate(
               'GET /repos/{owner}/{repo}/commits/{ref}/check-runs',
               {
@@ -96,8 +107,8 @@ jobs:
               return params;
             };
 
-            // A Build check this updater can actually sync: not 
action_required (notify writes
-            // that when it missed the fork run, and it carries no run params) 
and with output
+            // A Build check this updater can actually sync: not 
action_required (written when
+            // no fork run was found, and it carries no run params) and with 
output
             // text carrying the {owner, repo, run_id} needed to fetch the 
fork run. A Build check
             // with empty/malformed/fieldless output can never be synced, so 
it must not count as
             // present - otherwise it would suppress the backfill and leave 
the PR stuck with an
@@ -107,7 +118,7 @@ jobs:
                 && parseRunParams(cr) != null;
 
             // An action_required Build check: the contributor-facing "enable 
Actions / rebase"
-            // status notify writes when it found no fork run. It is not 
syncable, but it already
+            // status written when no fork run was found. It is not syncable, 
but it already
             // carries the guidance the no-run backfill branch would create, 
so it is "useful".
             const isActionRequiredBuildCheck = (cr) =>
               cr.name == 'Build' && cr.conclusion == 'action_required';
@@ -121,22 +132,20 @@ jobs:
                 if (pr.mergeable_state == null || 
maybeReady.includes(pr.mergeable_state)) {
                   const checkRuns = await listCheckRuns(pr.head.sha)
 
-                  // Does this SHA already carry an action_required Build 
check? notify (or an
-                  // earlier pass of this updater) writes one when no fork run 
was found. It is
+                  // Does this SHA already carry an action_required Build 
check? An earlier pass
+                  // of this updater writes one when no fork run was found. It 
is
                   // not syncable, so it never suppresses the backfill below - 
which means a PR
                   // that permanently lacks a fork run (Actions disabled, old 
master) would
                   // otherwise re-poll on every 15-minute pass forever. When 
one is already
                   // present, the backfill's re-poll is unnecessary (see 
below).
                   const hasActionRequiredBuildCheck = 
checkRuns.some(isActionRequiredBuildCheck)
 
-                  // Track whether a syncable Build check exists (see 
isSyncableBuildCheck).
-                  // notify_test_workflow.yml creates one per push; if that 
job never completed
-                  // (e.g. cancelled while starved of an ASF runner) the check 
is missing and the
-                  // backfill after this loop recreates it. Sync every match 
(no early break): a
-                  // SHA can carry more than one Build check (reopened PRs, or 
a backfill that
-                  // raced notify). Branch protection evaluates the newest 
check-run of a given
-                  // name, so syncing only the first would leave a newer 
duplicate stuck in
-                  // 'queued' and block the PR.
+                  // Track whether a syncable Build check exists (see 
isSyncableBuildCheck); if
+                  // not, the backfill after this loop creates one. Sync every 
match (no early
+                  // break): a SHA can carry more than one Build check 
(reopened PRs, or
+                  // overlapping updater runs). Branch protection evaluates 
the newest check-run
+                  // of a given name, so syncing only the first would leave a 
newer duplicate
+                  // stuck in 'queued' and block the PR.
                   let syncableBuildCheck = false
                   // Build checks whose referenced fork run is permanently 
gone (404). They parse
                   // as syncable (their output still carries valid run params) 
but can never be
@@ -203,14 +212,13 @@ jobs:
                     }
                   }
 
-                  // No syncable Build check: notify_test_workflow.yml never 
created one, or only
-                  // a stale action_required one exists. Recreate it here, 
mirroring notify. Skip
-                  // if the head repo was deleted.
+                  // No syncable Build check: none was created yet, or only a 
stale action_required
+                  // one exists. Create it here. Skip if the head repo was 
deleted.
                   if (!syncableBuildCheck && pr.head.repo) {
                     const forkOwner = pr.head.repo.owner.login
                     const forkRepo = pr.head.repo.name
-                    // Look up the fork's build_main.yml runs for the PR 
branch, matching
-                    // notify_test_workflow.yml. Filter by branch (head.ref), 
not just head_sha:
+                    // Look up the fork's build_main.yml runs for the PR 
branch.
+                    // Filter by branch (head.ref), not just head_sha:
                     // build_main.yml skips fork pushes to master (the "Sync 
fork" case), so the
                     // same SHA can carry a skipped/unrelated run on another 
branch, and a
                     // head_sha-only lookup could attach the Build check to 
that run instead of
@@ -270,18 +278,15 @@ jobs:
                     // An action_required Build check already exists and we 
still found no run:
                     // the guidance check the no-run branch below would create 
is already present,
                     // so there is nothing to backfill. Skip the recheck 
listing and the create
-                    // outright. (This is only reachable via the single, 
sleepless lookup above,
-                    // so there was no ~6s window for notify to have created a 
syncable check in
-                    // the meantime; even if it had, skipping is safe - with 
no run there is
-                    // nothing to promote to 'queued' this pass anyway.)
+                    // outright.
                     if (hasActionRequiredBuildCheck && !matched_run) {
                       console.log('  action_required Build check present; 
nothing to backfill')
                       continue
                     }
 
                     // Recheck for an existing Build check immediately before 
creating one. The
-                    // poll above can span up to ~6 seconds, during which 
notify_test_workflow.yml
-                    // may create the check; this recheck skips the backfill 
in that common case.
+                    // poll above can span up to ~6 seconds, during which an 
overlapping updater run
+                    // may create the check; this recheck skips the backfill 
in that case.
                     // It narrows but cannot fully close the window (there is 
no atomic
                     // create-if-absent, and the list endpoint is eventually 
consistent) - the
                     // sync loop above stays tolerant of duplicates so any 
that slip through still
@@ -341,8 +346,8 @@ jobs:
                       } else {
                         // No run for this head commit after re-polling (empty 
result, permanent
                         // lookup failure, or build_main.yml missing): Actions 
is disabled, the
-                        // branch is on an old master, or the commit never 
triggered a run. Mirror
-                        // notify's action_required check so the PR carries a 
required status
+                        // branch is on an old master, or the commit never 
triggered a run. Create
+                        // an action_required check so the PR carries a 
required status
                         // telling the contributor how to fix it.
                         console.log('  No forked run for head SHA; creating 
action_required check')
                         await github.rest.checks.create({


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to