Copilot commented on code in PR #13535: URL: https://github.com/apache/gluten/pull/13535#discussion_r4563904740
########## .github/workflows/update_fork_build_status.yml: ########## @@ -0,0 +1,52 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# A pull request from a fork runs velox_backend_x86.yml, velox_backend_arm.yml and +# velox_kafka.yml in the fork, on push. This workflow reports those runs as checks on +# the pull request, using util/update-fork-build-status.js. It only reads run metadata, +# so it never runs pull request code and needs no pull_request_target. Adapted from +# apache/spark's update_build_status.yml (SPARK-60578). +name: Update fork build status + +on: + schedule: + # Every 5 minutes, off the top of the hour, when GitHub delays scheduled runs most. + - cron: '2-57/5 * * * *' + workflow_dispatch: + +permissions: {} + +concurrency: + group: ${{ github.workflow }} + cancel-in-progress: false + +jobs: + update: + if: github.repository == 'apache/gluten' + runs-on: ubuntu-slim Review Comment: `runs-on: ubuntu-slim` is not a standard GitHub-hosted runner label. Unless this repo/org has a self-hosted runner registered with that exact label, the workflow will fail to schedule. Use a known label (e.g., `ubuntu-24.04` / `ubuntu-latest`) or a documented self-hosted runner label used elsewhere in this repo. ########## .github/workflows/velox_kafka.yml: ########## @@ -32,27 +33,76 @@ on: - 'ep/build-velox/**' - 'dev/**' - 'build/mvn' + # A pull request from a fork is tested in the fork: each push to a branch + # other than main runs this workflow there (see merge-upstream). In + # apache/gluten a push run skips every job. + push: + branches-ignore: + - main workflow_dispatch: permissions: contents: read actions: read concurrency: - group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }} + # head_ref is empty on push. Keying push runs by branch lets a new push cancel the previous run. + group: >- + ${{ github.repository }}-${{ github.head_ref || (github.event_name == 'push' && github.ref) || + github.sha }}-${{ github.workflow }} cancel-in-progress: true env: CCACHE_DIR: ${{ github.workspace }}/.ccache PROMPT_ALWAYS_RESPOND: 'y' jobs: + # Every other job chains off this one, so its `if` decides whether the workflow runs: + # push and workflow_dispatch in forks, where the run tests a fork pull request; + # pull_request only in apache/gluten and only from its own branches (Dependabot); + # workflow_dispatch in apache/gluten. + # In a fork it squash-merges the commit under test onto apache/gluten main and + # publishes the result, which the jobs below check out by `revision`. It is the + # only job that can write, and none of its steps run in apache/gluten. + merge-upstream: + if: >- + (github.event_name == 'push' && github.repository_owner != 'apache') || + (github.event_name == 'pull_request' && github.repository_owner == 'apache' && + github.event.pull_request.head.repo.full_name == github.repository) || + github.event_name == 'workflow_dispatch' + runs-on: ubuntu-24.04 + permissions: + contents: write Review Comment: Because the job-level `if` includes `workflow_dispatch` unconditionally, `merge-upstream` will run in `apache/gluten` on manual dispatch and request `contents: write` even though the fork-only steps are guarded and won’t execute there. To follow least privilege, consider tightening the `if` so the job only runs on `workflow_dispatch` when `github.repository_owner != 'apache'`, or split into an upstream-only gate job (read-only permissions) and a fork-only merge/publish job (write permissions). ########## .github/workflows/iceberg_spark_ut.yml: ########## @@ -60,11 +68,12 @@ jobs: steps: - uses: actions/checkout@v7 with: + ref: ${{ inputs.revision }} Review Comment: The comment says an empty `inputs.revision` should mean “the event’s commit”, but the workflow passes `ref: ${{ inputs.revision }}` directly. While `actions/checkout` may treat an empty string as “use default ref”, it’s clearer and more robust to express the intent explicitly (e.g., `inputs.revision || github.sha`) so behavior doesn’t depend on action-specific handling of empty inputs. ########## .github/workflows/velox_backend_arm.yml: ########## @@ -40,6 +41,13 @@ on: - '!cpp/bolt.CMakeLists.cmake' - 'dev/**' - 'build/mvn' + # A pull request from a fork is tested in the fork: each push to a branch + # other than main runs this workflow there (see detect-changes). In + # apache/gluten a push run skips every job. + push: + branches-ignore: + - main + workflow_dispatch: Review Comment: This workflow is configured to trigger on `workflow_dispatch`, but `merge-upstream` (and therefore the rest of the workflow via `needs: merge-upstream`) never runs for `workflow_dispatch` in `apache/gluten` because the job-level `if` excludes that case. The result is a manual dispatch in upstream that triggers a run but skips all jobs. Either remove `workflow_dispatch` from `on:` if it’s intended to be fork-only, or update `merge-upstream.if` to explicitly allow `workflow_dispatch` in `apache/gluten` (and ensure downstream jobs behave correctly). ########## .github/workflows/util/update-fork-build-status.js: ########## @@ -0,0 +1,252 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// Run by update_fork_build_status.yml. For each open pull request from a fork, reports +// the fork's push run of each workflow below as the check named next to it. + +const WORKFLOWS = [ + { file: 'velox_backend_x86.yml', check: 'Velox Backend (x86) in fork' }, + { file: 'velox_backend_arm.yml', check: 'Velox Backend (ARM) in fork' }, + { file: 'velox_kafka.yml', check: 'Velox Kafka in fork' }, +]; + +// Check runs accept fewer conclusions than a workflow run reports; anything else +// means the run produced no result. +const CONCLUSIONS = ['success', 'failure', 'neutral', 'cancelled', 'timed_out', 'action_required', 'skipped']; + +// Rate limiting, which GitHub reports as 429 or as 403 with rate-limit headers. +function isRateLimited(e) { + if (e.status == 429) return true; + if (e.status != 403) return false; + const h = (e.response && e.response.headers) || {}; + return !!h['retry-after'] || h['x-ratelimit-remaining'] === '0' || /rate limit/i.test(e.message || ''); +} + +// 5xx and network errors are retried on a later pass. +function isTransient(e) { + return !e.status || e.status >= 500; +} + +const runPath = (w) => '.github/workflows/' + w.file; + +// Job conclusions that make the run fail. No job in these workflows sets +// continue-on-error, so a check can turn red before the run ends. +const FAILED = ['failure', 'timed_out']; + +// What a check records in its output text about its fork run, or null. +function recordOf(check) { + if (check.conclusion == 'action_required') return null; + try { + const p = JSON.parse(check.output.text); + return p && p.run_id ? p : null; + } catch (e) { + return null; + } +} + +// The fork run attempt a check points at, as "run_id/attempt", or null. Checks +// written before attempts were recorded count as attempt 1. +function attemptOf(check) { + const p = recordOf(check); + return p ? `${p.run_id}/${p.run_attempt || 1}` : null; +} + +// The check status (and conclusion) for a run's current state and its failed jobs. +function stateOf(run, failedJobs) { + if (failedJobs.length) return { status: 'completed', conclusion: 'failure' }; + if (run.status == 'completed') { + return { status: 'completed', conclusion: CONCLUSIONS.includes(run.conclusion) ? run.conclusion : 'failure' }; + } + return { status: run.status == 'in_progress' ? 'in_progress' : 'queued' }; +} + +function noRunSummary(w, sha) { + return [ + `${w.file} runs in your fork on each push to a branch other than main. No run was found for ${sha}.`, + '', + '1. Enable GitHub Actions in the Actions tab of your fork, then push again. See [Managing GitHub Actions settings for a repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository).', + '2. If your branch predates fork runs, rebase it on the latest apache/gluten main and push again.', + "3. If this pull request comes from your fork's main branch, push it to another branch and open the pull request from there.", + ].join('\n'); +} + +module.exports = async ({ github, context, core }) => { + const repo = { owner: context.repo.owner, repo: context.repo.repo }; + + const listChecks = (sha) => + github.paginate('GET /repos/{owner}/{repo}/commits/{ref}/check-runs', { ...repo, ref: sha, filter: 'all', per_page: 100 }); + + // The fork's push runs for the pull request's head commit. Re-polled until every + // workflow in `wanted` has one, because a run can take a few seconds to register. + const listForkRuns = async (pr, wanted, attempts) => { + let runs = []; + for (let i = 0; i < attempts; i++) { + try { + const { data } = await github.request('GET /repos/{owner}/{repo}/actions/runs', { + owner: pr.head.repo.owner.login, + repo: pr.head.repo.name, + event: 'push', + branch: pr.head.ref, + head_sha: pr.head.sha, Review Comment: `GET /repos/{owner}/{repo}/actions/runs` does not (historically) accept a `head_sha` filter parameter; passing unknown query params can be ignored or can trigger a 422 depending on API behavior. To avoid hard-to-diagnose failures, either (a) drop `head_sha` from the request and rely on the existing `filter((r) => r.head_sha == pr.head.sha)` client-side filtering, or (b) query per-workflow via `GET /repos/{owner}/{repo}/actions/workflows/{workflow_id}/runs` (workflow file name is accepted as `workflow_id`) which supports `head_sha`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
