yuqi1129 commented on code in PR #13049: URL: https://github.com/apache/gravitino/pull/13049#discussion_r4337455231
########## .github/workflows/fork-ci.yml: ########## @@ -0,0 +1,235 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Run the complete CI entry point from branch pushes in the contributor's repository. +name: Fork CI + +on: + push: + branches: ['**'] + +concurrency: + group: fork-ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + eligibility: + name: Resolve fork CI eligibility + # Downstream mirrors can set this repository variable to keep the + # inherited workflow completely dormant. + if: vars.DISABLE_FORK_CI != 'true' + runs-on: ubuntu-slim Review Comment: `ubuntu-slim` is picked up in personal forks as well: the eligibility job ran on it in every fork run I did, for example https://github.com/qqqttt123/gravitino/actions/runs/39581484861/job/119160356453 for this head. The job only calls the API and runs `git merge-tree`, so it stays well under the 15-minute limit. ########## .github/workflows/fork-ci.yml: ########## @@ -0,0 +1,235 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Run the complete CI entry point from branch pushes in the contributor's repository. +name: Fork CI + +on: + push: + branches: ['**'] + +concurrency: + group: fork-ci-${{ github.ref }} + cancel-in-progress: true + +jobs: + eligibility: + name: Resolve fork CI eligibility + # Downstream mirrors can set this repository variable to keep the + # inherited workflow completely dormant. + if: vars.DISABLE_FORK_CI != 'true' + runs-on: ubuntu-slim + outputs: + applicable: ${{ steps.resolve.outputs.applicable }} + valid: ${{ steps.resolve.outputs.valid }} + steps: + - name: Resolve repository and pull request + id: resolve + env: + GH_TOKEN: ${{ github.token }} + HEAD_BRANCH: ${{ github.ref_name }} + HEAD_OWNER: ${{ github.repository_owner }} + HEAD_REPOSITORY: ${{ github.repository }} + HEAD_SHA: ${{ github.sha }} + run: | + set -euo pipefail + + echo "applicable=false" >> "${GITHUB_OUTPUT}" + echo "valid=false" >> "${GITHUB_OUTPUT}" + + if [[ "${HEAD_REPOSITORY}" == "apache/gravitino" && \ + ("${GITHUB_REF}" == "refs/heads/main" || "${GITHUB_REF}" == refs/heads/branch-*) ]]; then + echo "applicable=true" >> "${GITHUB_OUTPUT}" + echo "base_ref=${GITHUB_REF_NAME}" >> "${GITHUB_OUTPUT}" + echo "valid=true" >> "${GITHUB_OUTPUT}" + exit 0 + fi + + repository_response="$(curl --fail-with-body --silent --show-error \ + --header "Accept: application/vnd.github+json" \ + --header "Authorization: Bearer ${GH_TOKEN}" \ + --header "X-GitHub-Api-Version: 2022-11-28" \ + "${GITHUB_API_URL}/repos/${HEAD_REPOSITORY}")" + source_repository="$(jq -r '.source.full_name // empty' <<< "${repository_response}")" + + if [[ "${HEAD_REPOSITORY}" != "apache/gravitino" && \ + "${source_repository}" != "apache/gravitino" ]]; then + echo "This repository is not in the apache/gravitino fork network; Fork CI is dormant." + exit 0 + fi + + echo "applicable=true" >> "${GITHUB_OUTPUT}" + matches="[]" + match_count=0 + for attempt in 1 2 3; do + response="$(curl --fail-with-body --silent --show-error \ + --header "Accept: application/vnd.github+json" \ + --header "Authorization: Bearer ${GH_TOKEN}" \ + --header "X-GitHub-Api-Version: 2022-11-28" \ + --get \ + --data-urlencode "state=open" \ + --data-urlencode "head=${HEAD_OWNER}:${HEAD_BRANCH}" \ + --data-urlencode "per_page=10" \ + "https://api.github.com/repos/apache/gravitino/pulls";)" + matches="$(jq -r \ + --arg head_sha "${HEAD_SHA}" \ + --arg head_repository "${HEAD_REPOSITORY}" \ + '[.[] | select(.head.sha == $head_sha and .head.repo.full_name == $head_repository)]' \ + <<< "${response}")" + match_count="$(jq -r 'length' <<< "${matches}")" + if [[ "${match_count}" == "1" ]]; then + break + fi + if [[ "${attempt}" != "3" ]]; then + sleep 3 + fi + done + + if [[ "${match_count}" != "1" ]]; then + echo "::error::Expected one open apache/gravitino pull request for ${HEAD_REPOSITORY}:${HEAD_BRANCH} at ${HEAD_SHA}, found ${match_count}." + exit 0 + fi + + echo "base_ref=$(jq -r '.[0].base.ref' <<< "${matches}")" >> "${GITHUB_OUTPUT}" + echo "valid=true" >> "${GITHUB_OUTPUT}" + + - name: Reject an unmatched Apache fork branch Review Comment: No longer fails. A push without an open PR now tests against the inferred base (the default branch or one of the latest release branches, whichever the branch is fewest commits ahead of), and the result is mirrored once the PR is opened. Syncing a fork's `main` is such a push and passes (https://github.com/yuqi1129/gravitino/actions/runs/39581500994 ran the full suites on current `main`). Branches in `apache/gravitino` without a PR, such as `cherry-pick-*`, stay dormant. ########## .github/workflows/build.yml: ########## @@ -216,36 +214,30 @@ jobs: ./gradlew "${gradle_args[@]}" - name: Fetch base branch for coverage diff - if: github.event_name == 'pull_request' - run: git fetch origin ${{ github.base_ref }} --depth=1 + if: steps.build-checkout.outputs.synced == 'true' + run: git fetch origin ${{ steps.build-checkout.outputs.base-ref }} --depth=1 - name: Generate Coverage Report id: coverage - if: github.event_name == 'pull_request' + if: steps.build-checkout.outputs.synced == 'true' run: | python3 dev/ci/jacoco_report.py \ - --base-ref "${{ github.base_ref }}" \ - --head-sha "${{ github.event.pull_request.head.sha }}" \ - --repo-url "${{ github.event.pull_request.head.repo.html_url }}" \ + --base-ref "${{ steps.build-checkout.outputs.base-ref }}" \ + --head-sha "${{ github.sha }}" \ + --repo-url "${{ github.server_url }}/${{ github.repository }}" \ --min-overall 40 \ --min-changed 60 \ --output coverage-report.md - - name: Save PR number - if: github.event_name == 'pull_request' && steps.coverage.outputs.has_reports == 'true' - run: echo "${{ github.event.pull_request.number }}" > pr-number.txt - - name: Upload Coverage Report - if: github.event_name == 'pull_request' && steps.coverage.outputs.has_reports == 'true' + if: steps.build-checkout.outputs.synced == 'true' && steps.coverage.outputs.has_reports == 'true' uses: actions/upload-artifact@v7 with: name: coverage-report - path: | - coverage-report.md - pr-number.txt + path: coverage-report.md Review Comment: Kept on purpose: `branch-*` PRs still run `build.yml` on `pull_request` in `apache/gravitino`, and `workflow_run` uses the file on `main`, so `coverage-comment.yml` keeps serving them. For PRs to `main`, coverage is in the `build / coverage` job log and the `coverage-report` artifact in the fork; this is documented in `CONTRIBUTING.md` (Known limitations of Fork CI). ########## .github/workflows/fork-ci-status-update.yml: ########## @@ -0,0 +1,111 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. +# +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +name: Update Fork CI Status + +on: + schedule: + - cron: '*/15 * * * *' + +permissions: + actions: read + checks: write + pull-requests: read + +jobs: + update: + name: Update fork CI checks + if: github.repository == 'apache/gravitino' + runs-on: ubuntu-slim + steps: + - name: Synchronize fork CI checks + uses: actions/github-script@v7 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const prs = await github.paginate( + 'GET /repos/{owner}/{repo}/pulls', + { owner: context.repo.owner, repo: context.repo.repo, state: 'open', per_page: 100 }); + + for (const pr of prs) { + if (!pr.head.repo) continue; + const owner = pr.head.repo.owner.login; + const repo = pr.head.repo.name; + let runs; + try { + runs = await github.paginate( Review Comment: Addressed. The sweep reads fork runs with `head_sha`, wraps each PR in its own `try/catch`, and keeps the previous sweep's state, so settled PRs cost no calls. Since `e062791` it runs every 5 minutes, but only running checks are read on every sweep; other PRs at most every 15 minutes. In the sandbox a sweep took 4 to 8 calls for 3 PRs, and each sweep logs the `GITHUB_TOKEN` rate limit so we can watch it after merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
