voonhous opened a new issue, #20471: URL: https://github.com/apache/hudi/issues/20471
Part of #19910. Depends on #19612 (merge queue). **Problem.** Every fork PR runs the full Java CI (18 test jobs) on the shared ASF runner pool, where it queues behind other ASF projects. With `strict: false` and only `validate-pr-and-source` required, that run is also tested against a master that may have moved by the time the PR merges, which is how green PRs break compilation on master. **Proposal**, following apache/arrow-rs: 1. **Merge queue (#19612).** The full suite runs on ASF runners against current master plus the PRs queued ahead, and this is the only merge gate. arrow-rs enabled theirs entirely through a `rulesets` entry in `.asf.yaml` (apache/arrow-rs#11097), with no infrastructure to run. 2. **Fork CI.** Fork PRs run the full suite on the contributor's fork (a `push` event on the fork's own runners). The upstream `pull_request` run does lint only, and a small upstream job posts the fork's result on the PR as an advisory `CI (fork)` check (apache/arrow-rs#11788). 3. **Codecov bridge (optional).** Each fork job uploads its `jacoco-report.xml` as an artifact named after the commit. An upstream job holding `CODECOV_TOKEN` matches it to the PR's head commit and uploads it with `override_commit` / `override_pr`. Without the bridge, coverage still arrives from the queue run at merge time; only the PR comment during review is lost. **Order.** Lands before #19892; the rest of the stack rebases onto the new job layout. <details> <summary>Reference setup and open questions</summary> - arrow-rs files: [`ci.yml`](https://github.com/apache/arrow-rs/blob/main/.github/workflows/ci.yml) (selector job, single `Required Checks` aggregate, `merge_group` trigger), [`fork_ci_status.yml`](https://github.com/apache/arrow-rs/blob/main/.github/workflows/fork_ci_status.yml) (the status bridge), [`.asf.yaml`](https://github.com/apache/arrow-rs/blob/main/.asf.yaml) (merge queue ruleset: squash, `max_entries_to_build: 2`, ASF Infra as bypass actor), [contributor guide](https://github.com/apache/arrow-rs/blob/main/CONTRIBUTING.md#pull-requests-from-forks). Example queue run: [40827211943](https://github.com/apache/arrow-rs/actions/runs/40827211943). - arrow-rs shipped the queue first (2026-10-02) and fork CI three weeks later (2026-10-23). - `workflow_run` in apache/hudi does not fire for runs in a fork, so the bridge has to poll (arrow-rs polls every 15 minutes) and match the fork run whose head SHA and branch equal the PR head. - Artifacts of a public repo: listing works without authentication; downloading needs a signed-in token, and a token with only `pull` access works (checked against apache/arrow-rs). Open: whether apache/hudi's `GITHUB_TOKEN` can download a fork's artifact, or a PAT / GitHub App token is needed. - Our workflows filter paths at the trigger (#19890). A required check from a path-filtered workflow stays pending forever when the filter skips it, so the queue needs one unfiltered workflow with routing inside it and an always-reporting aggregate check. - Azure CI is unaffected. </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
