This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/main by this push:
new f319f044e3 ci: speed up merge queue via HEADGREEN and stack selection
(#7042)
f319f044e3 is described below
commit f319f044e38d3e37a9063ea8ea5058f73a2b14b9
Author: Yicong Huang <[email protected]>
AuthorDate: Wed Jul 29 15:46:09 2026 -0400
ci: speed up merge queue via HEADGREEN and stack selection (#7042)
### What changes were proposed in this PR?
Two complementary merge-queue speedups.
**1. Retune the `merge_queue` ruleset in `.asf.yaml`:**
| Parameter | Before | After | Why |
| --- | --- | --- | --- |
| `grouping_strategy` | ALLGREEN | HEADGREEN | Merge a whole batch on
one green head-group run instead of one green run per queued PR; also
cuts flaky-test exposure from N draws per batch to 1. |
| `max_entries_to_build` | 5 | 2 | Under HEADGREEN, non-head speculative
builds no longer gate merging; keep one in flight for pipelining, save
shared ASF runner quota (the pool has been at its 900-job limit lately;
policy caps a project at 20 concurrent jobs, and one full group is
already ~20). |
| `min_entries_to_merge` | 1 | 2 | Batch PRs so one full CI run merges
several at peak times. |
| `min_entries_to_merge_wait_minutes` | 5 | 3 | Short batching window;
off-peak a lone PR waits at most 3 minutes. |
| `check_response_timeout_minutes` | 30 | 45 | Headroom for runner-queue
delays so a late-reporting check is not misjudged as failed. |
**2. Smarter stack selection for `merge_group` runs (precheck.yml).**
Previously merge groups ran every stack unconditionally. Now, per group:
```
merge_group
├─ resolve the group's PRs from squash messages "(#N)" (same convention
as direct-backport-push.yml)
├─ tier 1: group tree == a PR head tree that already passed Required
Checks
│ → byte-identical content was just validated → skip every stack
├─ tier 2: otherwise, union the labels of every PR in the group and apply
│ the same LABEL_STACKS filter a PR event gets
└─ any resolution failure → fail open, run every stack
```
Tier 1 typically fires for a single fresh PR based on the current `main`
tip (or a stacked PR containing the group's other entries) — tree
equality makes the skip content-exact, not heuristic. Backport targets
stay PR-event-only; merge-group backports are still discovered
post-merge by `direct-backport-push.yml`, which already iterates every
commit in a multi-PR push.
### Any related issues, documentation, discussions?
Closes #7041. Ruleset parameters follow the GitHub rulesets
`merge_queue` schema: https://docs.github.com/en/rest/repos/rules
### How was this PR tested?
- Config-only + workflow-logic change; `.asf.yaml` and both workflows
pass YAML parsing, and the embedded github-script block passes `node
--check`.
- The merge-group path fails open on any API/parse error (single
try/catch → full run), so a bad lookup can never skip CI silently.
- Real merge_group behavior can only be observed on `main`; will watch
the first few queue batches after merge (skip/label-union decisions are
logged in the precheck step).
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Fable 5)
---------
Co-authored-by: Claude Fable 5 <[email protected]>
---
.asf.yaml | 10 +--
.github/workflows/precheck.yml | 125 ++++++++++++++++++++++++++++++++--
.github/workflows/required-checks.yml | 6 +-
3 files changed, 129 insertions(+), 12 deletions(-)
diff --git a/.asf.yaml b/.asf.yaml
index 8a23cb6f45..c0f520b666 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -83,12 +83,12 @@ github:
- type: merge_queue
parameters:
merge_method: SQUASH
- max_entries_to_build: 5
- min_entries_to_merge: 1
+ max_entries_to_build: 2
+ min_entries_to_merge: 2
max_entries_to_merge: 5
- min_entries_to_merge_wait_minutes: 5
- grouping_strategy: ALLGREEN
- check_response_timeout_minutes: 30
+ min_entries_to_merge_wait_minutes: 3
+ grouping_strategy: HEADGREEN
+ check_response_timeout_minutes: 45
- type: pull_request
parameters:
allowed_merge_methods:
diff --git a/.github/workflows/precheck.yml b/.github/workflows/precheck.yml
index c98589060b..4ce1d24092 100644
--- a/.github/workflows/precheck.yml
+++ b/.github/workflows/precheck.yml
@@ -19,9 +19,12 @@
# Checks (backport preflight/build) call, so the label -> stack mapping and
# the labeler-wait live in exactly one place. On PR events it waits for the
# Pull Request Labeler to finish, then maps the PR's labels to the build
-# stacks each requires and to the release/* backport targets. Non-PR events
-# (push / merge_group / workflow_dispatch) run every stack and request no
-# backports.
+# stacks each requires and to the release/* backport targets. Push and
+# workflow_dispatch events run every stack. Merge groups resolve the PRs in
+# the group and either skip every stack (group tree identical to a PR head
+# that already passed Required Checks) or apply the same label -> stack
+# filter using the union of the group's PR labels; resolution failures fall
+# open to a full run. Non-PR events request no backports.
name: Precheck
on:
@@ -50,6 +53,9 @@ on:
permissions:
checks: read
+ # contents: read backs the git-data commit lookups (tree SHAs) in the
+ # merge-queue fast path.
+ contents: read
pull-requests: read
jobs:
@@ -100,6 +106,97 @@ jobs:
const eventName = context.eventName;
let labels = [];
+ // When true, the LABEL_STACKS filter below applies to `labels`;
+ // when false the event runs every stack.
+ let applyLabelFilter = false;
+
+ // Merge-queue path, two tiers, evaluated per group:
+ //
+ // 1. Fast path — a merge group whose squash result has the same
+ // git tree as a PR head that already passed Required Checks
+ // would re-validate byte-identical content: skip every stack.
+ // Tree equality only holds for a single-entry group whose PR
+ // was based on the then-current main (or a stacked PR that
+ // contains the group's other entries), so the skip is
+ // content-exact rather than heuristic.
+ // 2. Label union — otherwise, resolve every PR in the group from
+ // its squash commit message (PR_TITLE_AND_DESC squash titles
+ // end with "(#N)", same convention direct-backport-push.yml
+ // relies on) and apply the same label -> stack filter a PR
+ // event gets, using the union of all the group's PR labels.
+ //
+ // Any resolution failure falls open to a full run.
+ let mergeGroupAlreadyValidated = false;
+ if (eventName === "merge_group") {
+ try {
+ const { owner, repo } = context.repo;
+ const mg = context.payload.merge_group;
+ const { data: comparison } = await
github.rest.repos.compareCommits({
+ owner,
+ repo,
+ base: mg.base_sha,
+ head: mg.head_sha,
+ });
+ const prNumbers = comparison.commits.map((c) => {
+ const m = c.commit.message.split("\n",
1)[0].match(/\(#(\d+)\)\s*$/);
+ if (!m) throw new Error(`no PR number in the message of
${c.sha}`);
+ return Number(m[1]);
+ });
+ if (prNumbers.length === 0) {
+ throw new Error(`no commits between ${mg.base_sha} and
${mg.head_sha}`);
+ }
+ const pullRequests = [];
+ for (const pullNumber of prNumbers) {
+ const { data: pr } = await github.rest.pulls.get({
+ owner,
+ repo,
+ pull_number: pullNumber,
+ });
+ pullRequests.push(pr);
+ }
+
+ // Tier 1: tree match against the newest entry's PR head.
+ const headPr = pullRequests[pullRequests.length - 1];
+ const mgTree = (
+ await github.rest.git.getCommit({ owner, repo, commit_sha:
mg.head_sha })
+ ).data.tree.sha;
+ const prTree = (
+ await github.rest.git.getCommit({ owner, repo, commit_sha:
headPr.head.sha })
+ ).data.tree.sha;
+ if (mgTree === prTree) {
+ // The aggregator job in required-checks.yml publishes the
+ // "Required Checks" check run on the PR head.
+ const { data: checkData } = await
github.rest.checks.listForRef({
+ owner,
+ repo,
+ ref: headPr.head.sha,
+ check_name: "Required Checks",
+ });
+ const check = checkData.check_runs[0];
+ if (check && check.status === "completed" &&
check.conclusion === "success") {
+ mergeGroupAlreadyValidated = true;
+ core.info(
+ `Merge group tree matches PR #${headPr.number} head
${headPr.head.sha}, which already passed Required Checks; skipping all stacks.`
+ );
+ }
+ }
+
+ // Tier 2: label union across the group's PRs.
+ if (!mergeGroupAlreadyValidated) {
+ labels = [...new Set(pullRequests.flatMap((pr) =>
pr.labels.map((l) => l.name)))];
+ applyLabelFilter = true;
+ core.info(
+ `Merge group PRs: ${prNumbers.map((n) => `#${n}`).join(",
")}; label union: ${labels.join(", ") || "(none)"}`
+ );
+ }
+ } catch (e) {
+ core.warning(`Merge-group PR resolution failed (${e.message});
running all stacks.`);
+ labels = [];
+ applyLabelFilter = false;
+ mergeGroupAlreadyValidated = false;
+ }
+ }
+
if (eventName === "pull_request") {
// Re-fetch labels: the labeler may have just added some.
const { data: pr } = await github.rest.pulls.get({
@@ -108,6 +205,7 @@ jobs:
pull_number: context.payload.pull_request.number,
});
labels = pr.labels.map((l) => l.name);
+ applyLabelFilter = true;
core.info(`PR labels: ${labels.join(", ") || "(none)"}`);
}
@@ -183,7 +281,7 @@ jobs:
let runInfra = true;
let runPyrightLanguageService = true;
- if (eventName === "pull_request") {
+ if (applyLabelFilter) {
const stacks = new Set();
for (const label of labels) {
for (const stack of LABEL_STACKS[label] || []) {
@@ -204,6 +302,18 @@ jobs:
);
}
+ if (mergeGroupAlreadyValidated) {
+ runFrontend = false;
+ runAmber = false;
+ runAmberIntegration = false;
+ runPlatform = false;
+ runPlatformIntegration = false;
+ runPyamber = false;
+ runAgentService = false;
+ runInfra = false;
+ runPyrightLanguageService = false;
+ }
+
core.setOutput("run_frontend", runFrontend ? "true" : "false");
core.setOutput("run_amber", runAmber ? "true" : "false");
core.setOutput("run_amber_integration", runAmberIntegration ?
"true" : "false");
@@ -219,7 +329,12 @@ jobs:
// a feature that does not exist on any release branch). The veto
// wins even if release/* labels are also present, so adding it
// mid-review cancels the backport apply-check/build on the re-run.
- let targets = [...new Set(labels.filter((n) =>
/^release\/.+$/.test(n)))].sort();
+ // PR events only: merge_group also populates `labels` now, but its
+ // backports are discovered post-merge by direct-backport-push.yml.
+ let targets =
+ eventName === "pull_request"
+ ? [...new Set(labels.filter((n) =>
/^release\/.+$/.test(n)))].sort()
+ : [];
if (labels.includes("no-backport-needed")) {
core.info("no-backport-needed present; skipping all backport
targets.");
targets = [];
diff --git a/.github/workflows/required-checks.yml
b/.github/workflows/required-checks.yml
index f2ee2efb89..ae47606443 100644
--- a/.github/workflows/required-checks.yml
+++ b/.github/workflows/required-checks.yml
@@ -43,8 +43,10 @@ concurrency:
cancel-in-progress: ${{ github.ref != 'refs/heads/main' &&
!startsWith(github.ref, 'refs/heads/release/') }}
jobs:
- # Decide which build stacks this event needs (label union on PR events; all
- # stacks on push/merge_group/dispatch). Shared with Backport Checks via the
+ # Decide which build stacks this event needs (label union on PR events and
+ # merge groups — a merge group unions the labels of every PR it contains,
+ # and skips every stack when its tree matches an already-green PR head;
+ # all stacks on push/dispatch). Shared with Backport Checks via the
# reusable precheck workflow so the label -> stack mapping lives in one place
# (.github/workflows/precheck.yml). The backport_targets output is unused
# here — backport preflight/build moved to the Backport Checks workflow.
