nzw921rx opened a new issue, #11543:
URL: https://github.com/apache/seatunnel/issues/11543
## Search before asking
- [x] I had searched in the feature issues and found no dedicated Merge
Queue rollout issue. #11513 describes the overall CI redesign and mentions
Merge Queue as part of the target architecture, so this issue is proposed as a
focused child of #11513.
## Description
### Summary
Introduce GitHub Merge Queue for the `dev` branch so that an approved pull
request is validated against the latest `dev` state immediately before GitHub
merges it.
This issue focuses on the Merge Queue rollout, required `merge_group` CI,
failure handling, and risks caused by an unstable `dev` branch. It is a child
task of #11513 rather than a replacement for the overall CI redesign.
### Why is this needed?
PR-head CI only proves that a change works with the base commit used by that
CI run. For a long-lived PR, `dev` may have changed substantially by the time
the PR is ready to merge.
For example:
1. A long-lived PR adds a method or changes an API.
2. Another PR adds the same method, or makes an incompatible change, and is
merged into `dev`.
3. The original PR may still show a successful CI result from an older base
commit.
4. The combined result may contain a duplicate method definition, a
compilation failure, or a behavioral regression.
The recently added required `Build` check with `strict: true` in #11532
protects `dev` by forcing an out-of-date PR to synchronize with the latest base
branch and rerun CI. However, this also creates repeated manual updates and
repeated CI runs whenever `dev` advances.
There is a second reliability problem: some new features or tests require
multiple CI reruns before they become green. A retry-until-green workflow can
hide flaky behavior and allow flaky tests to reach `dev`. Merge Queue does not
fix flaky tests by itself, but it creates a final integration boundary where
the exact candidate to be merged can be tested and rejected without changing
`dev`.
### What is GitHub Merge Queue?
After a PR has received the required approval and passed its entry checks, a
maintainer adds it to the queue instead of merging it directly.
GitHub then:
1. Places the PR in an ordered queue.
2. Creates a temporary merge group containing the latest `dev` plus one or
more queued PRs.
3. Emits the `merge_group` event.
4. Runs the required CI checks on that temporary integrated commit.
5. Automatically merges the PR only if the merge group passes.
6. Blocks or removes the failing PR from the queue while leaving `dev`
unchanged.
The key difference is that GitHub, rather than each contributor, maintains
and tests the final candidate against the latest `dev`.
```mermaid
flowchart TD
A["PR is approved and entry checks pass"] --> B["Maintainer adds PR to
Merge Queue"]
B --> C["GitHub creates a temporary merge group"]
C --> D["Candidate = latest dev + queued PR or PRs"]
D --> E["merge_group CI runs on the exact candidate"]
E --> F{"All required checks pass?"}
F -- "Yes" --> G["GitHub merges automatically"]
F -- "No" --> H["Block or remove the failing PR"]
H --> I["dev remains unchanged"]
```
### Example: detecting integration drift
Assume PR A was opened from `dev@1` and adds `parseValue()`. While PR A
waits for review, PR B adds a method with the same signature and is merged into
`dev@2`.
With final merge-group validation, GitHub tests `dev@2 + PR A`. The
duplicate definition is detected before the candidate reaches `dev`.
```mermaid
flowchart LR
A["PR A based on dev@1"] --> B["Adds parseValue()"]
C["PR B merges into dev@2"] --> D["Also adds parseValue()"]
B --> E["Merge Queue candidate: dev@2 + PR A"]
D --> E
E --> F["Required merge-group Build"]
F --> G{"Compilation succeeds?"}
G -- "No" --> H["PR A is blocked; dev@2 stays unchanged"]
G -- "Yes" --> I["PR A is merged"]
```
### Proposed merge flow
Current flow:
1. A PR receives approval and passes `Build`.
2. If `dev` advances, `strict: true` marks the PR out of date.
3. The PR author or maintainer updates the branch with the latest `dev`.
4. The full required CI runs again.
5. This cycle may repeat until the PR is merged.
Proposed flow:
1. A PR receives the required approval and passes lightweight entry checks.
2. A maintainer adds the PR to Merge Queue.
3. GitHub creates a candidate from the latest `dev` and the queued PR.
4. A trusted upstream workflow triggered by `merge_group` runs the required
integration checks.
5. GitHub automatically merges a green candidate.
6. A failing candidate is blocked or removed, and `dev` is not modified.
```mermaid
flowchart LR
subgraph Current["Current strict branch protection"]
A["PR checks pass"] --> B{"Has dev advanced?"}
B -- "Yes" --> C["Update PR branch"]
C --> D["Run required CI again"]
D --> B
B -- "No" --> E["Merge"]
end
subgraph Proposed["Proposed Merge Queue"]
F["Approval and entry checks"] --> G["Enter queue"]
G --> H["Latest dev + queued PR"]
H --> I["merge_group CI"]
I --> J{"Green?"}
J -- "Yes" --> K["Automatic merge"]
J -- "No" --> L["Block or dequeue"]
end
```
### Benefits
- Validates the exact integrated commit immediately before merge.
- Prevents known compilation and integration failures from being written to
`dev`.
- Removes repeated manual branch synchronization from contributors.
- Serializes or carefully groups concurrent merges, reducing the race
between independently green PRs.
- Gives maintainers a consistent final gate instead of relying on the age of
a PR's last successful run.
- Provides explicit queue metrics such as wait time, throughput, failure
rate, and removal reasons.
- Creates a safer place for exhaustive or trusted checks described in #11513.
### Risks and blocking points while `dev` is unstable
Merge Queue amplifies the health of the base branch: if `dev` or its
required CI is unstable, the queue can stop even when the queued PR is correct.
| Risk | Possible queue behavior | Proposed mitigation |
| --- | --- | --- |
| A flaky test already exists on `dev` | An unrelated PR fails its merge
group and is removed or retried | Measure baseline health, quarantine known
flaky tests, and distinguish base failures from PR-introduced failures |
| Required CI takes several hours | Long queue wait time and very low merge
throughput | Start with one PR per merge group, set a realistic timeout above
observed CI duration, and optimize CI under #11513 |
| The head entry repeatedly fails | Head-of-line blocking prevents later PRs
from progressing | Remove the failing entry automatically or manually and
provide a clear requeue policy |
| Multiple PRs are batched | A failure is harder to attribute to a single PR
| Initially set the build and merge group size to `1`; increase only after
stability is demonstrated |
| Retry-until-green is allowed without classification | Flaky candidates may
eventually merge and add more instability to `dev` | Limit retries to known
infrastructure failures and require investigation for test failures |
| Existing CI only mirrors the contributor-fork `Build` status | A required
check may not be reported for the `merge_group` event | Add a trusted upstream
`merge_group` workflow that reports the required check directly |
| Both generic `push` CI and `merge_group` CI run | Queue candidates consume
duplicate CI resources | Avoid duplicate runs for `gh-readonly-queue/**` when a
dedicated merge-group workflow is enabled |
| The queue timeout is shorter than the current Build duration | Correct
candidates time out before CI completes | Configure the timeout from measured
p95/p99 duration rather than a small default |
### Proposed phased rollout
1. Add a dedicated workflow triggered by `merge_group` and verify that it
reports a uniquely named required check.
2. Test the workflow and ruleset on a shadow branch, without changing `dev`.
3. Start with one PR per merge group and the `ALLGREEN` grouping strategy.
4. Test one successful PR and one intentionally failing PR.
5. Record queue wait time, CI duration, failure causes, retries, and
incorrect removals.
6. Stabilize or quarantine known base-branch flaky tests before making the
queue check required on `dev`.
7. Enable Merge Queue on `dev` with a documented bypass and rollback
procedure.
8. Increase batching only after the single-entry queue is stable.
### Acceptance criteria
- [ ] A workflow handles the GitHub `merge_group` event.
- [ ] The required check is emitted directly for every merge-group candidate.
- [ ] A candidate based on the latest `dev` merges automatically when all
checks pass.
- [ ] A failing candidate does not modify `dev`.
- [ ] A failing queue entry does not permanently block later entries.
- [ ] CI does not run twice unnecessarily for the same queue candidate.
- [ ] Timeout and retry policies are documented and based on measured CI
duration.
- [ ] Known `dev` baseline failures can be distinguished from failures
introduced by a queued PR.
- [ ] The shadow-branch test succeeds before enforcement is enabled on `dev`.
- [ ] Maintainer documentation includes enqueue, dequeue, retry, bypass, and
rollback procedures.
### Non-goals
- Merge Queue alone will not remove flaky tests.
- This issue does not replace the broader CI optimization and trust model
described in #11513.
- This issue does not initially propose batching multiple PRs in one merge
group.
## Usage Scenario
Several approved PRs are ready to merge while `dev` continues to receive
changes.
A maintainer adds them to Merge Queue. For each entry, GitHub creates a
candidate from the latest `dev` and the queued change, then runs the required
integration workflow. A compatible candidate is merged automatically. If a
candidate introduces a duplicate method, compilation error, test regression, or
other integration failure, it is blocked without breaking `dev`, and later
entries can continue according to the dequeue policy.
During the initial rollout, a shadow branch and a single-entry merge group
are used to validate event handling, status reporting, failure isolation, and
timeout behavior before the policy is enforced on `dev`.
## Related issues
- Parent/umbrella issue: https://github.com/apache/seatunnel/issues/11513
- Required Build check on `dev`:
https://github.com/apache/seatunnel/pull/11532
- CI resiliency and flaky E2E handling:
https://github.com/apache/seatunnel/issues/10457
- Long-running connector E2E test standardization:
https://github.com/apache/seatunnel/issues/10995
- GitHub Merge Queue documentation:
https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/configuring-pull-request-merges/managing-a-merge-queue
- Apache Texera Merge Queue configuration example:
https://github.com/apache/texera/blob/main/.asf.yaml
## Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
## Code of Conduct
- [x] I agree to follow this project's Code of Conduct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
