jjj-n opened a new pull request, #12201:
URL: https://github.com/apache/seatunnel/pull/12201
### Purpose of this pull request
Fixes #12120.
After a master switch, rebuilding the pending queue from unordered `IMap`
entries can change the order of jobs waiting for resources. Submission
timestamps cannot recover that order because jobs initialize concurrently
before entering the queue.
This change records an enqueue sequence in `JobInfo`. Sequence allocation,
replicated metadata publication, and local queue insertion are serialized
together. The successor restores active jobs first, rebuilds the `WAIT` backlog
in sequence order, and appends new submissions after restoration. Restored
waiting jobs initialize their `JobMaster` when they reach the queue head;
status, history, diagnostics, and completion queries can inspect the backlog
without constructing every execution plan.
Cancellation and stopping materialize only the selected waiting job and
complete its existing waiters. Stepping down interrupts local waiters without
materializing the backlog. A failed deferred initialization completes
exceptionally and releases the queue head.
### Does this PR introduce _any_ user-facing change?
Yes. Under `schedule-strategy: WAIT`, waiting jobs retain their relative
resource-admission order across master failover. FIFO does not imply completion
order, and already-running jobs recover before the waiting backlog.
The existing two-argument `JobInfo` constructor and serialized field prefix
are preserved. Metadata without an enqueue sequence is read as sequence zero,
placed after recorded admissions in initialization-timestamp/job-ID order, and
assigned a sequence for subsequent failovers. The original order of legacy
entries cannot be reconstructed. The guarantee requires surviving replicated
metadata and masters that support the sequence; older masters in a
mixed-version cluster do not preserve it.
English and Chinese recovery documentation describe these rules. Initial
submission still initializes before acknowledgement. The broader two-phase
admission design in #12119 remains separate.
### How was this patch tested?
Validation passed with JDK 17: 42 focused server tests and 4 metadata
compatibility tests, with no failures, errors, or skips. Spotless apply/check
and `-DskipTests verify` passed for both affected engine modules. Server tests
ran in `maven:3.9.11-eclipse-temurin-17`; this validation does not include the
full repository or connector E2E suite.
New coverage includes:
- Legacy/current `JobInfo` serialization using safe and unsafe Hazelcast
inputs, unchanged legacy field prefix, and truncated metadata rejection.
- Concurrent admissions with the first metadata publication blocked, proving
a later admission cannot overtake it even when submission timestamps have the
opposite order.
- Deferred initialization, non-initializing queries, concurrent
scheduling/cancellation, step-down during initialization, and failed
initialization.
- A real embedded Hazelcast cluster with 50 waiting jobs, abrupt
active-member termination, queue-order recovery, a new submission, cancellation
and stopping with completion waiters, and a second active-member termination.
Membership and partition safety are checked before fault injection.
Existing regression coverage includes coordinator generation changes,
WAIT/REJECT scheduling, interrupted insertion, operation-thread submission
acknowledgement, restored state notifications, DAG queries, queue behavior,
history, and cancellation. The affected mock-coordinator fixtures activate
their master flag after stopping their background schedulers, and the
restore-insertion fixture waits for the coordinator to become ready.
To build the required modules and run the new tests from a clean checkout:
```bash
./mvnw -pl seatunnel-engine/seatunnel-engine-server -am -DskipTests install
./mvnw -pl seatunnel-engine/seatunnel-engine-core
-Dtest=JobInfoCompatibilityTest test
./mvnw -pl seatunnel-engine/seatunnel-engine-server \
-Dtest=PendingJobInfoTest,PendingJobOrderTest,CoordinatorServiceTest#testConcurrentAdmissionsCannotOvertakeBlockedMetadataPublication
\
'-Dsurefire.jvm.args=-Dfile.encoding=UTF-8 -Xmx2g
-XX:ActiveProcessorCount=4 --add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED' \
test
```
The `--add-opens` options above are for JDK 17.
### Check list
- [x] English and Chinese recovery documentation updated.
- [x] No new dependencies or binary packages.
- [x] Existing constructor and legacy metadata remain readable; ordering
limits for legacy and mixed-version clusters are documented.
- [x] Changes are limited to Zeta pending-queue ordering and its regression
coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
