loustler opened a new pull request, #11718: URL: https://github.com/apache/seatunnel/pull/11718
### Purpose of this pull request `PaimonWithS3IT` can hang for the full 180-minute workflow timeout, and when it does the `paimon-connector-it` job produces no `Tests run:` line at all — just `##[error]The operation was canceled.` two hours later. This bounds each job submission in the class so a recurrence fails in minutes with a message that names the cause. This is **containment, not a fix.** The underlying engine behaviour is #11679 and is untouched here. Raised by @DanielLeens in [review of #11648](https://github.com/apache/seatunnel/pull/11648#pullrequestreview-4873922426), who identified the hang, declined to attribute it to that PR without proof, and asked for the bound to be opened as its own PR against `dev` rather than folded into a dependency-upgrade branch — his reasoning being that a change to E2E test infrastructure is reviewable and revertible independently of a Hadoop shade bump, and that bundling them means reverting one drags the other along. cc @SEZ9, who reviewed #11679. ### What goes wrong today When a batch job is cancelled because Paimon's privilege manager rejects it, the cluster stops reporting the job and the `seatunnel.sh` client never exits. The JUnit thread is blocked inside the container exec that `executeJob` performs, so it never returns and the assertion after it is never reached. Verified on pristine `dev` — branch `claude/dev-baseline-paimon-s3` @ `80fe7dc77`, which is `origin/dev` plus a three-line comment added only so the `changes` filter would schedule `paimon-connector-it` — run [`31100290272`](https://github.com/loustler/seatunnel/actions/runs/31100290272): | leg | `PaimonWithS3IT` | |---|---| | JDK 11 | passed — `Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.481 s` | | JDK 8 | started 13:10:34, no result line, killed at the 180-minute timeout | On the hung leg the engine goes silent 90 seconds in: after 13:12:03 the whole window contains **one** line that is not classloader churn, out of ~6,270, and no `WARN` or `ERROR` at all. At the end the runner has to kill an orphaned `sh` and two `java` processes — the client, still alive after two hours. On the same JDK 8 leg `PaimonIT` completed normally (`Tests run: 26 ... 1,206.934 s`), so this is not a slow or overloaded runner. Full analysis, including the located frame (`SubPlan.updatePipelineState`) and the JDK 11 control, is in [#11679](https://github.com/apache/seatunnel/issues/11679). ### What this PR does Overrides `executeJob(String)` in `PaimonWithS3IT` to run the submission under `Assertions.assertTimeoutPreemptively` with a 5-minute deadline. On timeout the test fails with the conf file name and a pointer to #11679 instead of parking. ```java @Override public Container.ExecResult executeJob(String confFile) throws IOException, InterruptedException { return Assertions.assertTimeoutPreemptively( JOB_EXECUTION_TIMEOUT, () -> super.executeJob(confFile), () -> "Job " + confFile + " did not return within ..."); } ``` **Why 5 minutes.** Measured, not guessed. A green run of this class completes all four tests — eight job submissions — in ~120 s, so one submission normally costs roughly 15 seconds. Five minutes is about 20× that, which leaves ample room for a contended runner while still cutting a hang from 180 minutes to 5. **Why an override rather than eight wrapped call sites.** The bound is a property of this class, not of individual calls. A job submission added to this class later is bounded without anyone having to remember to wrap it, and the diff stays at one method instead of eight edits. **Why only this class.** The hang is demonstrated here and nowhere else. Putting the bound in shared E2E infrastructure would change behaviour for every Testcontainers-based IT in the repo on the strength of one observed case; if the pattern proves useful it can be promoted later, on its own evidence. ### Known limitation, stated rather than hidden `assertTimeoutPreemptively` runs the supplier on a separate thread and **abandons** it on timeout rather than interrupting it, so a genuinely stuck `execInContainer` stays parked until the JVM exits. That is acceptable for the purpose here — the goal is to free the test run and the runner slot, not the thread — but it is a real property of the mechanism and it is noted in the code as well. Interrupting would not help: the thread is blocked in a socket read against the Docker daemon, not in an interruptible wait. ### Does this PR introduce a user-facing change? No. E2E test code only. No production Java, no config option, no public API, no checkpoint or serialization format. ### How was this patch tested? - `mvn test-compile` on `connector-paimon-e2e`, and the override confirmed present in the compiled class: ``` $ javap -p PaimonWithS3IT.class | grep executeJob public org.testcontainers.containers.Container$ExecResult executeJob(java.lang.String) throws java.io.IOException, java.lang.InterruptedException; ``` - `spotless:check` passes on the module. - **The IT itself was not run locally, and I want to be explicit about that rather than imply coverage I don't have.** This repo pins `testcontainer.version` to 1.17.6, whose docker-java negotiates Docker API 1.32; the daemon available to me requires 1.40+, so every Testcontainers-based `*IT` fails at `DockerClientProviderStrategy` before any test body runs. Bumping the pin in a throwaway tree does work, but it perturbs container startup timing — the exact variable this timing-sensitive hang turns on — so a local run would not have been evidence for the behaviour that matters. CI is the check here. - The timeout path is by construction untestable without reproducing the hang, which is non-deterministic. What CI can confirm is the negative: that the bound does not fire on a healthy run. ### Check list * [x] Code changed are covered with tests, or it does not need tests * [x] If any new Jar binary package adding in your PR, please add License Notice according [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md) — no new dependencies * [x] If necessary, please update the documentation to describe the new feature — none needed, test-only * [x] If you are contributing the connector code, please check that the following files are updated — n/a * [x] Update the `docs/en/seatunnel-engine/download-seatunnel.md` if need — n/a -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
