nzw921rx opened a new issue, #11583: URL: https://github.com/apache/seatunnel/issues/11583
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened Many SeaTunnel E2E tests publish container service ports to fixed host ports through `setPortBindings(...)`, Docker Compose `ports`, or equivalent configurations. This introduces host-wide port dependencies into the test lifecycle. If a test fails before its containers are fully cleaned up, a later test invocation in the same job may be unable to start another container using the same host port. Two Databend E2E jobs provide direct evidence of this problem: - https://github.com/apache/seatunnel/actions/runs/30025357669/job/89273887251 - https://github.com/apache/seatunnel/actions/runs/30109335763/job/89535147141 In both jobs, an earlier Flink execution encountered a resource-related failure. A later test invocation then failed while starting another Flink container: ```text Bind for 0.0.0.0:8081 failed: port is already allocated ``` The fixed `8081:8081` binding was not necessarily the original cause of the Flink resource failure. However, it amplified the failure because the next invocation could not start after the previous container failed to release the port completely. The same structural risk exists in other E2E modules that publish fixed host ports, including Databend, Druid, Redis Cluster, HBase, Hive, IoTDB, JDBC databases, DynamoDB, and others. Some historical failures in affected lanes, such as `all-connectors-it-4`, Redis, and `all-connectors-it-7`, could not be diagnosed because their logs did not expose the first causal exception. These failures are investigation candidates only and are not treated as confirmed port-collision incidents. ### Scope clarification Java 8 and Java 11 matrix jobs using `ubuntu-latest` normally run on separate GitHub-hosted runners. They do not share the same host or Docker daemon, so this issue is not primarily about Java 8 and Java 11 jobs competing for the same port. The actual risk exists within a single runner or job when: - an earlier test fails before its container is fully removed; - a parameterized test starts another container using the same host port; - multiple test resources are active in the same JVM; - Docker container or port cleanup is delayed; - or tests run on a self-hosted environment sharing a Docker daemon. ## Expected behavior SeaTunnel E2E containers should not depend on globally fixed host ports. - Container-to-container communication should use Docker network aliases and container service ports. - Clients running in the test JVM should use `container.getHost()` and `container.getMappedPort(...)`. - Protocols that advertise stable service ports, such as Redis Cluster, HBase, and TiDB, should use isolated forwarding or endpoint rewriting. - A failed test invocation should not prevent later invocations from starting their containers. ### SeaTunnel Version `dev`, before the dynamic-port changes proposed in #11574. ### SeaTunnel Config ```conf Not applicable. The problem is located in the E2E Testcontainers and Docker Compose infrastructure. ``` ### Running Command ```shell A representative CI command is: ./mvnw -B -T 1 verify \ -DskipUT=true \ -DskipIT=false \ -Dlicense.skipAddThirdParty=true \ -Dskip.ui=true \ --no-snapshot-updates \ -pl <affected-e2e-module> -am -Pci ``` ### Error Exception ```log Bind for 0.0.0.0:8081 failed: port is already allocated Depending on the affected service, the same problem may also appear as: Ports are not available Address already in use BindException ContainerLaunchException ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version java8 and java11 ### Screenshots _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
