benjava commented on issue #10569:
URL: https://github.com/apache/seatunnel/issues/10569#issuecomment-4011241966
> Thanks for reporting this issue. Based on the logs and configuration, this
appears to be a **port configuration mismatch** rather than a code bug.
>
> ## Root Cause
> The issue is in your docker-compose.yml. You're mixing host ports with
container ports:
>
> ports:
> - 5802:5801 # Host 5802 -> Container 5801
> environment:
> - ST_DOCKER_MEMBER_LIST=st-master,st-worker1:5802,st-worker2:5803
> Problems:
>
> 1. Workers listen on port **5802** inside the container (per
`config/hazelcast-worker.yaml`)
> 2. Your `ST_DOCKER_MEMBER_LIST` references `:5802/:5803`, which are the
**host-mapped ports**
> 3. In Docker overlay networks, containers must communicate using
**internal ports**
>
> ## Solution
> Change `ST_DOCKER_MEMBER_LIST` to use service names with the **container
port (5801)**:
>
> environment:
> - ST_DOCKER_MEMBER_LIST=st-master:5801,st-worker1:5801,st-worker2:5801
> The port mappings (`5802:5801`, `5803:5801`) are only for external access.
Inter-container communication uses port 5801 internally.
>
> Please try this corrected configuration and let us know if the issue
persists.
i use your method at begin;but the work node can't start perfect at all;
`services:
st-master:
image: apache/seatunnel:2.3.12
environment:
- ST_DOCKER_MEMBER_LIST=st-master:5081,st-worker1:5801,st-worker2:5801
entrypoint: >
/bin/sh -c "/opt/seatunnel/bin/seatunnel-cluster.sh -r master"
ports:
- 5801:5801
- 8080:8080
deploy:
replicas: 1
mode: replicated
placement:
constraints:
- "node.hostname==server-200"
restart_policy:
condition: on-failure
st-worker1:
image: apache/seatunnel:2.3.12
environment:
- ST_DOCKER_MEMBER_LIST=st-master:5081,st-worker1:5801,st-worker2:5801
entrypoint: >
/bin/sh -c "/opt/seatunnel/bin/seatunnel-cluster.sh -r worker"
depends_on:
- st-master
deploy:
replicas: 1
mode: replicated
placement:
constraints:
- "node.hostname==server-201"
restart_policy:
condition: on-failure
st-worker2:
image: apache/seatunnel:2.3.12
environment:
- ST_DOCKER_MEMBER_LIST=st-master:5081,st-worker1:5801,st-worker2:5801
entrypoint: >
/bin/sh -c "/opt/seatunnel/bin/seatunnel-cluster.sh -r worker"
depends_on:
- st-master
deploy:
replicas: 1
mode: replicated
placement:
constraints:
- "node.hostname==server-202"
restart_policy:
condition: on-failure`
`
[] 2026-03-06 11:37:13,685 WARN [a.s.e.s.s.s.DefaultSlotService]
[hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource
manager, will retry later. this address: [172.18.0.3]:5801
[] 2026-03-06 11:37:18,685 WARN [a.s.e.s.s.s.DefaultSlotService]
[hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource
manager, will retry later. this address: [172.18.0.3]:5801
[] 2026-03-06 11:37:19,390 INFO [c.h.i.s.t.TcpServerConnection ]
[hz.main.IO.thread-in-2] - [172.18.0.3]:5801 [seatunnel] [5.1] Initialized new
cluster connection between /10.0.16.6:5801 and /10.0.16.4:34075
[] 2026-03-06 11:37:19,398 INFO [.o.SplitBrainMergeValidationOp]
[hz.main.priority-generic-operation.thread-0] - [172.18.0.3]:5801 [seatunnel]
[5.1] Ignoring join check from [172.18.0.3]:5801, because this node is not
joined to a cluster yet...
[] 2026-03-06 11:37:19,398 ERROR [.o.SplitBrainMergeValidationOp]
[hz.main.priority-generic-operation.thread-0] - [172.18.0.3]:5801 [seatunnel]
[5.1] Target is this node! -> [172.18.0.3]:5801
java.lang.IllegalArgumentException: Target is this node! -> [172.18.0.3]:5801
at
com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.checkTarget(OutboundResponseHandler.java:226)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendNormalResponse(OutboundResponseHandler.java:125)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendResponse(OutboundResponseHandler.java:88)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.Operation.sendResponse(Operation.java:483)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.call(OperationRunnerImpl.java:283)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:248)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
~[seatunnel-starter.jar:2.3.12]
[] 2026-03-06 11:37:19,399 WARN [.h.s.i.o.i.OperationRunnerImpl]
[hz.main.priority-generic-operation.thread-0] - [172.18.0.3]:5801 [seatunnel]
[5.1] While sending op error... op:
com.hazelcast.internal.cluster.impl.operations.SplitBrainMergeValidationOp{serviceName='hz:core:clusterService',
identityHash=43890303, partitionId=-1, replicaIndex=0, callId=10,
invocationTime=1772797039397 (2026-03-06 11:37:19.397), waitTimeout=-1,
callTimeout=60000,
tenantControl=com.hazelcast.spi.impl.tenantcontrol.NoopTenantControl@0}, error:
java.lang.IllegalArgumentException: Target is this node! -> [172.18.0.3]:5801
java.lang.IllegalArgumentException: Target is this node! ->
[172.18.0.3]:5801, response: ErrorResponse{callId=10, urgent=true,
cause=java.lang.IllegalArgumentException: Target is this node! ->
[172.18.0.3]:5801}
at
com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.send(OutboundResponseHandler.java:113)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OutboundResponseHandler.sendResponse(OutboundResponseHandler.java:96)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.Operation.sendResponse(Operation.java:483)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.sendResponseAfterOperationError(OperationRunnerImpl.java:426)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.handleOperationError(OperationRunnerImpl.java:420)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:253)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:471)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:197)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:137)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.executeRun(OperationThread.java:123)
~[seatunnel-starter.jar:2.3.12]
at
com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:102)
~[seatunnel-starter.jar:2.3.12]
[] 2026-03-06 11:37:23,685 WARN [a.s.e.s.s.s.DefaultSlotService]
[hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource
manager, will retry later. this address: [172.18.0.3]:5801
[] 2026-03-06 11:37:28,695 WARN [a.s.e.s.s.s.DefaultSlotService]
[hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource
manager, will retry later. this address: [172.18.0.3]:5801
[] 2026-03-06 11:37:33,689 WARN [a.s.e.s.s.s.DefaultSlotService]
[hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource
manager, will retry later. this address: [172.18.0.3]:5801
[] 2026-03-06 11:37:38,700 WARN [a.s.e.s.s.s.DefaultSlotService]
[hz.main.seaTunnel.slotService.thread] - failed send heartbeat to resource
manager, will retry later. this address: [172.18.0.3]:5801
[] 2026-03-06 11:37:38,916 ERROR [c.h.i.i.Node ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Could not join cluster. Shutting down now!
[] 2026-03-06 11:37:38,916 INFO [c.h.c.LifecycleService ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] [172.18.0.3]:5801 is SHUTTING_DOWN
[] 2026-03-06 11:37:38,917 INFO [c.h.i.i.Node ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Shutting down connection manager...
[] 2026-03-06 11:37:38,917 WARN [c.h.i.i.Node ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Terminating forcefully...
[] 2026-03-06 11:37:38,917 INFO [c.h.i.s.t.TcpServerConnection ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Connection[id=3,
/10.0.16.6:5801->/10.0.16.10:33247, qualifier=null, endpoint=[172.18.0.3]:5801,
remoteUuid=70b1847d-3be8-4fee-a8a3-17400c5f3b73, alive=false,
connectionType=MEMBER, planeIndex=0] closed. Reason: TcpServer is stopping
[] 2026-03-06 11:37:38,917 INFO [c.h.i.s.t.TcpServerConnection ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Connection[id=4,
/10.0.16.6:5801->/10.0.16.4:34075, qualifier=null, endpoint=[172.18.0.3]:5801,
remoteUuid=1bd655b9-cc7f-40a8-94ce-4de32edcf6f7, alive=false,
connectionType=MEMBER, planeIndex=0] closed. Reason: TcpServer is stopping
[] 2026-03-06 11:37:38,918 INFO [c.h.i.i.Node ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Shutting down node engine...
[] 2026-03-06 11:37:38,923 INFO [.s.s.o.e.j.s.AbstractConnector] [main] -
Stopped ServerConnector@4a37191a{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
[] 2026-03-06 11:37:38,923 INFO [o.a.s.s.o.e.j.s.session ] [main] -
node0 Stopped scavenging
[] 2026-03-06 11:37:38,924 INFO [a.s.s.o.e.j.s.h.ContextHandler] [main] -
Stopped o.a.s.s.o.e.j.s.ServletContextHandler@18b74ea{/,null,STOPPED}
[] 2026-03-06 11:37:38,925 INFO [.c.c.DefaultClassLoaderService] [main] -
close classloader service
[] 2026-03-06 11:37:38,925 INFO [o.a.s.e.s.EventService ]
[event-forwarder-0] - Event forward thread interrupted
[] 2026-03-06 11:37:38,930 INFO [c.h.i.i.NodeExtension ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Destroying node NodeExtension.
[] 2026-03-06 11:37:38,930 INFO [c.h.i.i.Node ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Hazelcast Shutdown is completed in 13 ms.
[] 2026-03-06 11:37:38,930 INFO [c.h.c.LifecycleService ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] [172.18.0.3]:5801 is SHUTDOWN
[] 2026-03-06 11:37:38,934 WARN [c.h.i.u.p.PhoneHome ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Could not schedule phone home task! Most
probably Hazelcast failed to start.
[] 2026-03-06 11:37:38,935 INFO [c.h.i.i.Node ] [main] -
[172.18.0.3]:5801 [seatunnel] [5.1] Node is already shutting down... Waiting
for shutdown process to complete...
[] 2026-03-06 11:37:38,935 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
===============================================================================
[] 2026-03-06 11:37:38,935 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
Fatal Error,
[] 2026-03-06 11:37:38,935 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
Please submit bug report in https://github.com/apache/seatunnel/issues
[] 2026-03-06 11:37:38,935 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
Reason:Node failed to start!
[] 2026-03-06 11:37:38,936 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
Exception StackTrace:java.lang.IllegalStateException: Node failed to start!
at
com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:126)
at
com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:217)
at
com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:196)
at
org.apache.seatunnel.engine.server.SeaTunnelServerStarter.initializeHazelcastInstance(SeaTunnelServerStarter.java:70)
at
org.apache.seatunnel.engine.server.SeaTunnelServerStarter.createHazelcastInstance(SeaTunnelServerStarter.java:52)
at
org.apache.seatunnel.core.starter.seatunnel.command.ServerExecuteCommand.execute(ServerExecuteCommand.java:84)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at
org.apache.seatunnel.core.starter.seatunnel.SeaTunnelServer.main(SeaTunnelServer.java:34)
Exception in thread "main" java.lang.IllegalStateException: Node failed to
start!
at
com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:126)
at
com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:217)
at
com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:196)
at
org.apache.seatunnel.engine.server.SeaTunnelServerStarter.initializeHazelcastInstance(SeaTunnelServerStarter.java:70)
at
org.apache.seatunnel.engine.server.SeaTunnelServerStarter.createHazelcastInstance(SeaTunnelServerStarter.java:52)
at
org.apache.seatunnel.core.starter.seatunnel.command.ServerExecuteCommand.execute(ServerExecuteCommand.java:84)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at
org.apache.seatunnel.core.starter.seatunnel.SeaTunnelServer.main(SeaTunnelServer.java:34)
[] 2026-03-06 11:37:38,936 ERROR [o.a.s.c.s.SeaTunnel ] [main] -
===============================================================================
`
**PS: the cluster start with docker swarm, not docker compse! and i run on
docker compose ok!**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
