Hello!
Please consider the following messages:
[2021-04-21T14:55:09,203][WARN
][tcp-comm-worker-#1%EDIFCustomer%][TcpCommunicationSpi] Connect timed out
(consider increasing 'failureDetectionTimeout' configuration property)
[addr=/10.40.0.78:47100, failureDetectionTimeout=60000]
[2021-04-21T14:55:09,203][WARN
][tcp-comm-worker-#1%EDIFCustomer%][TcpCommunicationSpi] Failed to connect
to a remote node (make sure that destination node is alive and operating
system firewall is disabled on local and remote hosts) [addrs=[/
10.40.0.78:47100, /127.0.0.1:47100]]
I can see that communication threads will spend a lot of time on connect(),
indicating network or firewall issues:
Thread [name="tcp-comm-worker-#1%EDIFCustomer%", id=365, state=RUNNABLE,
blockCnt=1294, waitCnt=12569]
at sun.nio.ch.Net.poll(Native Method)
at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:954)
- locked java.lang.Object@65ec5b09
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:110)
- locked java.lang.Object@9ecd49c
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3299)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
at
o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237)
at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
I think this is the root cause. Your server node cannot connect to some of
your remaining nodes' communication port. Maybe your server node is behind
NAT or firewall. Consider enabling NAT traversal feature:
https://ignite.apache.org/docs/latest/clustering/running-client-nodes-behind-nat
Regards,
--
Ilya Kasnacheev
чт, 22 апр. 2021 г. в 21:58, Gangaiah Gundeboina <gangaiah.ya...@gmail.com>:
> HI Ilya,
>
> Please find attached full log file.
>
> Regards,
> Gangaiah
>
> server_log.zip
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2396/server_log.zip>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
