Re: Starvation in striped pool

2019-10-18 Thread Ilya Kasnacheev
Hello! We, we had IGNITE_ENABLE_FORCIBLE_NODE_KILL, but the best solution is, in my opinion, to avoid adding anything unstable to the cluster. Regards, -- Ilya Kasnacheev пт, 18 окт. 2019 г. в 08:35, ihalilaltun : > Hi Ilya, > > From time to time, we have faced exactly the same problem. Is th

Re: Starvation in striped pool

2019-10-18 Thread Mahesh Renduchintala
current setting idleTImeout = LONG_MAX in tcp communication spi has solved the problem for us.

Re: Starvation in striped pool

2019-10-17 Thread ihalilaltun
Hi Ilya, >From time to time, we have faced exactly the same problem. Is there any best practices for handling network issues? What i mean is, if there is any network issues between client/s and server/s we want the cluster keeps living. As for the clients, they can be disconnected from servers. R

Re: Starvation in striped pool

2019-10-11 Thread Ilya Kasnacheev
Hello! I don't think there is any problem with idleConnectionTimeout, but you *should not* use nodes which are not mutually connectible to each other anyway. I can't really comment on the feasibility of dropping client when it can't be reached via Communication. You can start a discussion about i

Re: Starvation in striped pool

2019-10-11 Thread maheshkr76private
>>I'm almost certain is that problem is that server node cannot open a connection to client node (and while it tries, it will reject connection attempts from client node) The default idleTimeout of TCP communication spi is 6 minutes. So I assume, after this timeout, the connection is closed and re

Re: Starvation in striped pool

2019-10-10 Thread Ilya Kasnacheev
Hello! I'm almost certain is that problem is that server node cannot open connection to client node (and while it tries, it will reject connection attempts from client node) clientReconnectDisabled=true will only concern discovery. In your case, there's no problems with discovery, the problem is

Re: Starvation in striped pool

2019-10-09 Thread maheshkr76private
Ilya. What is most mysterious to me is, I disabled reconnect of think client (clientReconnectDisabled=true). Still the server prints, the below, where the same thick client is making an immediate attempt to reconnect back to the cluster, while the previous connecting isn't still successful. [0

Re: Starvation in striped pool

2019-10-09 Thread maheshkr76private
Attached are the logs. In the server log, you will see the thick client continuously pinging server indefinitely...there is no recovery of the thick client. SO, the problem is, we can't even reboot the thick client in a production scenario, as it does even fail (meaning, the configured failure hand

Re: Starvation in striped pool

2019-10-09 Thread Ilya Kasnacheev
Hello! It's hard to say what happens here. What timeout settings do you have? Can you provide complete log from client node as well? Regards, -- Ilya Kasnacheev вт, 8 окт. 2019 г. в 19:25, maheshkr76private : > Hello Ilya > Once connection goes bad between client and server, which configurati

Re: Starvation in striped pool

2019-10-08 Thread maheshkr76private
Hello Ilya Once connection goes bad between client and server, which configuration parameter on the thick client-side would force stop the thick client from pinging server... Tried join timeout, connection timeouts, in communicaiton and discovery spis and nothing ever worked. Have seen in a few

Re: Starvation in striped pool

2019-10-08 Thread Ilya Kasnacheev
Hello! If client node continues to respond via Discovery, server node is not going to drop it by unreachability. This is the default behavior. Regards, -- Ilya Kasnacheev вт, 8 окт. 2019 г. в 15:43, maheshkr76private : > OK. There could have been a temporary network issue between the server a

Re: Starvation in striped pool

2019-10-08 Thread maheshkr76private
OK. There could have been a temporary network issue between the server and client node. However, I was expecting the server node to throw the client out of the cluster and resume normal functioning. But what bothers me is that the server node never recovered after the network issue and finally got

Re: Starvation in striped pool

2019-10-08 Thread Ilya Kasnacheev
Hello! Thread [name="sys-stripe-4-#5", id=27, state=RUNNABLE, blockCnt=0, waitCnt=10004] at sun.nio.ch.Net.poll(Native Method) at sun.nio.ch.SocketChannelImpl.poll(SocketChannelImpl.java:954) at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:110) at o.a.i.spi.c