Re: binary protocol server side sockets

2014-05-29 Thread Eric Plowe
Michael, The ask is for letting keep alive be configurable for native transport, with Socket.setKeepAlive. By default, SO_KEEPALIVE is false ( http://docs.oracle.com/javase/7/docs/api/java/net/StandardSocketOptions.html#SO_KEEPALIVE). Regards, Eric Plowe On Wed, Apr 9, 2014 at 1:25 PM,

Re: binary protocol server side sockets

2014-04-11 Thread Phil Luckhurst
We are also seeing this in our development environment. We have a 3 node Cassandra 2.0.5 cluster running on Ubuntu 12.04 and are connecting from a Tomcat based application running on Windows using the 2.0.0 Cassandra Java Driver. We have setKeepAlive(true) when building the cluster in the

Re: binary protocol server side sockets

2014-04-11 Thread Chris Lohfink
TCP keep alives (by the setTimeout) are notoriously useless... The default 2 hours is generally far longer then any timeout in NAT translation tables (generally ~5 min) and even if you decrease the keep alive to a sane value a log of networks actually throw away TCP keep alive packets. You see

Re: binary protocol server side sockets

2014-04-11 Thread Phil Luckhurst
We have considered this but wondered how well it would work as the Cassandra Java Driver opens multiple connections internally to each Cassandra node. I suppose it depends how those connections are used internally, if it's round robin then it should work. Perhaps we just need to to try it. --

Re: binary protocol server side sockets

2014-04-11 Thread Nate McCall
Out of curiosity, any folks seeing backups in the send or receive queues via netstat while this is happening? (netstat -tulpn for example) I feel like I had this happen once and it ended up being a sysconfig tuning issue (net.core.* and net.ipv4.* stuff specifically). Can't seem to find anything

Re: binary protocol server side sockets

2014-04-11 Thread Eric Plowe
The situation I am seeing is this: To access my companies development environment I need to VPN. I do some development on the application, and for some reason my VPN drops, but I had established connections to my development cassandra server. When I reconnect and check netstat I see the

Re: binary protocol server side sockets

2014-04-10 Thread Eric Plowe
I am having the exact same issue. I see the connections pile up and pile up, but they never seem to come down. Any insight into this would be amazing. Eric Plowe On Wed, Apr 9, 2014 at 4:17 PM, graham sanderson gra...@vast.com wrote: Thanks Michael, Yup keepalive is not the default. It is

Re: binary protocol server side sockets

2014-04-09 Thread DuyHai Doan
Hello Graham You can use the following code with the official Java driver: SocketOptions socketOptions = new SocketOptions(); socketOptions.setKeepAlive(true); Cluster.builder().addContactPoints(contactPointsList) .withPort(cql3Port)

Re: binary protocol server side sockets

2014-04-09 Thread graham sanderson
Thanks, but I would think that just sets keep alive from the client end; I’m talking about the server end… this is one of those issues where there is something (e.g. switch, firewall, VPN in between the client and the server) and we get left with orphaned established connections to the server

Re: binary protocol server side sockets

2014-04-09 Thread Michael Shuler
On 04/09/2014 11:39 AM, graham sanderson wrote: Thanks, but I would think that just sets keep alive from the client end; I’m talking about the server end… this is one of those issues where there is something (e.g. switch, firewall, VPN in between the client and the server) and we get left with

Re: binary protocol server side sockets

2014-04-09 Thread graham sanderson
Michael, it is not that the connections are being dropped, it is that the connections are not being dropped. These server side sockets are ESTABLISHED, even though the client connection on the other side of the network device is long gone. This may well be an issue with the network device (it

Re: binary protocol server side sockets

2014-04-09 Thread Michael Shuler
On 04/09/2014 12:41 PM, graham sanderson wrote: Michael, it is not that the connections are being dropped, it is that the connections are not being dropped. Thanks for the clarification. These server side sockets are ESTABLISHED, even though the client connection on the other side of the

Re: binary protocol server side sockets

2014-04-09 Thread graham sanderson
Thanks Michael, Yup keepalive is not the default. It is possible they are going away after nf_conntrack_tcp_timeout_established; will have to do more digging (it is hard to tell how old a connection is - there are no visible timers (thru netstat) on an ESTABLISHED connection))… This is