Hi Reid, I don't have anymore this loading problem. I solved by changing the Cassandra Driver Configuration. Now my cluster is pretty stable and I don't have machines with crazy CPU Load. The only thing not urgent but I need to investigate is the number of ESTABLISHED TCP connections. I see just one node having 7K TCP connections ESTABLISHED while the others are having around 4-6K connection opened. So the newest nodes added into the cluster have a higher number of ESTABLISHED TCP connections.
default['cassandra']['sysctl'] = { 'net.ipv4.tcp_keepalive_time' => 60, 'net.ipv4.tcp_keepalive_probes' => 3, 'net.ipv4.tcp_keepalive_intvl' => 10, 'net.core.rmem_max' => 16777216, 'net.core.wmem_max' => 16777216, 'net.core.rmem_default' => 16777216, 'net.core.wmem_default' => 16777216, 'net.core.optmem_max' => 40960, 'net.ipv4.tcp_rmem' => '4096 87380 16777216', 'net.ipv4.tcp_wmem' => '4096 65536 16777216', 'net.ipv4.ip_local_port_range' => '10000 65535', 'net.ipv4.tcp_window_scaling' => 1, 'net.core.netdev_max_backlog' => 2500, 'net.core.somaxconn' => 65000, 'vm.max_map_count' => 1048575, 'vm.swappiness' => 0 } These are my tweaked value and I used the values recommended from datastax. Do you have something different? Best, Sergio Il giorno mer 30 ott 2019 alle ore 13:27 Reid Pinchback < rpinchb...@tripadvisor.com> ha scritto: > Oh nvm, didn't see the later msg about just posting what your fix was. > > R > > > On 10/30/19, 4:24 PM, "Reid Pinchback" <rpinchb...@tripadvisor.com> > wrote: > > Message from External Sender > > Hi Sergio, > > Assuming nobody is actually mounting a SYN flood attack, then this > sounds like you're either being hammered with connection requests in very > short periods of time, or your TCP backlog tuning is off. At least, > that's where I'd start looking. If you take that log message and google it > (Possible SYN flooding... Sending cookies") you'll find explanations. Or > just googling "TCP backlog tuning". > > R > > > On 10/30/19, 3:29 PM, "Sergio Bilello" <lapostadiser...@gmail.com> > wrote: > > > > >Oct 17 00:23:03 prod-personalization-live-data-cassandra-08 > kernel: TCP: request_sock_TCP: Possible SYN flooding on port 9042. Sending > cookies. Check SNMP counters. > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > >