Hello Nginx Users, I am running nginx as a TCP load balancer (streams module). I am trying to find a way to redistribute client TCP connections to upstream servers, specifically, rebalance the load on the upstream servers (on some event) w/o having to restart nignx. Clients use persistent TCP connections.
The scenario is as follows Application protocol - There is an application level ack for unit-of-work, so unclean connection termination is ok. Persistent TCP connections - Client opens persistent TCP connections, focus is on near real-time delivery of small amount of data. Connection stays open for days. Maintenance/Downtime - When one of the upstream servers is shutdown for maintenance, all it's client connections break, clients reconnect and switch to one of the remaining active upstream servers. When the upstream is brought back up post maintenance, the load isnt redistributed. ie., existing connections (since they are persistent) remain with other servers. Only new connections can go to the new server. This is more pronounced in 2 upstream server setup...where all connections switch between servers....kind of like thundering herd problem. I would like to have the ability to terminate some/all client connections explicitly and have them reconnect back. I understand that with nginx maintaining 2 connections for every client, there might not be a 'clean' time to close the connection, but since there is an application ack on top...an unclean termination is acceptable. I currently have to restart nginx to rebalance the upstreams which effectively is the same. Restarting all upstream servers and synchronizing their startup is non-trivial. So is signalling all clients(1000s) to close and reconnect. I can achieve this partially by disabling keepalive on nginx listen port (so_keepalive=off) and then having least_conn as the load-balancer method on my upstream. However, this is not desirable in steady state (see persistent TCP connections above), and even though connections get evenly distributed...the load might no be...as idle and busy clients will end up with different upstreams. Nginx plus features like, "On the fly configuration" upstream_conf allows one to change the upstream configuration, but it doesnt affect existing connections, even if a server is marked as down. "Draining of sessions" is only applicable to http requests and not to TCP connections. Did anyone else face such a problem? How did you resolve it? Any pointers will be much appreciated. thanks, balaji -- Balaji Viswanathan Bangalore India
_______________________________________________ nginx mailing list [email protected] http://mailman.nginx.org/mailman/listinfo/nginx
