On Wed, May 06, 2015 at 12:03:12PM +0200, Baptiste wrote: > On Wed, May 6, 2015 at 7:15 AM, Krishna Kumar (Engineering) > <[email protected]> wrote: > > Hi Baptiste, > > > > On Wed, May 6, 2015 at 1:24 AM, Baptiste <[email protected]> wrote: > >> > >> > Also, during the test, the status of various backend's change often > >> > between > >> > OK to DOWN, > >> > and then gets back to OK almost immediately: > >> > > >> > > >> > www-backend,nginx-3,0,0,0,10,30000,184,23843,96517588,,0,,27,0,0,180,DOWN > >> > > >> > 1/2,1,1,0,7,3,6,39,,7,3,1,,220,,2,0,,37,L4CON,,0,0,184,0,0,0,0,0,,,,0,0,,,,,6,Out > >> > of local source ports on the system,,0,2,3,92, > >> > >> this error is curious with the type of traffic your generating! > >> Maybe you should let HAProxy manage the source ports on behalf of the > >> server. > >> Try adding the "source 0.0.0.0:1024-65535" parameter in your backend > >> description. > > > > > > Yes, this has fixed the issue - I no longer get state change after an hour > > testing. > > The performance didn't improve though. I will check the sysctl parameters > > that > > were different between haproxy/nginx nodes. > > > > Thanks, > > - Krishna Kumar > > > You have to investigate why this issue happened. > I mean, it is not normal. As Pavlos mentionned, you connection rate is > very low, since you do keep alive and you opened only 500 ports. > > Wait, I know, could you share the keep-alive connection from your nginx > servers? > By default, they close connections every 100 requests... This might be > the root of the issue.
But even then there is no reason why the local ports would remain in use. There definitely is a big problem. It also explains why servers are going up and down all the time and errors are reported. Willy
