Hi Maxime, On Thu, Oct 28, 2010 at 01:15:31PM -0400, Maxime Ducharme wrote: > > - check sys.net.core.somaxconn. If it's 128, then your TCP stack is not > > tuned for a high connection rate, and you're surely dropping incoming > > connections from time to time. Try to first increase that single > > parameter to 10000, restart haproxy and check if it changes anything. > > > > was set to 128. Raised to value to 10000 and we see better results now. > A new problem appeared tough which is : > Oct 28 19:07:40 v-2-fg09-d861-15 kernel: [735810.205858] TCP: drop open > request from 1.1.1.1/42274 > Oct 28 19:07:45 v-2-fg09-d861-15 kernel: [735815.237132] TCP: drop open > request from 1.1.1.2/2847 > Oct 28 19:07:50 v-2-fg09-d861-15 kernel: [735820.276368] TCP: drop open > request from 1.1.1.3/3925 > Oct 28 19:07:55 v-2-fg09-d861-15 kernel: [735825.308858] TCP: drop open > request from 1.1.1.4/49952 > ... > > I also see unreplied SYNs in netstat : > # netstat -an |grep SYN_RECV |grep -cv grep > 1426 > > Now I am taking a look at tcp_max_syn_backlog value, I am thinking of > raising this value also but I would like to have your opinion. We see > this issue when req/s get to 2300/s, only in peak time of day. Rest of > day is ok and response time is excellent.
yes you should raise it. The socket's queue is set to the minimum of (somaxconn, tcp_max_syn_backlog, haproxy's backlog). Haproxy's backlog is set to the frontend's maxconn by default. Generally it makes sense to use values around 10-20000. Avoid using too large values because in case of a SYN flood attack, your system will take a lot of time to decide to activate SYN cookies and will die under the load caused by the SYN-ACK retransmits. Regards, Willy
