These two server are all have the problem: > Can you post the output of haproxy -vv? HA-Proxy version 1.5.15 2015/11/01 Copyright 2000-2015 Willy Tarreau <wi...@haproxy.org> Build options : TARGET = linux2628 CPU = generic CC = gcc CFLAGS = -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 OPTIONS = USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.3.4 Compression algorithms supported : identity, deflate, gzip Built with OpenSSL version : OpenSSL 1.0.1 14 Mar 2012 Running on OpenSSL version : OpenSSL 1.0.1 14 Mar 2012 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.12 2011-01-15 PCRE library supports JIT : no (USE_PCRE_JIT not set) Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll.
============================================================================================== HA-Proxy version 1.6.2 2015/11/03 Copyright 2000-2015 Willy Tarreau <wi...@haproxy.org> Build options : TARGET = linux2628 CPU = generic CC = gcc CFLAGS = -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 OPTIONS = USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.3.4 Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip") Built with OpenSSL version : OpenSSL 1.0.1 14 Mar 2012 Running on OpenSSL version : OpenSSL 1.0.1 14 Mar 2012 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.12 2011-01-15 PCRE library supports JIT : no (USE_PCRE_JIT not set) Built with Lua version : Lua 5.3.1 Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. As you can see, One is 1.5.15 and another is 1.6.2. Because I degraded one of them from 1.6.2 to 1.5.15, and the problem still exist. > Which OS and kernel are you using? Ubuntu 12.04 LTS Server > Can you post more configuration (not just timeouts)? Of course, but other thins in it is quite normal and no computationally intensive there, please see the attachment. > Are you using SSL? Yes, you can see in the previous mail, one spike is caused by a https request, another caused by a plain http request, So I think not ssl's fault here :-) > Are you using nbproc? No. > Which release did you use previously that worked fine? I'm using 1.5.14 before upgraded to 1.6.2. I believe everythins is ok before the last upgrade. I download it from launchpad.net. > Did you only upgrade haproxy to a new release or did you do any other changes > as well (config changes, OS/kernel changes or tuning)? No, only haproxy is upgraded and this issue exist. Although I've done a "apt-get update && apt-get upgrade" while troubleshooting this problem (The upgrade is done after the problem already exists). -- Best Regards BaiYang baiy...@gmail.com http://baiy.cn **** < END OF EMAIL > **** 发件人: Lukas Tribus 发送时间: 2015-11-19 23:40 收件人: baiyang; haproxy 主题: RE: CPU 100% when waiting for the client timeout Hello! > Sorry for send it again, I just forgot to provide the attachments. > > Affect verions: at least 1.5.15 and 1.6.2 > > Here is the related part in my configuration: > timeout client 15m # 客户端响应超时 > timeout client-fin 10s # 对客户端连接完成 TCP 4 次挥手超时 > timeout connect 5s # HAProxy 向后端 Server 建立连接时的等候超时 > timeout server 15m # 服务器端响应超时 > timeout server-fin 5s # 对后端 Server 连接完成 TCP 4 次挥手超时 > > timeout http-keep-alive 5m # http keep-alive 连接空闲(等待下一请求到达)保持超时(对前后端均有效,影响 > http-reuse 选项) > timeout http-request 15s # http 请求 *头部* 接收超时(对前后端均有效) > > And I found after upgraded to 1.6.2 and 1.5.15, my haproxy server offen > using 100% CPU So you are seeing CPU spikes at 100%, about fifty/fifty between system (kernel) and usr (haproxy). Which OS and kernel are you using? Can you post more configuration (not just timeouts)? Are you using SSL? Are you using nbproc? Can you post the output of haproxy -vv? You are saying the this happens after an upgrade to 1.6.2 or 1.5.15. Which release did you use previously that worked fine? Did you only upgrade haproxy to a new release or did you do any other changes as well (config changes, OS/kernel changes or tuning)? Regards, Lukas
haproxy_more.conf
Description: Binary data