On 13 окт. 2014 г., at 14:37, Lukas Tribus <luky...@hotmail.com> wrote:
> Hi Dmitry, > > > >> I am using haproxy-1.5.4 on FreeBSD-10. >> >> Upon startup, it looks like this: >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 8459 www 1 37 0 86376K 28824K CPU16 16 0:16 26.56% haproxy >> >> (about 80MB RES) > > Its 80MB SIZE and 28M RES here. > > > >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 82720 www 1 36 0 244M 108M CPU29 29 29.2H 26.95% haproxy >> >> (244MB RES). > > Its 244M SIZE and 108M RES. So 108M of real RAM used here. > > Yes, I am sorry, I meant SIZE. > >> When I do reload, I see that old process is in swread state for some time, >> and >> swap usage decreases for about 150MB when old process finishes. >> >> Does it mean memory leak is somewhere? Any additional information I could >> provide will be useful? > > Share you configuration, especially maxconn related stuff, the output of defaults log global mode tcp balance roundrobin maxconn 10000 option abortonclose option allbackups # option dontlog-normal # option dontlognull option redispatch option tcplog # option log-separate-errors option socket-stats retries 4 timeout check 500ms timeout client 15s timeout connect 100ms timeout http-keep-alive 3s timeout http-request 5s timeout queue 1s timeout server 15s fullconn 3000 default-server inter 5s downinter 1s fastinter 500ms fall 3 rise 1 slowstart 60s maxqueue 1 minconn 5 maxconn 150 I can send you full config in private e-mail if necessary. > "haproxy -vv" HA-Proxy version 1.5.4 2014/09/02 Copyright 2000-2014 Willy Tarreau <w...@1wt.eu> Build options : TARGET = freebsd CPU = generic CC = cc CFLAGS = -pipe -pipe -g -DFREEBSD_PORTS OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 USE_PC1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.8 Compression algorithms supported : identity, deflate, gzip Built with OpenSSL version : OpenSSL 1.0.1i-freebsd 6 Aug 2014 Running on OpenSSL version : OpenSSL 1.0.1i-freebsd 6 Aug 2014 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.33 2013-05-28 PCRE library supports JIT : yes Built with transparent proxy support using: IP_BINDANY IPV6_BINDANY Available polling systems : kqueue : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use kqueue. > a > nd possibly "show info;show stat;show pools" > from the unix admin socket. > show info: Name: HAProxy Version: 1.5.4 Release_date: 2014/09/02 Nbproc: 1 Process_num: 1 Pid: 32459 Uptime: 4d 6h09m46s Uptime_sec: 367786 Memmax_MB: 0 Ulimit-n: 131218 Maxsock: 131218 Maxconn: 65500 Hard_maxconn: 65500 CurrConns: 508 CumConns: 517986272 CumReq: 602369265 MaxSslConns: 0 CurrSslConns: 16 CumSslConns: 452700 Maxpipes: 0 PipesUsed: 0 PipesFree: 0 ConnRate: 2611 ConnRateLimit: 0 MaxConnRate: 3965 SessRate: 2611 SessRateLimit: 0 MaxSessRate: 3965 SslRate: 4 SslRateLimit: 0 MaxSslRate: 33 SslFrontendKeyRate: 2 SslFrontendMaxKeyRate: 34 SslFrontendSessionReuse_pct: 50 SslBackendKeyRate: 0 SslBackendMaxKeyRate: 0 SslCacheLookups: 74867 SslCacheMisses: 60826 CompressBpsIn: 0 CompressBpsOut: 0 CompressBpsRateLim: 0 ZlibMemUsage: 0 MaxZlibMemUsage: 0 Tasks: 1550 Run_queue: 1 Idle_pct: 55 show pools on freshly started process: Dumping pools usage. Use SIGQUIT to flush them. - Pool pipe (32 bytes) : 19 allocated (608 bytes), 5 used, 3 users [SHARED] - Pool capture (64 bytes) : 0 allocated (0 bytes), 0 used, 1 users [SHARED] - Pool channel (80 bytes) : 766 allocated (61280 bytes), 672 used, 1 users [SHARED] - Pool task (112 bytes) : 1426 allocated (159712 bytes), 1378 used, 1 users [SHARED] - Pool uniqueid (128 bytes) : 0 allocated (0 bytes), 0 used, 1 users [SHARED] - Pool connection (320 bytes) : 424 allocated (135680 bytes), 360 used, 1 users [SHARED] - Pool hdr_idx (416 bytes) : 383 allocated (159328 bytes), 335 used, 1 users [SHARED] - Pool session (864 bytes) : 385 allocated (332640 bytes), 337 used, 1 users [SHARED] - Pool requri (1024 bytes) : 51 allocated (52224 bytes), 22 used, 1 users [SHARED] - Pool buffer (32800 bytes) : 766 allocated (25124800 bytes), 672 used, 1 users [SHARED] Total: 10 pools, 26026272 bytes allocated, 22818112 used. show pools after few days of uptime: Dumping pools usage. Use SIGQUIT to flush them. - Pool pipe (32 bytes) : 961 allocated (30752 bytes), 5 used, 3 users [SHARED] - Pool capture (64 bytes) : 0 allocated (0 bytes), 0 used, 1 users [SHARED] - Pool channel (80 bytes) : 4136 allocated (330880 bytes), 648 used, 1 users [SHARED] - Pool task (112 bytes) : 3109 allocated (348208 bytes), 1367 used, 1 users [SHARED] - Pool uniqueid (128 bytes) : 0 allocated (0 bytes), 0 used, 1 users [SHARED] - Pool connection (320 bytes) : 2537 allocated (811840 bytes), 343 used, 1 users [SHARED] - Pool hdr_idx (416 bytes) : 2068 allocated (860288 bytes), 323 used, 1 users [SHARED] - Pool session (864 bytes) : 2068 allocated (1786752 bytes), 326 used, 1 users [SHARED] - Pool requri (1024 bytes) : 1491 allocated (1526784 bytes), 16 used, 1 users [SHARED] - Pool buffer (32800 bytes) : 4136 allocated (135660800 bytes), 648 used, 1 users [SHARED] Total: 10 pools, 141356304 bytes allocated, 22001680 used. (compare the last line) I can send you "show stat" in private if necessary. > I don't think that its a memory leak, but haproxy its just allocating > what it can according to its maxconn values and doesn't necessarily > free all those buffers when there aren't used anymore. > Problem is that during reload I often get a bunch of similar errors in the log: Oct 8 18:23:42 hostname haproxy[992]: IP:42908 [08/Oct/2014:18:23:38.947] ONE-front ONE-back/backend 93/176/-1/-1/3067 503 212 - - CQ-- 2055/759/480/20/0 0/163 "POST /some/url HTTP/1.1" Oct 8 18:23:44 hostname haproxy[992]: IP:23920 [08/Oct/2014:18:23:39.041] ONE-front ONE-back/<NOSRV> 75/841/-1/-1/4997 503 212 - - CQ-- 2560/1023/589/0/0 0/203 "POST /some/url HTTP/1.1" There are a lot of similar errors and only during reload.