Hi. I play a little bit with proto fcgi and see something what I don't understand.
Hopefully someone can explain it a bit. My php-fpm have the following settings. ``` pm = dynamic pm.max_children = 10 pm.min_spare_servers = 4 pm.start_servers = 5 pm.max_spare_servers = 6 pm.max_requests = 500 ``` The haproxy server line have the following settings. Btw: is fullconn deprecated or removed in 2.2 because it's still in the doc? I don't know if the health check connection is counted in the maxconn value or not, is it? ``` server server1 127.0.0.1:9000 proto fcgi check maxconn 9 minconn 5 maxqueue 0 ``` Now my assumption is that I should never get this message in the php-fpm output because maxconn is 9 and pm.max_children = 10 but I got it. ``` [03-May-2020 07:27:44] WARNING: [pool www] server reached pm.max_children setting (10), consider raising it ``` This means that the connections to php-fpm are higher then maxcon configured in haproxy and this is what I not understand. Here some haproxy logs from the overload. ``` # normal run 127.0.0.1:36466 [03/May/2020:07:37:55.995] myproxy phpservers/server1 0/69/0/2/71 200 66318 - - ---- 50/50/8/4/0 0/9 "GET /pinf.php HTTP/1.1" 127.0.0.1:36462 [03/May/2020:07:37:56.000] myproxy phpservers/server1 0/66/0/2/68 200 66318 - - ---- 50/50/7/4/0 0/8 "GET /pinf.php HTTP/1.1" 127.0.0.1:36488 [03/May/2020:07:37:56.002] myproxy phpservers/server1 0/66/0/2/109 200 66318 - - ---- 50/50/6/4/0 0/8 "GET /pinf.php HTTP/1.1" 127.0.0.1:36470 [03/May/2020:07:37:56.004] myproxy phpservers/server1 0/106/0/2/109 200 66318 - - ---- 50/50/5/4/0 0/8 "GET /pinf.php HTTP/1.1" 127.0.0.1:36546 [03/May/2020:07:37:56.007] myproxy phpservers/server1 0/106/0/2/108 200 66318 - - ---- 50/50/4/4/0 0/8 "GET /pinf.php HTTP/1.1" # I think this a queued request 127.0.0.1:36576 [03/May/2020:07:37:55.334] myproxy phpservers/server1 0/69/0/1402/1471 200 66318 - - ---- 50/50/3/3/0 0/38 "GET /pinf.php HTTP/1.1" 127.0.0.1:36484 [03/May/2020:07:37:55.363] myproxy phpservers/server1 0/41/0/1402/1443 200 66318 - - ---- 50/50/3/3/0 0/39 "GET /pinf.php HTTP/1.1" # normal run 127.0.0.1:36576 [03/May/2020:07:37:56.807] myproxy phpservers/server1 0/0/0/1/1 200 66318 - - ---- 50/50/3/3/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36484 [03/May/2020:07:37:56.808] myproxy phpservers/server1 0/0/0/1/1 200 66318 - - ---- 50/50/3/3/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36576 [03/May/2020:07:37:56.809] myproxy phpservers/server1 0/0/0/1/2 200 66318 - - ---- 50/50/3/3/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36484 [03/May/2020:07:37:56.810] myproxy phpservers/server1 0/0/0/1/2 200 66318 - - ---- 50/50/2/2/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36576 [03/May/2020:07:37:56.812] myproxy phpservers/server1 0/0/0/1/1 200 66318 - - ---- 50/50/3/3/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36484 [03/May/2020:07:37:56.812] myproxy phpservers/server1 0/0/0/1/1 200 66318 - - ---- 50/50/2/2/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36480 [03/May/2020:07:37:55.363] myproxy phpservers/server1 0/41/0/2402/2444 200 66318 - - ---- 50/50/1/1/0 0/40 "GET /pinf.php HTTP/1.1" 127.0.0.1:36486 [03/May/2020:07:37:55.363] myproxy phpservers/server1 0/42/0/2402/2444 200 66318 - - ---- 50/50/0/0/0 0/41 "GET /pinf.php HTTP/1.1" 127.0.0.1:36480 [03/May/2020:07:37:57.808] myproxy phpservers/server1 0/0/0/1/2 200 66318 - - ---- 50/50/1/1/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36486 [03/May/2020:07:37:57.809] myproxy phpservers/server1 0/0/0/1/2 200 66318 - - ---- 50/50/0/0/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36480 [03/May/2020:07:37:57.811] myproxy phpservers/server1 0/0/0/1/2 200 66318 - - ---- 50/50/1/1/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36486 [03/May/2020:07:37:57.811] myproxy phpservers/server1 0/0/0/1/2 200 66318 - - ---- 50/50/0/0/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36480 [03/May/2020:07:37:57.814] myproxy phpservers/server1 0/0/0/2/2 200 66318 - - ---- 50/50/1/1/0 0/0 "GET /pinf.php HTTP/1.1" 127.0.0.1:36486 [03/May/2020:07:37:57.814] myproxy phpservers/server1 0/0/0/2/2 200 66318 - - ---- 50/50/0/0/0 0/0 "GET /pinf.php HTTP/1.1" [WARNING] 123/073758 (7) : Health check for server phpservers/server1 failed, reason: Layer7 timeout, check duration: 2001ms, status: 2/3 UP. Health check for server phpservers/server1 failed, reason: Layer7 timeout, check duration: 2001ms, status: 2/3 UP. [WARNING] 123/073802 (7) : Health check for server phpservers/server1 failed, reason: Layer7 timeout, check duration: 2002ms, status: 1/3 UP. Health check for server phpservers/server1 failed, reason: Layer7 timeout, check duration: 2002ms, status: 1/3 UP. [WARNING] 123/073806 (7) : Health check for server phpservers/server1 failed, reason: Layer7 timeout, check duration: 2001ms, status: 0/2 DOWN. Health check for server phpservers/server1 failed, reason: Layer7 timeout, check duration: 2001ms, status: 0/2 DOWN. [WARNING] 123/073806 (7) : Server phpservers/server1 is DOWN. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue. Server phpservers/server1 is DOWN. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue. [ALERT] 123/073806 (7) : backend 'phpservers' has no server available! backend phpservers has no server available! # now the k6 client wait for answer from server. # PHP-FPM Kills the 11th process and haproxy is now able to send the health check successfully. [WARNING] 123/073846 (7) : Health check for server phpservers/server1 succeeded, reason: Layer7 check passed, code: 200, check duration: 1945ms, status: 1/2 DOWN. Health check for server phpservers/server1 succeeded, reason: Layer7 check passed, code: 200, check duration: 1945ms, status: 1/2 DOWN. [WARNING] 123/073848 (7) : Health check for server phpservers/server1 succeeded, reason: Layer7 check passed, code: 200, check duration: 0ms, status: 3/3 UP. Health check for server phpservers/server1 succeeded, reason: Layer7 check passed, code: 200, check duration: 0ms, status: 3/3 UP. [WARNING] 123/073848 (7) : Server phpservers/server1 is UP. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue. Server phpservers/server1 is UP. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue. ``` Here some infos. ``` HA-Proxy version 2.2-dev6-9bcd97-178 2020/05/02 - https://haproxy.org/ Status: development branch - not safe for use in production. Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open Running on: Linux 5.3.0-45-generic #37-Ubuntu SMP Thu Mar 26 20:41:27 UTC 2020 x86_64 Build options : TARGET = linux-glibc CPU = generic CC = gcc CFLAGS = -O2 -g -Wall -Wextra -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wno-implicit-fallthrough -Wno-stringop-overflow -Wno-cast-function-type -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_GETADDRINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE -PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS Default settings : bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 Built with multi-threading support (MAX_THREADS=64, default=8). Built with OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019 Running on OpenSSL version : OpenSSL 1.1.1d 10 Sep 2019 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3 Built with Lua version : Lua 5.3.3 Built with network namespace support. Built with gcc compiler version 8.3.0 Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Built with PCRE2 version : 10.32 2018-09-10 PCRE2 library supports JIT : yes Encrypted password support via crypt(3): yes Built with zlib version : 1.2.11 Running on zlib version : 1.2.11 Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip") Built with the Prometheus exporter as a service Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. Available multiplexer protocols : (protocols marked as <default> cannot be specified using 'proto' keyword) h2 : mode=HTTP side=FE|BE mux=H2 fcgi : mode=HTTP side=BE mux=FCGI <default> : mode=HTTP side=FE|BE mux=H1 <default> : mode=TCP side=FE|BE mux=PASS Available services : prometheus-exporter Available filters : [SPOE] spoe [CACHE] cache [FCGI] fcgi-app [TRACE] trace [COMP] compression ``` ``` cat /tmp/haproxy-config/haproxy.cfg global log stdout format raw daemon debug defaults log global mode http option httplog option dontlognull option log-health-checks timeout connect 5s timeout client 50s timeout server 50s frontend myproxy bind :8080 alpn h2 default_backend phpservers backend phpservers use-fcgi-app php-fpm option httpchk http-check connect default proto fcgi http-check send meth GET uri /ping # ver HTTP/1.1 http-check expect string pong server server1 127.0.0.1:9000 proto fcgi check maxconn 9 minconn 5 maxqueue 0 fcgi-app php-fpm log-stderr global docroot /var/www/html index index.php path-info ^(/.+\.php)(/.*)?$ ``` PHP-FPM config. ``` $ egrep -v '^(;|$)' /tmp/php-conf/php-fpm.conf [global] pid = /run/php7.4-fpm.pid error_log = /dev/stderr daemonize = no include=/mnt/www.conf $ egrep -v '^(;|$)' /tmp/php-conf/www.conf [www] user = www-data listen = 9000 pm = dynamic pm.max_children = 10 pm.min_spare_servers = 4 pm.start_servers = 5 pm.max_spare_servers = 6 pm.max_requests = 500 pm.status_path = /status ping.path = /ping ping.response = pong access.log= /proc/self/fd/1 php_admin_value[error_log] = /proc/self/fd/1 php_admin_flag[log_errors] = on php_admin_flag[expose_php] = off ``` Thanks for help. Aleks
