Hi! I've mentioned that since moving from 1.9.8 to 2.0-branch of haproxy, ereq counter of frontend tcp-mode sections began to grow. I had zeroes in that counter before haproxy 2.0, now the number of "error requests" is much higher. Example: listen sample.service:1234 bind ipv6@xxx:yyy mode tcp balance leastconn timeout server 1h timeout client 1h option tcp-check default-server weight 1 inter 2s rise 3 server server1 server1:1234 weight 100 check server server2 server2:1234 weight 100 check
"show errors" shows nothing: Total events captured on [24/Jul/2019:09:56:12.544] : 0 And there are no errors in my log file also. But look at the error-counters from the output of 'show stat': $ echo "show stat sample.service:1234 7 -1 typed"| sudo socat unix-connect:/var/run/haproxy.sock stdio | egrep -v :0$ F.194.0.0.pxname.1:KNS:str:sample.service:1234 F.194.0.1.svname.1:KNS:str:FRONTEND F.194.0.5.smax.1:MMP:u32:2 F.194.0.6.slim.1:CLP:u32:4096 F.194.0.7.stot.1:MCP:u64:37 F.194.0.12.ereq.1:MCP:u64:37 F.194.0.17.status.1:SGP:str:OPEN F.194.0.26.pid.1:KGP:u32:1 F.194.0.27.iid.1:KGS:u32:194 F.194.0.35.rate_max.1:MMP:u32:1 F.194.0.75.mode.1:CGS:str:tcp F.194.0.78.conn_rate_max.1:MMP:u32:1 F.194.0.79.conn_tot.1:MCP:u64:37 S.194.1.0.pxname.1:KNS:str:sample.service:1234 S.194.1.1.svname.1:KNS:str:server1:1234 S.194.1.17.status.1:SGP:str:UP S.194.1.18.weight.1:MaP:u32:100 S.194.1.19.act.1:SGP:u32:1 S.194.1.23.lastchg.1:MAP:u32:184 S.194.1.26.pid.1:KGP:u32:1 S.194.1.27.iid.1:KGS:u32:194 S.194.1.28.sid.1:KGS:u32:1 S.194.1.32.type.1:CGS:u32:2 S.194.1.36.check_status.1:MOP:str:L4OK S.194.1.55.lastsess.1:MAP:s32:-1 S.194.1.56.last_chk.1:MOP:str: S.194.1.65.check_desc.1:MOP:str:Layer4 check passed S.194.1.67.check_rise.1:CGS:u32:3 S.194.1.68.check_fall.1:CGS:u32:3 S.194.1.69.check_health.1:CGS:u32:5 S.194.1.73.addr.1:CGS:str:[zzz]:yyy S.194.1.75.mode.1:CGS:str:tcp S.194.2.0.pxname.1:KNS:str:sample.service:1234 S.194.2.1.svname.1:KNS:str:server2:1234 S.194.2.17.status.1:SGP:str:UP S.194.2.18.weight.1:MaP:u32:1 S.194.2.19.act.1:SGP:u32:1 S.194.2.23.lastchg.1:MAP:u32:184 S.194.2.26.pid.1:KGP:u32:1 S.194.2.27.iid.1:KGS:u32:194 S.194.2.28.sid.1:KGS:u32:2 S.194.2.32.type.1:CGS:u32:2 S.194.2.36.check_status.1:MOP:str:L4OK S.194.2.38.check_duration.1:MDP:u64:3 S.194.2.55.lastsess.1:MAP:s32:-1 S.194.2.56.last_chk.1:MOP:str: S.194.2.65.check_desc.1:MOP:str:Layer4 check passed S.194.2.67.check_rise.1:CGS:u32:3 S.194.2.68.check_fall.1:CGS:u32:3 S.194.2.69.check_health.1:CGS:u32:5 S.194.2.73.addr.1:CGS:str:[xxx]:yyy S.194.2.75.mode.1:CGS:str:tcp B.194.0.0.pxname.1:KNS:str:sample.service:1234 B.194.0.1.svname.1:KNS:str:BACKEND B.194.0.5.smax.1:MMP:u32:1 B.194.0.6.slim.1:CLP:u32:410 B.194.0.7.stot.1:MCP:u64:37 B.194.0.17.status.1:SGP:str:UP B.194.0.18.weight.1:MaP:u32:101 B.194.0.19.act.1:MGP:u32:2 B.194.0.23.lastchg.1:MAP:u32:184 B.194.0.26.pid.1:KGP:u32:1 B.194.0.27.iid.1:KGS:u32:194 B.194.0.32.type.1:CGS:u32:1 B.194.0.35.rate_max.1:MGP:u32:1 B.194.0.49.cli_abrt.1:MCP:u64:37 B.194.0.55.lastsess.1:MAP:s32:-1 B.194.0.75.mode.1:CGS:str:tcp B.194.0.76.algo.1:CGS:str:leastconn How can I find out the real reason for these errors? Could it happen if a client uses RST instead of a regular FIN/ACK sequence to close the session? Example of such behavior (tcp-connect probe from keepalived to haproxy): 10:42:48.086780 IP6 client.43930 > haproxy.12346: Flags [S], seq 4136298350, win 28800, options [mss 1440,nop,nop,sackOK,nop,wscale 7], length 0 10:42:48.086837 IP6 haproxy.12346 > client.43930: Flags [S.], seq 2234519198, ack 4136298351, win 26520, options [mss 8840,nop,nop,sackOK,nop,wscale 11], length 0 10:42:48.087177 IP6 client.43930 > haproxy.12346: Flags [.], ack 1, win 225, length 0 10:42:48.087181 IP6 client.43930 > haproxy.12346: Flags [R.], seq 1, ack 1, win 225, length 0 $ haproxy -vvv HA-Proxy version 2.0.3-1 2019/07/24 - https://haproxy.org/ Build options : TARGET = linux-glibc CPU = generic CC = gcc CFLAGS = -O2 -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_REGPARM=1 USE_GETADDRINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_TFO=1 USE_SYSTEMD=1 Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE -PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4 -MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS Default settings : bufsize = 16384, maxrewrite = 1024, maxpollevents = 200 Built with multi-threading support (MAX_THREADS=64, default=32). Built with OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016 Running on OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 Built with Lua version : Lua 5.3.1 Built with network namespace support. Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Built with zlib version : 1.2.8 Running on zlib version : 1.2.8 Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip") Built with PCRE2 version : 10.21 2016-01-12 PCRE2 library supports JIT : yes Encrypted password support via crypt(3): yes Built with the Prometheus exporter as a service Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. Available multiplexer protocols : (protocols marked as <default> cannot be specified using 'proto' keyword) h2 : mode=HTX side=FE|BE mux=H2 h2 : mode=HTTP side=FE mux=H2 <default> : mode=HTX side=FE|BE mux=H1 <default> : mode=TCP|HTTP side=FE|BE mux=PASS Available services : prometheus-exporter Available filters : [SPOE] spoe [COMP] compression [CACHE] cache [TRACE] trace -- Best regards, Maksim Kupriianov
