Re: haproxy 1.9 status_code=-1
Hi.
Please keep the ML in the loop, thanks.
Thu Aug 29 22:42:18 GMT+02:00 2019 Karthik P :
> Thanks for your response. Here are the details
> Log:
> ST=-1 http_request="POST /api/v2 HTTP/1.1" sslc=ECDHE-RSA-AES256-GCM-SHA384
> sslv=TLSv1.2 Th=107 ci=10.X.X.X B=6830 si=10.Y.Y.Y b=ch s=service_onprem rc=0
> U=1603 Tr=-1 Tc=-1 Tt=192 ts=--
Okay so you have your own log format
Is ts "termination_state"?
I miss the be/se part, can you try to use the default format for the debug
session, as you can see there are some more Info's the in your format
Quote from doc.
```
>>> haproxy[18113]: 127.0.0.1:34548 [15/Oct/2003:15:18:55.798] px-http \
px-http/ -1/-1/-1/-1/8490 -1 0 - - CR-- 2/2/2/0/0 0/0 ""
=> the client never completed its request and aborted itself ("C---") after
8.5s, while the proxy was waiting for the request headers ("-R--").
Nothing was sent to any server.
```
> haproxy -vv:
Looks like you have built your own haproxy , right?
> HA-Proxy version 1.9.5 2019/03/19 - https://haproxy.org/
> [https://haproxy.org/]
> Build options :
> TARGET = linux2628
> CPU = generic
> CC = gcc
> CFLAGS = -O2 -g -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
> -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
> -Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
> -Wno-missing-field-initializers -Wtype-limits
> OPTIONS = USE_OPENSSL=1 USE_LUA=1
> Default settings :
> maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
> Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
> Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
> OpenSSL library supports TLS extensions : yes
> OpenSSL library supports SNI : yes
> OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
> Built with Lua version : Lua 5.3.3
> Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
> IP_FREEBIND
> Built without compression support (neither USE_ZLIB nor USE_SLZ are set).
> Compression algorithms supported : identity("identity")
> Built without PCRE or PCRE2 support (using libc's regex instead)
> Encrypted password support via crypt(3): yes
> Built with multi-threading support.
> Available polling systems :
> epoll : pref=300, test result OK
> poll : pref=200, test result OK
> select : pref=150, test result OK
> Total: 3 (3 usable), will use epoll.
> Available multiplexer protocols :
> (protocols marked as cannot be specified using 'proto' keyword)
> h2 : mode=HTX side=FE|BE
> h2 : mode=HTTP side=FE
> : mode=HTX side=FE|BE
> : mode=TCP|HTTP side=FE|BE
> Available filters :
> [SPOE] spoe
> [COMP] compression
> [CACHE] cache
> [TRACE] trace
>
>
> Let me know if the above helps.
Not too much.
> Let me know if you need any other information.
The anonymized config would help a lot.
> I can try 2.x. But would like to know if there is any known issue with 1.9
I would strongly recommend to go directly to 2.0, jm2c.
> Thanks, Karthik
Regards
Aleks
> On Thu, Aug 29, 2019 at 11:52 AM Aleksandar Lazic < al-hapr...@none.at [] >
> wrote:
>
>>
>> Hi.
>>
>> Thu Aug 29 20:33:46 GMT+02:00 2019 Karthik P :
>>
>> > Hello, Can someone please help? We tried to upgrade haproxy from 1.8 to 1.9
>> > By doing this we started seeing status_code (%ST) = -1
>> > Can someone please provide some direction on how to approach debugging
>> > this issue?
>> > Did anything change in 1.9 which is causing this?
>>
>> Ähm there are a lot of changes between 1.8 and 1.9. You should share some
>> more informations about the setup and the line with ST -1.
>>
>> haproxy -vv
>> Your config
>> The whole logline with -1 entry
>>
>> Quote from doc
>>
>> ```
>> - "status_code" is the HTTP status code returned to the client. This status
>> is generally set by the server, but it might also be set by haproxy when
>> the server cannot be reached or when its response is blocked by haproxy.
>> ```
>>
>> As you see in the code could be there several reasons for the -1 and this
>> shows not the possibilities from origin server.
>>
>> http://git.haproxy.org/?p=haproxy-1.9.git=search=HEAD=grep=status_code
>>
>> [http://git.haproxy.org/?p=haproxy-1.9.git=search=HEAD=grep=status_code]
>>
>>
>> Why not using 2.0.5?
>>
>> > Thanks, Karthik
>>
>> Regards
>> Aleks
>>
>>
>
Re: haproxy 1.9 status_code=-1
Hi. Thu Aug 29 20:33:46 GMT+02:00 2019 Karthik P : > Hello, Can someone please help? We tried to upgrade haproxy from 1.8 to 1.9 > By doing this we started seeing status_code (%ST) = -1 > Can someone please provide some direction on how to approach debugging this > issue? > Did anything change in 1.9 which is causing this? Ähm there are a lot of changes between 1.8 and 1.9. You should share some more informations about the setup and the line with ST -1. haproxy -vv Your config The whole logline with -1 entry Quote from doc ``` - "status_code" is the HTTP status code returned to the client. This status is generally set by the server, but it might also be set by haproxy when the server cannot be reached or when its response is blocked by haproxy. ``` As you see in the code could be there several reasons for the -1 and this shows not the possibilities from origin server. http://git.haproxy.org/?p=haproxy-1.9.git=search=HEAD=grep=status_code Why not using 2.0.5? > Thanks, Karthik Regards Aleks
haproxy 1.9 status_code=-1
Hello, Can someone please help? We tried to upgrade haproxy from 1.8 to 1.9 By doing this we started seeing status_code (%ST) = -1 Can someone please provide some direction on how to approach debugging this issue? Did anything change in 1.9 which is causing this? Thanks, Karthik
[PATCH] MINOR: send-proxy-v2: sends authority TLV according to TLV received
Hi, This patch follows Geoff's patch. ++ Manu 0001-MINOR-send-proxy-v2-sends-authority-TLV-according-to.patch Description: Binary data
haproxy=2.0.5: A bogus APPCTX is spinning and refuses to die
Hi!
Sometimes on reload of 2.0.5 I got this in logs:
A bogus APPCTX [0x7fc1a06ff0e0] is spinning at 122591 calls per second and
refuses to die, aborting now! Please report this error to developers
[strm=0x557eb7f4e630 src=xxx fe=yyy be=yyy dst= rqf=c48202 rqa=0
rpf=80048202 rpa=0 sif=EST,200040 sib=EST,244058 af=(nil),0
csf=0x7fc1ac085680,200 ab=0x7fc1a06ff0e0,7 csb=(nil),0
cof=0x557eb7fe9f50,201366:PASS(0x7fc16c0ad080)/RAW((nil))/tcpv6(1275)
cob=(nil),0:NONE((nil))/NONE((nil))/NONE(0) ]
...and this in coredump:
Program terminated with signal SIGABRT, Aborted.
#0 0x7fc1e7c0b428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
[Current thread is 1 (Thread 0x7fc1e95021c0 (LWP 3539))]
(gdb) bt
#0 0x7fc1e7c0b428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x7fc1e7c0d02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x557ea75b8b5e in stream_dump_and_crash (obj=obj@entry=0x7fc1a06ff0e0,
rate=122591) at src/stream.c:2983
#3 0x557ea7696d2d in task_run_applet (t=0x7fc1746a5790,
context=0x7fc1a06ff0e0, state=) at src/applet.c:80
#4 0x557ea7692fd5 in process_runnable_tasks () at src/task.c:414
#5 0x557ea75fbc18 in run_poll_loop () at src/haproxy.c:2517
#6 run_thread_poll_loop (data=) at src/haproxy.c:2638
#7 0x557ea7558cb7 in main (argc=, argv=0x7ffd7db57428)
at src/haproxy.c:3315
Version: haproxy -vv
HA-Proxy version 2.0.5-1 2019/08/27 - https://haproxy.org/
Build options :
TARGET = linux-glibc
CPU = generic
CC = gcc
CFLAGS = -O2 -g -O2 -fPIE -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2
-fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
-Wno-unused-label -Wno-sign-compare -Wno-unused-parameter
-Wno-old-style-declaration -Wno-ignored-qualifiers -Wno-clobbered
-Wno-missing-field-initializers -Wtype-limits
OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_REGPARM=1 USE_GETADDRINFO=1
USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_TFO=1 USE_SYSTEMD=1
Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER -PCRE
-PCRE_JIT +PCRE2 +PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED
+REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE
+LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL +LUA +FUTEX +ACCEPT4
-MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS
-51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=56).
Built with OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016
Running on OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.1
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with zlib version : 1.2.8
Running on zlib version : 1.2.8
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE2 version : 10.21 2016-01-12
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with the Prometheus exporter as a service
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as cannot be specified using 'proto' keyword)
h2 : mode=HTXside=FE|BE mux=H2
h2 : mode=HTTP side=FEmux=H2
: mode=HTXside=FE|BE mux=H1
: mode=TCP|HTTP side=FE|BE mux=PASS
Available services :
prometheus-exporter
Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace
--
Best regards,
Maksim Kupriianov
maxconn threaded vs multiproc
Hi How does maxconn work when you switch from multi process (nbproc) to to multithreaded (nbthread). So I had in multiproc global maxconn 224288 default maxconn 131072 So now I've switched to multithreaded mode. And I used to believe it was maxconn per process. So does it mean I should actually take that number times the number of processes I used to run? I tried doubling the numbers to global maxconn 524288 default maxconn 262144 And I ran out of file descriptors Aug 29 11:20:13 aalb02 haproxy[45828]: Proxy default reached process FD limit (maxsock=1048766). Please check 'ulimit-n' and restart. I thought that the file descriptors was handled automatically but haproxy or the systemd unit. And my fs.file-max = 3274636. So this leads me to believe that the maxconn is per process and per thread? So switching form multi proc to threaded I should leave the maxconn numbers as they where? Can anyone shed some light on this? Thanks Elias
