stable-bot: NOTICE: 2 bug fixes in queue for next release - 1.9
Hi, This is a friendly bot that watches fixes pending for the next haproxy-stable release! One such e-mail is sent periodically once patches are waiting in the last maintenance branch, and an ideal release date is computed based on the severity of these fixes and their merge date. Responses to this mail must be sent to the mailing list. Last release 1.9.13 was issued on 2019/11/25. There are currently 2 patches in the queue cut down this way: - 2 MINOR, first one merged on 2019/11/25 Thus the computed ideal release date for 1.9.14 would be 2019/12/23, which is in three weeks or less. The current list of patches in the queue is: - MINOR : stream: init variables when the list is empty - MINOR : http-htx: Don't make http_find_header() fail if the value is empty --- The haproxy stable-bot is freely provided by HAProxy Technologies to help improve the quality of each HAProxy release. If you have any issue with these emails or if you want to suggest some improvements, please post them on the list so that the solutions suiting the most users can be found.
stable-bot: NOTICE: 4 bug fixes in queue for next release - 2.0
Hi, This is a friendly bot that watches fixes pending for the next haproxy-stable release! One such e-mail is sent periodically once patches are waiting in the last maintenance branch, and an ideal release date is computed based on the severity of these fixes and their merge date. Responses to this mail must be sent to the mailing list. Last release 2.0.10 was issued on 2019/11/25. There are currently 4 patches in the queue cut down this way: - 4 MINOR, first one merged on 2019/11/25 Thus the computed ideal release date for 2.0.11 would be 2019/12/23, which is in three weeks or less. The current list of patches in the queue is: - MINOR : http-htx: Don't make http_find_header() fail if the value is empty - MINOR : contrib/prometheus-exporter: Use HTX errors and not legacy ones - MINOR : contrib/prometheus-exporter: decode parameter and value only - MINOR : stream: init variables when the list is empty --- The haproxy stable-bot is freely provided by HAProxy Technologies to help improve the quality of each HAProxy release. If you have any issue with these emails or if you want to suggest some improvements, please post them on the list so that the solutions suiting the most users can be found.
OpenBSD/2.1.0 - weird syslogging?
Hello,
just build 2.1.0 on OpenBSD.. while the actual processing (fe->be)
works,
I am getting quite some weird logging (in /var/log/messages) per
http-request:
Nov 30 23:01:05 tyo haproxy: vfprintf %s NULL in ">%s %2d
%02d:%02d:%02d %.*s%*s"
syslog.conf pretty straight forward:
*.notice;auth,authpriv,cron,ftp,kern,lpr,mail,user.none
/var/log/messages #default
local0.*/var/log/haproxy
with syslog config in haproxy.cfg
log 127.0.0.1 local0 debug
I am confused.. this is a plain test/sandbox (working on "new"
monitoring), so can do whatever needed on tracing and such.
PS: logging is the same for HTTP or HTTPS (just in case libressl..)
tyo# haproxy -vv
HA-Proxy version 2.1.0 2019/11/25 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2021.
Known bugs: http://www.haproxy.org/bugs/bugs-2.1.0.html
Build options :
TARGET = openbsd
CPU = generic
CC = cc
CFLAGS = -O2 -pipe -fno-strict-aliasing
OPTIONS = USE_PCRE=1 USE_OPENSSL=1
Feature list : -EPOLL +KQUEUE -MY_EPOLL -MY_SPLICE -NETFILTER +PCRE
-PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD
-PTHREAD_PSHARED -REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY
-LINUX_TPROXY -LINUX_SPLICE -LIBCRYPT -CRYPT_H -VSYSCALL -GETADDRINFO
+OPENSSL -LUA -FUTEX +ACCEPT4 -MY_ACCEPT4 -ZLIB -SLZ -CPU_AFFINITY -TFO
-NS -DL -RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER
-PRCTL -THREAD_DUMP -EVPORTS
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=1).
Built with OpenSSL version : LibreSSL 2.8.2
Running on OpenSSL version : LibreSSL 2.8.2
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2
Built with transparent proxy support using: SO_BINDANY
Built with PCRE version : 8.41 2017-07-05
Running on PCRE version : 8.41 2017-07-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built without compression support (neither USE_ZLIB nor USE_SLZ are
set).
Compression algorithms supported : identity("identity")
Available polling systems :
kqueue : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use kqueue.
Available multiplexer protocols :
(protocols marked as cannot be specified using 'proto'
keyword)
h2 : mode=HTTP side=FE|BE mux=H2
fcgi : mode=HTTP side=BEmux=FCGI
: mode=HTTP side=FE|BE mux=H1
: mode=TCPside=FE|BE mux=PASS
Available services : none
Available filters :
[SPOE] spoe
[CACHE] cache
[FCGI] fcgi-app
[TRACE] trace
[COMP] compression
TIA,
--
pb
Re: substring matching stops on a Null byte (\0) in a binary fetch
Hi. Looks like it works as designed because currently are used the "str*cmp" functions for matching. Your solution with hex convert looks like how the '\0' byte issue could be fixed. http://git.haproxy.org/?p=haproxy-2.0.git;a=blob;f=src/pattern.c;hb=6d9a455da17251c34d3c552f2a963447f52fdd80#l724 HAProxy can trie to use memcmp but the "sub" match will then work different. https://stackoverflow.com/questions/13095513/what-is-the-difference-between-memcmp-strcmp-and-strncmp-in-c/13095574#13095574 >From binary point of view is '\0' not the same as ''? Maybe the mentioned https://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string-search_algorithm in the code will fix this behavior but as far as I known no one works on such a patch. It would be nice when you send us a patch to fix the doc. Regards Aleks Nov 30, 2019 11:35:24 AM Mathias Weiersmüller (cyberheads GmbH) : > (CCing Thierry Fournier as maintainer of the pattern matching part) > > > > We use HAProxy in TCP Mode for non-HTTP protocols. > > > > The request of one particular protocol looks like this: > > > > - length of message (binary value, 4 bytes long) > > > - binary part (40-200 bytes) > > - XML part > > > > Goal: We want to use a particular backend when the XML part of the request > > contains the string "". > > > > We used this ACL: > > acl tag_found req.payload(0,0) -m sub > > > > The problem: > > The substring matching stops on a Null byte (\0) in a binary fetch. We > > always have this case (the request normally starts with Null > > bytes). Therefore, the match never succeeds. As there might be null bytes > > in the binary part too, we cannot just start the payload > > fetch > > after byte 4. > > > > == > > frontend fe_test > > bind *:3000 > > > > tcp-request inspect-delay 5s > > > > acl content_present req_len gt 0 > > acl tag_found req.payload(0,0) -m sub > > > > tcp-request content accept if content_present > > tcp-request content reject > > > > # depending on if the payload contains the string "", we use different > > backends > > # right now, the two backends are exactly the same. > > use_backend be_tag if tag_found > > default_backend be_default > > > > backend be_tag > > server srv_1:4000 > > > > backend be_default > > server srv_1:4000 > > > > Test cases: > > (tested on versions 2.0.10, 1.5.18) > > echo -e '' | nc 127.0.0.1 3000 # will use backend be_tag > > echo -e '\0' | nc 127.0.0.1 3000 # will use backend be_default, but > > should use be_tag > > == > > > > Workaround: > > =>convert payload into hexified string, parse against hex: > > acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in > > hexadecimal > > > > Dear list members, these are the questions I am twisting my mind with. Do > > you have a good take one these? > > > > - Is there another (better) way to do a substring match on a payload which > > contains Null bytes? > > - Would another, new match method make sense here (something like sub_bin ? > > ) > > - Do we run into a problem with the hex conversion because the size of the > > sample has double the size than the original (maybe > > bigger than bufsize?) > > > > > > If this behavior is intended, then the configuration manual (7.1.3 Matching > strings) should be updated to reflect this: > > Do not use string matches for binary fetches which might contain null bytes > (0x00), > as the comparison stops at the occurrence of the first null byte. Instead, > convert > the binary fetch to a hex string with the hex converter first. > > Example: > acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in > hexadecimal > > Does that make sense? > > Best regards > > Mathias >
RE: substring matching stops on a Null byte (\0) in a binary fetch
(CCing Thierry Fournier as maintainer of the pattern matching part) > We use HAProxy in TCP Mode for non-HTTP protocols. > > The request of one particular protocol looks like this: > - length of message (binary value, 4 bytes long) > - binary part (40-200 bytes) > - XML part > > Goal: We want to use a particular backend when the XML part of the request > contains the string "". > > We used this ACL: > acl tag_found req.payload(0,0) -m sub > > The problem: > The substring matching stops on a Null byte (\0) in a binary fetch. We always > have this case (the request normally starts with Null > bytes). Therefore, the match never succeeds. As there might be null bytes in > the binary part too, we cannot just start the payload > fetch > after byte 4. > > == > frontend fe_test > bind *:3000 > > tcp-request inspect-delay 5s > > acl content_present req_len gt 0 > acl tag_found req.payload(0,0) -m sub > > tcp-request content accept if content_present > tcp-request content reject > > # depending on if the payload contains the string "", we use different > backends > # right now, the two backends are exactly the same. > use_backend be_tag if tag_found > default_backend be_default > > backend be_tag > server srv_1:4000 > > backend be_default > server srv_1:4000 > > Test cases: > (tested on versions 2.0.10, 1.5.18) > echo -e '' | nc 127.0.0.1 3000 # will use backend be_tag > echo -e '\0' | nc 127.0.0.1 3000# will use backend be_default, but > should use be_tag > == > > Workaround: > =>convert payload into hexified string, parse against hex: >acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in > hexadecimal > > Dear list members, these are the questions I am twisting my mind with. Do you > have a good take one these? > > - Is there another (better) way to do a substring match on a payload which > contains Null bytes? > - Would another, new match method make sense here (something like sub_bin ? ) > - Do we run into a problem with the hex conversion because the size of the > sample has double the size than the original (maybe > bigger than bufsize?) > If this behavior is intended, then the configuration manual (7.1.3 Matching strings) should be updated to reflect this: Do not use string matches for binary fetches which might contain null bytes (0x00), as the comparison stops at the occurrence of the first null byte. Instead, convert the binary fetch to a hex string with the hex converter first. Example: acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in hexadecimal Does that make sense? Best regards Mathias
