[PATCH] DOC: config: added optional rst-ttl argument to silent-drop in action
This patch adds the optional silent-drop rst-ttl argument to the action lists in 5 places in the configuration manual. It should be backported to 2.7. I'll send you a patch for 2.7 when this one is accepted to save you from legwork, if requested. Best regards Mathias Weiersmueller cyberheads GmbH 0001-added-optional-rst-ttl-argument-to-silent-drop-in-ac.patch Description: 0001-added-optional-rst-ttl-argument-to-silent-drop-in-ac.patch
[PATCH] MEDIUM: frontend: add parameter rst-ttl to silent-drop
Hi, I've accepted Willy's challenge from HAProxy.conf 2022, here is my shot: The silent-drop action was extended with an additional optional parameter, [rst-ttl ], causing HAProxy to send a TCP RST with the specified TTL towards the client. With this behaviour, the connection state on your own client-facing middle-boxes (load balancers, firewalls) will be purged, but the client will still assume the TCP connection is up because the TCP RST packet expires before reaching the client. I think of it as another DDoS defense mechanism - this will save resources on the middle-boxes (connection table exhaustion). Subsequent packets from a silently-dropped connection will be already dropped by the upstream equipment, saving resources on HAProxy and the network between middle-box(es) and HAProxy. Comments welcome! Best regards Mathias 0001-MEDIUM-frontend-add-parameter-rst-ttl-to-silent-drop.patch Description: 0001-MEDIUM-frontend-add-parameter-rst-ttl-to-silent-drop.patch
[PATCH] DOC: fix TOC in starter guide for subsection 3.3.8. and 3.4.9
Dear all, The subsection 3.4.9 (Standard features : Statistics) in the starter guide from 2.4 up to latest points to a non-existing anchor. It looks like this subsection was moved from 3.4.9 to 3.3.8 (Basic features : Statistics), but the TOC was not updated accordingly. Please check the attached patch to fix this. Best regards Mathias 0001-DOC-fix-TOC-in-starter-guide-for-subsection-3.3.8.-S.patch Description: 0001-DOC-fix-TOC-in-starter-guide-for-subsection-3.3.8.-S.patch
Re: payload inspection using req.payload
Hi Micha,
> My problem is that the "req.payload(0,10)" fetch, which I am using for
> that purpose, does not seem to reliably have access to the payload at
> all times.
The problem is not the fetch per se, it is the timing of the evaluation
of the rule: tcp-request content rules are evaluated very early - there's
a high probability the payload buffer is empty at this moment.
if you add a condition to check if there is already any content present,
it will always match (checked using your config, thanks!):
example:
tcp-request content set-var(txn.rawPayload) req.payload(0,2),hex if { req_len
gt 0 }
As a side note: In case you want to match the payload in a binary (non-HTTP)
protocol,
make sure you convert the payload to hex first, see section 7.1.3 in the
newest configuration docs, here's the excerpt:
Do not use string matches for binary fetches which might contain null bytes
(0x00), as the comparison stops at the occurrence of the first null byte.
Instead, convert the binary fetch to a hex string with the hex converter first.
Example:
# matches if the string is present in the binary sample
acl tag_found req.payload(0,0),hex -m sub 3C7461673E
Best regards
Mathias
Re: [PATCH] DOC: clarify matching strings on binary fetches
Willy, Will I (pardon the pun) send backported patches for the documentation for versions 1.5 - 2.1? Or is that handled by you? Regards Mathias From: Willy Tarreau Sent: Tuesday, December 3, 2019 07:35 To: Mathias Weiersmüller (cyberheads GmbH) Cc: haproxy@formilux.org ; tfourn...@arpalert.org ; Aleksandar Lazic Subject: Re: [PATCH] DOC: clarify matching strings on binary fetches Hi Mathias, On Mon, Dec 02, 2019 at 09:01:56AM +, Mathias Weiersmüller (cyberheads GmbH) wrote: > Documentation patch provided - I would backport it all the way down to 1.5 as > it might save from headaches. Good point indeed, now merged, thank you! Willy
[PATCH] DOC: clarify matching strings on binary fetches
Documentation patch provided - I would backport it all the way down to 1.5 as it might save from headaches. Best regards Mathias > > It would be nice when you send us a patch to fix the doc. > > Regards > Aleks > > Nov 30, 2019 11:35:24 AM Mathias Weiersmüller (cyberheads GmbH) > : > > > (CCing Thierry Fournier as maintainer of the pattern matching part) > > > > > > > We use HAProxy in TCP Mode for non-HTTP protocols. > > > > > > The request of one particular protocol looks like this: > > > > > - length of message (binary value, 4 bytes long) > > > > > > - binary part (40-200 bytes) > > > - XML part > > > > > > Goal: We want to use a particular backend when the XML part of the > > > request contains the string "". > > > > > > We used this ACL: > > > acl tag_found req.payload(0,0) -m sub > > > > > > The problem: > > > The substring matching stops on a Null byte (\0) in a binary fetch. > > > We always have this case (the request normally starts with Null > > > bytes). Therefore, the match never succeeds. As there might be null > > > bytes in the binary part too, we cannot just start the payload fetch > > > after byte 4. > > > > > > == > > > frontend fe_test > > > bind *:3000 > > > > > > tcp-request inspect-delay 5s > > > > > > acl content_present req_len gt 0 > > > acl tag_found req.payload(0,0) -m sub > > > > > > tcp-request content accept if content_present tcp-request content > > > reject > > > > > > # depending on if the payload contains the string "", we use > > > different backends # right now, the two backends are exactly the same. > > > use_backend be_tag if tag_found > > > default_backend be_default > > > > > > backend be_tag > > > server srv_1:4000 > > > > > > backend be_default > > > server srv_1:4000 > > > > > > Test cases: > > > (tested on versions 2.0.10, 1.5.18) > > > echo -e '' | nc 127.0.0.1 3000 # will use backend be_tag echo > > > -e '\0' | nc 127.0.0.1 3000 # will use backend be_default, but > > > should use be_tag == > > > > > > Workaround: > > > =>convert payload into hexified string, parse against hex: > > > acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is > > > in hexadecimal > > > > > > Dear list members, these are the questions I am twisting my mind with. Do > > > you have a good take one these? > > > > > > - Is there another (better) way to do a substring match on a payload > > > which contains Null bytes? > > > - Would another, new match method make sense here (something like > > > sub_bin ? ) > > > - Do we run into a problem with the hex conversion because the size > > > of the sample has double the size than the original (maybe bigger > > > than bufsize?) > > > > > > > > > > If this behavior is intended, then the configuration manual (7.1.3 Matching > > strings) should be updated to reflect this: > > > > Do not use string matches for binary fetches which might contain null > > bytes (0x00), as the comparison stops at the occurrence of the first > > null byte. Instead, convert the binary fetch to a hex string with the hex > > converter first. > > > > Example: > > acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is > > in hexadecimal 0001-DOC-clarify-matching-strings-on-binary-fetches.patch Description: 0001-DOC-clarify-matching-strings-on-binary-fetches.patch
RE: substring matching stops on a Null byte (\0) in a binary fetch
(CCing Thierry Fournier as maintainer of the pattern matching part) > We use HAProxy in TCP Mode for non-HTTP protocols. > > The request of one particular protocol looks like this: > - length of message (binary value, 4 bytes long) > - binary part (40-200 bytes) > - XML part > > Goal: We want to use a particular backend when the XML part of the request > contains the string "". > > We used this ACL: > acl tag_found req.payload(0,0) -m sub > > The problem: > The substring matching stops on a Null byte (\0) in a binary fetch. We always > have this case (the request normally starts with Null > bytes). Therefore, the match never succeeds. As there might be null bytes in > the binary part too, we cannot just start the payload > fetch > after byte 4. > > == > frontend fe_test > bind *:3000 > > tcp-request inspect-delay 5s > > acl content_present req_len gt 0 > acl tag_found req.payload(0,0) -m sub > > tcp-request content accept if content_present > tcp-request content reject > > # depending on if the payload contains the string "", we use different > backends > # right now, the two backends are exactly the same. > use_backend be_tag if tag_found > default_backend be_default > > backend be_tag > server srv_1:4000 > > backend be_default > server srv_1:4000 > > Test cases: > (tested on versions 2.0.10, 1.5.18) > echo -e '' | nc 127.0.0.1 3000 # will use backend be_tag > echo -e '\0' | nc 127.0.0.1 3000# will use backend be_default, but > should use be_tag > == > > Workaround: > =>convert payload into hexified string, parse against hex: >acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in > hexadecimal > > Dear list members, these are the questions I am twisting my mind with. Do you > have a good take one these? > > - Is there another (better) way to do a substring match on a payload which > contains Null bytes? > - Would another, new match method make sense here (something like sub_bin ? ) > - Do we run into a problem with the hex conversion because the size of the > sample has double the size than the original (maybe > bigger than bufsize?) > If this behavior is intended, then the configuration manual (7.1.3 Matching strings) should be updated to reflect this: Do not use string matches for binary fetches which might contain null bytes (0x00), as the comparison stops at the occurrence of the first null byte. Instead, convert the binary fetch to a hex string with the hex converter first. Example: acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in hexadecimal Does that make sense? Best regards Mathias
substring matching stops on a Null byte (\0) in a binary fetch
Dear list! We use HAProxy in TCP Mode for non-HTTP protocols. The request of one particular protocol looks like this: - length of message (binary value, 4 bytes long) - binary part (40-200 bytes) - XML part Goal: We want to use a particular backend when the XML part of the request contains the string "". We used this ACL: acl tag_found req.payload(0,0) -m sub The problem: The substring matching stops on a Null byte (\0) in a binary fetch. We always have this case (the request normally starts with Null bytes). Therefore, the match never succeeds. As there might be null bytes in the binary part too, we cannot just start the payload fetch after byte 4. == frontend fe_test bind *:3000 tcp-request inspect-delay 5s acl content_present req_len gt 0 acl tag_found req.payload(0,0) -m sub tcp-request content accept if content_present tcp-request content reject # depending on if the payload contains the string "", we use different backends # right now, the two backends are exactly the same. use_backend be_tag if tag_found default_backend be_default backend be_tag server srv_1:4000 backend be_default server srv_1:4000 Test cases: (tested on versions 2.0.10, 1.5.18) echo -e '' | nc 127.0.0.1 3000 # will use backend be_tag echo -e '\0' | nc 127.0.0.1 3000# will use backend be_default, but should use be_tag == Workaround: =>convert payload into hexified string, parse against hex: acl tag_found req.payload(0,0),hex -m sub 3C7461673E # this is in hexadecimal Dear list members, these are the questions I am twisting my mind with. Do you have a good take one these? - Is there another (better) way to do a substring match on a payload which contains Null bytes? - Would another, new match method make sense here (something like sub_bin ? ) - Do we run into a problem with the hex conversion because the size of the sample has double the size than the original (maybe bigger than bufsize?) Best regards Mathias
