[PATCH] DOC/MINOR: Suggestions for percent encoding in param()

2022-06-03 Thread astrothayne
From: Thayne McCombs 

Add some documentation on how to handle percent encoded characters in
input to the param() converter.
---
 doc/configuration.txt | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index d9f47c2eb..9505fc71d 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -17423,12 +17423,19 @@ param(,[])
   an alternative to `urlp()` which only uses "&" as a delimiter, whereas 
urlp
   also uses "?" and ";".

+  Note that this converter doesn't do anything special with url encoded 
characters. If
+  you want to decode the value, you can use the url_dec converter on the 
output. If
+  the name of the paramater in the input might contain encoded characters, 
you'll probably
+  want do normalize the input before calling param. This can be done using
+  "http-request normalize-uri", in particular the percent-decode-unreserved and
+  percent-to-uppercase options.
+
   Example :
   str(a=b=d=r),param(a)   # b
   str(a=c),param(a) # ""
   str(a==a),param(b)  # ""
   str(a=1;b=2;c=4),param(b,;) # 2
-  query,param(redirect_uri)
+  query,param(redirect_uri),urldec()

 protobuf(,[])
   This extracts the protocol buffers message field in raw mode of an input 
binary
--
2.36.1




Re: [PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread Thayne McCombs



Just wondering (maybe something to add to the doc or test): Should 
this handle URL encoded parameter names or parameter values? It 
probably should not, because that's makes the converter less general.


But it would certainly be useful to explain how to properly retrieve 
those values. Simply url-decoding the full query string before doesn't 
do the job, because then the additional delimiters might be 
introduced. This likely needs to be combined with the URI 
normalization feature, as the encoding of a parameter name is not a 
1:1 relationship.


Hmm, Initially I was thinking that it would be sufficient to use 
`urldec` on the result to handle url encoding, but I didn't think about 
the name itself being encoded. I'll add something to the docs 
recommending using uri-normalization for now. Although, I suppose one 
downside to that is it doesn't help if the input doesn't come from the 
uri (for example if it is in the body or a header).






Re: [ANNOUNCE] haproxy-2.6.0

2022-06-03 Thread Willy Tarreau
On Fri, Jun 03, 2022 at 11:43:32PM +0200, Vincent Bernat wrote:
>  ? 31 May 2022 17:56 +02, Willy Tarreau:
> 
> > HAProxy 2.6.0 was released on 2022/05/31. It added 57 new commits
> > after version 2.6-dev12, essentially small bug fixes, QUIC counters
> > and doc updates.
> 
> It's available on haproxy.debian.net. No QUIC support as neither Debian
> nor Ubuntu has the appropriate library.

Many thanks for this, Vincent!

Willy



SV: SV: Traffic from HAproxy get error 401 and 500

2022-06-03 Thread Henning Svane
Hi Baptiste

Fantastic it works.:-)
All the strange Exchange errors solved with 3 lines deleted:-)

Thanks
Regards
Henning

Fra: Baptiste 
Sendt: 3. juni 2022 08:43
Til: Henning Svane 
Cc: Christopher Faulet ; haproxy@formilux.org
Emne: Re: SV: Traffic from HAproxy get error 401 and 500

Hi Henning,

Please remove this "option http-server-close" from your configuration, entirely 
:)

Baptiste


Re: [ANNOUNCE] haproxy-2.6.0

2022-06-03 Thread Vincent Bernat
 ❦ 31 May 2022 17:56 +02, Willy Tarreau:

> HAProxy 2.6.0 was released on 2022/05/31. It added 57 new commits
> after version 2.6-dev12, essentially small bug fixes, QUIC counters
> and doc updates.

It's available on haproxy.debian.net. No QUIC support as neither Debian
nor Ubuntu has the appropriate library.
-- 
The better part of valor is discretion.
-- William Shakespeare, "Henry IV"



Re: deviceatlas compiler error

2022-06-03 Thread David CARLIER
My pleasure Amol,

So for the remaining "failures", if this is those you mention

Performing Test HAS_STD_ATOMICS
-- Performing Test HAS_STD_ATOMICS - Failed
-- Performing Test HAS_BUILTINS_ATOMICS
-- Performing Test HAS_BUILTINS_ATOMICS - Success
-- Performing Test HAS_ATTR_COLD
-- Performing Test HAS_ATTR_COLD - Success
-- Performing Test HAS_ATTR_ALLOC
-- Performing Test HAS_ATTR_ALLOC - Failed
-- Performing Test HAS_WIN32_ATOMICS
-- Performing Test HAS_WIN32_ATOMICS - Failed
-- Performing Test HAS_WIN32_ATTR_ALLOC
-- Performing Test HAS_WIN32_ATTR_ALLOC - Failed
-- Performing Test HAS_WIN32_UNUSED
-- Performing Test HAS_WIN32_UNUSED - Failed

They are normal and expected since you re using gcc 4.8 and being on an
unix system.

I would suggest however typing this cmake command instead
cmake -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas -DCMAKE_BUILD_TYPE=Release

then when you ll type make, normally the api should compile.

Kindest regards.

On Fri, 3 Jun 2022 at 16:58, Amol Arote  wrote:

>   Thank you sir , For Prompt Reply
>
> I install Some dependancies on CentOS 7.6 as per your last update
> libcurl-devel , libzip-devel
> But Seen Some failed Test Below , So let us know what Further Changes to
> be done
> OR can Ignore these Failed Test
>
>
>
>
> ---
> Error while compiling
>
> ---
>
>
> [root@test Src]# cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> -- The C compiler identification is GNU 4.8.5
> -- The CXX compiler identification is GNU 4.8.5
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Found PCRE: /usr/include
> -- Found CURL: /usr/lib64/libcurl.so (found version "7.29.0")
> -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
> -- Performing Test HAVE_BUILTIN__BOOL
> -- Performing Test HAVE_BUILTIN__BOOL - Success
> -- Found OpenSSL: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so (found
> version "1.0.2k")
> -- Found OpenSSL MD5
> -- Performing Test HAS_CURLSSLSET
> -- Performing Test HAS_CURLSSLSET - Failed
> -- Found ZIP: /usr/lib64/libzip.so
> -- Performing Test HAS_STD_ATOMICS
> -- Performing Test HAS_STD_ATOMICS - Failed
> -- Performing Test HAS_BUILTINS_ATOMICS
> -- Performing Test HAS_BUILTINS_ATOMICS - Success
> -- Performing Test HAS_ATTR_COLD
> -- Performing Test HAS_ATTR_COLD - Success
> -- Performing Test HAS_ATTR_ALLOC
> -- Performing Test HAS_ATTR_ALLOC - Failed
> -- Performing Test HAS_WIN32_ATOMICS
> -- Performing Test HAS_WIN32_ATOMICS - Failed
> -- Performing Test HAS_WIN32_ATTR_ALLOC
> -- Performing Test HAS_WIN32_ATTR_ALLOC - Failed
> -- Performing Test HAS_WIN32_UNUSED
> -- Performing Test HAS_WIN32_UNUSED - Failed
> --  version
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /opt/deviceatlas/Src
>
>
>
> Regards,
>
>
>
> Amol Arote
>
> Senior IT Manager
>
>
>
> *Mobile*: 9773868585 | 8097988585
>
> *Phone:*  (022) 61934700 Ext 444
>
> *Email:* amol.ar...@naaptol.com
>
> *Web:* *https://www.naaptol.com *
>
>
>
>
> On Fri, Jun 3, 2022 at 12:25 PM David CARLIER  wrote:
>
>> Hi Amole and thanks for your report.
>>
>> The C api 2.4.0 version is a major upgrade which comes with additional
>> dependencies.
>> Indeed as mentioned by your report
>>
>> ...
>> > > -- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR)
>> ...
>> > > -- Could NOT find ZIP
>> ...
>>
>> Here a sample of the README.Unix.html doc page
>>
>>  RedHat/CentOS 
>> CentOS < 8 version
>> Note: the libzip-devel system package is obsolete, thus the
>> libzip-last-devel
>> ought to be used instead. A third party repository might need to be
>> enabled.
>> ```shell
>> % sudo dnf install gcc (or clang) pcre-devel cmake zlib-devel
>> libzip-last-devel \
>>   libcurl-devel
>> ```
>> CentOS >= 8 version
>> ```shell
>> % sudo dnf install gcc (or clang) pcre-devel cmake zlib-devel
>> libzip-devel \
>>   libcurl-devel
>> ```
>>
>> If you have any further question, please let me know.
>>
>> Kindest regards.
>>
>> On Fri, 3 Jun 2022 at 07:21, Willy Tarreau  wrote:
>> >
>> > Hello Amol,
>> >
>> > On Fri, Jun 03, 2022 at 11:09:07AM +0530, Amol Arote wrote:
>> >
>> > David, please find the rest of the report below.
>> >
>> > Thanks!
>> > Willy
>> >
>> > >
>> ---
>> > > *Versions*
>> > >
>> ---
>> > > HAProxy version 2.4.2-553dee3 2021/07/07
>> > > cmake version 2.8.12.2
>> > > CentOS Linux release 7.6.1810 (Core)
>> 

Re: deviceatlas compiler error

2022-06-03 Thread Amol Arote
  Thank you sir , For Prompt Reply

I install Some dependancies on CentOS 7.6 as per your last update
libcurl-devel , libzip-devel
But Seen Some failed Test Below , So let us know what Further Changes to be
done
OR can Ignore these Failed Test



---
Error while compiling
---


[root@test Src]# cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found PCRE: /usr/include
-- Found CURL: /usr/lib64/libcurl.so (found version "7.29.0")
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
-- Performing Test HAVE_BUILTIN__BOOL
-- Performing Test HAVE_BUILTIN__BOOL - Success
-- Found OpenSSL: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so (found
version "1.0.2k")
-- Found OpenSSL MD5
-- Performing Test HAS_CURLSSLSET
-- Performing Test HAS_CURLSSLSET - Failed
-- Found ZIP: /usr/lib64/libzip.so
-- Performing Test HAS_STD_ATOMICS
-- Performing Test HAS_STD_ATOMICS - Failed
-- Performing Test HAS_BUILTINS_ATOMICS
-- Performing Test HAS_BUILTINS_ATOMICS - Success
-- Performing Test HAS_ATTR_COLD
-- Performing Test HAS_ATTR_COLD - Success
-- Performing Test HAS_ATTR_ALLOC
-- Performing Test HAS_ATTR_ALLOC - Failed
-- Performing Test HAS_WIN32_ATOMICS
-- Performing Test HAS_WIN32_ATOMICS - Failed
-- Performing Test HAS_WIN32_ATTR_ALLOC
-- Performing Test HAS_WIN32_ATTR_ALLOC - Failed
-- Performing Test HAS_WIN32_UNUSED
-- Performing Test HAS_WIN32_UNUSED - Failed
--  version
-- Configuring done
-- Generating done
-- Build files have been written to: /opt/deviceatlas/Src



Regards,



Amol Arote

Senior IT Manager



*Mobile*: 9773868585 | 8097988585

*Phone:*  (022) 61934700 Ext 444

*Email:* amol.ar...@naaptol.com

*Web:* *https://www.naaptol.com *




On Fri, Jun 3, 2022 at 12:25 PM David CARLIER  wrote:

> Hi Amole and thanks for your report.
>
> The C api 2.4.0 version is a major upgrade which comes with additional
> dependencies.
> Indeed as mentioned by your report
>
> ...
> > > -- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR)
> ...
> > > -- Could NOT find ZIP
> ...
>
> Here a sample of the README.Unix.html doc page
>
>  RedHat/CentOS 
> CentOS < 8 version
> Note: the libzip-devel system package is obsolete, thus the
> libzip-last-devel
> ought to be used instead. A third party repository might need to be
> enabled.
> ```shell
> % sudo dnf install gcc (or clang) pcre-devel cmake zlib-devel
> libzip-last-devel \
>   libcurl-devel
> ```
> CentOS >= 8 version
> ```shell
> % sudo dnf install gcc (or clang) pcre-devel cmake zlib-devel libzip-devel
> \
>   libcurl-devel
> ```
>
> If you have any further question, please let me know.
>
> Kindest regards.
>
> On Fri, 3 Jun 2022 at 07:21, Willy Tarreau  wrote:
> >
> > Hello Amol,
> >
> > On Fri, Jun 03, 2022 at 11:09:07AM +0530, Amol Arote wrote:
> >
> > David, please find the rest of the report below.
> >
> > Thanks!
> > Willy
> >
> > >
> ---
> > > *Versions*
> > >
> ---
> > > HAProxy version 2.4.2-553dee3 2021/07/07
> > > cmake version 2.8.12.2
> > > CentOS Linux release 7.6.1810 (Core)
> > > deviceatlas-enterprise-c-2.4.0.zip
> > >
> ---
> > > *deviceatlas compile steps*
> > >
> ---
> > > # yum install cmake
> > > # unzip deviceatlas-enterprise-c-2.4.0.zip
> > > # mv deviceatlas-enterprise-c-2.4.0  deviceatlas
> > > # cd /opt/deviceatlas/Src/
> > > # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> > > # make
> > >
> ---
> > > *Error while compiling woth command *
> > > * # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas   *
> > >
> ---
> > > [root@tt Src]## cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> > > -- The C compiler identification is GNU 4.8.5
> > > -- The CXX compiler identification is GNU 4.8.5
> > > -- Check for working C compiler: /usr/bin/cc
> > > -- Check for working C compiler: /usr/bin/cc -- works
> > > -- Detecting C compiler ABI info
> > > -- Detecting C compiler ABI info - done
> > > -- Check for working CXX compiler: 

Re: Rate Limiting with token/leaky bucket algorithm

2022-06-03 Thread Seena Fallah
Do you see any diff between my conf and the one in the link? :/

On Fri, 3 Jun 2022 at 17:37, Aleksandar Lazic  wrote:

> Hi.
>
> On Fri, 3 Jun 2022 17:12:25 +0200
> Seena Fallah  wrote:
>
> > When using the below config to have 100req/s rate-limiting after passing
> > the 100req/s all of the reqs will deny not reqs more than 100req/s!
> > ```
> > listen test
> > bind :8000
> > stick-table  type ip  size 100k expire 30s store http_req_rate(1s)
> > http-request track-sc0 src
> > http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
> > http-request return status 200 content-type "text/plain" lf-string
> "200
> > OK"
> > ```
> >
> > Is there a way to deny reqs more than 100 not all of them?
> > For example, if we have 1000req/s, 100reqs get "200 OK" and the rest of
> > them (900reqs) gets "429"?
>
> Yes.
>
> Here are some examples with explanation.
> https://www.haproxy.com/blog/four-examples-of-haproxy-rate-limiting/
>
> Here some search outputs, maybe some of the examples helps you to.
> https://html.duckduckgo.com/html?q=haproxy%20rate%20limiting
>
> Regards
> Alex
>


Re: Rate Limiting with token/leaky bucket algorithm

2022-06-03 Thread Aleksandar Lazic
Hi.

On Fri, 3 Jun 2022 17:12:25 +0200
Seena Fallah  wrote:

> When using the below config to have 100req/s rate-limiting after passing
> the 100req/s all of the reqs will deny not reqs more than 100req/s!
> ```
> listen test
> bind :8000
> stick-table  type ip  size 100k expire 30s store http_req_rate(1s)
> http-request track-sc0 src
> http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
> http-request return status 200 content-type "text/plain" lf-string "200
> OK"
> ```
> 
> Is there a way to deny reqs more than 100 not all of them?
> For example, if we have 1000req/s, 100reqs get "200 OK" and the rest of
> them (900reqs) gets "429"?

Yes.

Here are some examples with explanation.
https://www.haproxy.com/blog/four-examples-of-haproxy-rate-limiting/

Here some search outputs, maybe some of the examples helps you to.
https://html.duckduckgo.com/html?q=haproxy%20rate%20limiting

Regards
Alex



Rate Limiting with token/leaky bucket algorithm

2022-06-03 Thread Seena Fallah
When using the below config to have 100req/s rate-limiting after passing
the 100req/s all of the reqs will deny not reqs more than 100req/s!
```
listen test
bind :8000
stick-table  type ip  size 100k expire 30s store http_req_rate(1s)
http-request track-sc0 src
http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }
http-request return status 200 content-type "text/plain" lf-string "200
OK"
```

Is there a way to deny reqs more than 100 not all of them?
For example, if we have 1000req/s, 100reqs get "200 OK" and the rest of
them (900reqs) gets "429"?


Re: haproxy 2.6.0 and quic

2022-06-03 Thread Shawn Heisey

On 6/3/22 06:47, Markus Rietzler wrote:

my build command was

make TARGET=linux-glibc USE_OPENSSL=1 SSL_INC=/opt/quictls/include 
SSL_LIB=/opt/quictls/lib64 LDFLAGS="-Wl,-rpath,/opt/quictls/lib64" 
ADDLIB="-lz -ldl" USE_ZLIB=1 USE_PCRE=1 USE_PCRE=yes USE_LUA=1 
LUA_LIB_NAME=lua5.3  LUA_INC=/usr/include/lua5.3 ;


You will need to add USE_QUIC=1 to the build flags.  A small note: you 
have USE_PCRE twice.  IMHO, you should install PCRE2 and configure 
USE_PCRE2_JIT=1 instead.  The original PCRE library isn't being 
maintained, only version 2 will see bugfixes.


A word of warning that you would probably also get from the devs here:  
HTTP3/QUIC support is still new and not entirely working. I have it 
configured and it only works correctly for VERY simple websites.  Any 
complex webapp I try it on will fail in some way, but if I disable HTTP3 
and use HTTP2, it works.


Thanks,
Shawn




Re: haproxy 2.6.0 and quic

2022-06-03 Thread Jarno Huuskonen
Hi,

On Fri, 2022-06-03 at 14:47 +0200, Markus Rietzler wrote:
> 
> Hi,
> 
> we are using haproxy 2.4.17 at the moment. i have compiled haproxy 2.6
> with quic support and quctls
> 
> when i no check my config i get
> 
> /opt/haproxy-260# /opt/haproxy-260/sbin/haproxy -c -f haproxy.cfg
> [NOTICE]   (35905) : haproxy version is 2.6.0-a1efc04
> [NOTICE]   (35905) : path to executable is /opt/haproxy-260/sbin/haproxy
> [WARNING]  (35905) : config : parsing [haproxy.cfg:100]: 'log-format'
> overrides previous 'option httplog' in 'defaults' 
> section.
> [ALERT]    (35905) : config : parsing [haproxy.cfg:213] : 'bind' :
> unsupported stream protocol for datagram family 2 
> address 'quic4@:4443'; QUIC is not compiled in if this is what you were
> looking for.

I don't think you've QUIC support compiled. I think you're missing
USE_QUIC=1 build option.

> 
> my build command was
> 
> make TARGET=linux-glibc USE_OPENSSL=1 SSL_INC=/opt/quictls/include
> SSL_LIB=/opt/quictls/lib64 
> LDFLAGS="-Wl,-rpath,/opt/quictls/lib64" ADDLIB="-lz -ldl" USE_ZLIB=1
> USE_PCRE=1 USE_PCRE=yes USE_LUA=1 
> LUA_LIB_NAME=lua5.3  LUA_INC=/usr/include/lua5.3 ;
> 
> 
> -PROCCTL +THREAD_DUMP -EVPORTS -OT -QUIC -PROMEX -MEMORY_PROFILING

-QUIC --> QUIC support missing.

-Jarno

-- 
Jarno Huuskonen



haproxy 2.6.0 and quic

2022-06-03 Thread Markus Rietzler



Hi,

we are using haproxy 2.4.17 at the moment. i have compiled haproxy 2.6 with 
quic support and quctls

when i no check my config i get

/opt/haproxy-260# /opt/haproxy-260/sbin/haproxy -c -f haproxy.cfg
[NOTICE]   (35905) : haproxy version is 2.6.0-a1efc04
[NOTICE]   (35905) : path to executable is /opt/haproxy-260/sbin/haproxy
[WARNING]  (35905) : config : parsing [haproxy.cfg:100]: 'log-format' overrides previous 'option httplog' in 'defaults' 
section.
[ALERT](35905) : config : parsing [haproxy.cfg:213] : 'bind' : unsupported stream protocol for datagram family 2 
address 'quic4@:4443'; QUIC is not compiled in if this is what you were looking for.

[ALERT](35905) : config : Error(s) found in configuration file : haproxy.cfg
[ALERT](35905) : config : Fatal errors found in configuration.

the bind part looks like


frontend https
bind 12.34.56.79:4443 ssl crt /opt/haproxy/haproxy.ssl.crt crt /opt/haproxy/domain.pem crt /opt/haproxy/domain2.pem 
alpn h2,http/1.1

# enables HTTP/3 over QUIC
bind quic4@:4443 ssl crt /opt/haproxy/haproxy.ssl.crt crt 
/opt/haproxy/domain.pem crt /opt/haproxy/domain2.pem alpn h3


could it be a problem with my network setup?

i have to network cards in my VM. one for internal and one for external 
connections

the external connects has to virtual ip address


2: eth0:  mtu 1500 qdisc pfifo_fast state UP 
group default qlen 1000
link/ether 02:01:4d:66:f4:62 brd ff:ff:ff:ff:ff:ff
inet 46.16.79.137/24 brd 46.16.79.137 scope global eth0
   valid_lft forever preferred_lft forever
inet 46.16.74.36/32 scope global eth0
   valid_lft forever preferred_lft forever
inet6 fe80::1:4dff:fe66:f462/64 scope link
   valid_lft forever preferred_lft forever




my build command was

make TARGET=linux-glibc USE_OPENSSL=1 SSL_INC=/opt/quictls/include SSL_LIB=/opt/quictls/lib64 
LDFLAGS="-Wl,-rpath,/opt/quictls/lib64" ADDLIB="-lz -ldl" USE_ZLIB=1 USE_PCRE=1 USE_PCRE=yes USE_LUA=1 
LUA_LIB_NAME=lua5.3  LUA_INC=/usr/include/lua5.3 ;




HAProxy version 2.6.0-a1efc04 2022/05/31 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2027.
Known bugs: http://www.haproxy.org/bugs/bugs-2.6.0.html
Running on: Linux Ubuntu
Build options :
  TARGET  = linux-glibc
  CPU = generic
  CC  = cc
  CFLAGS  = -O2 -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -fwrapv 
-Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered 
-Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment

  OPTIONS = USE_PCRE=yes USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1
  DEBUG   = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE -PCRE_JIT -PCRE2 -PCRE2_JIT +POLL +THREAD +BACKTRACE -STATIC_PCRE 
-STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -ENGINE +GETADDRINFO +OPENSSL +LUA +ACCEPT4 
-CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL 
-PROCCTL +THREAD_DUMP -EVPORTS -OT -QUIC -PROMEX -MEMORY_PROFILING


Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=2).
Built with OpenSSL version : OpenSSL 3.0.3+quic 3 May 2022
Running on OpenSSL version : OpenSSL 3.0.3+quic 3 May 2022
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
OpenSSL providers loaded : default
Built with Lua version : Lua 5.3.1
Built with network namespace support.
Support for malloc_trim() is enabled.
Built with zlib version : 
Running on zlib version : 
Compression algorithms supported : identity("identity"), deflate("deflate"), 
raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT 
IP_FREEBIND
Built with PCRE version : 
Running on PCRE version : 
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with gcc compiler version ...

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as  cannot be specified using 'proto' keyword)
 h2 : mode=HTTP  side=FE|BE  mux=H2flags=HTX|HOL_RISK|NO_UPG
   fcgi : mode=HTTP  side=BE mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
   : mode=HTTP  side=FE|BE  mux=H1flags=HTX
 h1 : mode=HTTP  side=FE|BE  mux=H1flags=HTX|NO_UPG
   : mode=TCP   side=FE|BE  mux=PASS  flags=
   none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : none

Available filters :
[CACHE] cache
[COMP] compression
[FCGI] fcgi-app
[SPOE] spoe
[TRACE] trace



Re: [PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread Tim Düsterhus

Thayne,

On 6/3/22 09:51, astrotha...@gmail.com wrote:

Add a converter that extracts a parameter from string of delimited
key/value pairs.


Just wondering (maybe something to add to the doc or test): Should this 
handle URL encoded parameter names or parameter values? It probably 
should not, because that's makes the converter less general.


But it would certainly be useful to explain how to properly retrieve 
those values. Simply url-decoding the full query string before doesn't 
do the job, because then the additional delimiters might be introduced. 
This likely needs to be combined with the URI normalization feature, as 
the encoding of a parameter name is not a 1:1 relationship.


Best regards
Tim Düsterhus



Re: [PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread Thayne McCombs

There were a couple of things I wasn't entirely sure about:

1. Should this allow specifying the separator between key and value, 
rather than always using "="?


2. How should it handle the case where there isn't a value given, the 
current implementation treats "a" as equivalent to "a="





[PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread astrothayne
From: Thayne McCombs 

Add a converter that extracts a parameter from string of delimited
key/value pairs.

Fixes: #1697
---
 doc/configuration.txt | 19 +
 reg-tests/converter/param.vtc | 80 +++
 src/sample.c  | 64 ++--
 3 files changed, 160 insertions(+), 3 deletions(-)
 create mode 100644 reg-tests/converter/param.vtc

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 927c97ce3..d9f47c2eb 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -17411,6 +17411,25 @@ or()
   This prefix is followed by a name. The separator is a '.'. The name may only
   contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.

+param(,[])
+  This extracts the first occurence of the parameter  in the input string
+  where parameters are delimited by , which defaults to "&", and the 
name
+  and value of the parameter are separated by a "=". If there is no "=" and 
value
+  before the end of the parameter segment, it is treated as equivalent to a 
value
+  of an empty string.
+
+  This can be useful for extracting parameters from a query string, or 
possibly a
+  x-www-form-urlencoded body. In particular, `query,param()` can be used 
as
+  an alternative to `urlp()` which only uses "&" as a delimiter, whereas 
urlp
+  also uses "?" and ";".
+
+  Example :
+  str(a=b=d=r),param(a)   # b
+  str(a=c),param(a) # ""
+  str(a==a),param(b)  # ""
+  str(a=1;b=2;c=4),param(b,;) # 2
+  query,param(redirect_uri)
+
 protobuf(,[])
   This extracts the protocol buffers message field in raw mode of an input 
binary
   sample representation of a protocol buffer message with  as 
field
diff --git a/reg-tests/converter/param.vtc b/reg-tests/converter/param.vtc
new file mode 100644
index 0..163360382
--- /dev/null
+++ b/reg-tests/converter/param.vtc
@@ -0,0 +1,80 @@
+varnishtest "param converter Test"
+
+feature ignore_unknown_macro
+
+server s1 {
+   rxreq
+   txresp -hdr "Connection: close"
+} -repeat 10 -start
+
+haproxy h1 -conf {
+   defaults
+   mode http
+   timeout connect "${HAPROXY_TEST_TIMEOUT-5s}"
+   timeout client  "${HAPROXY_TEST_TIMEOUT-5s}"
+   timeout server  "${HAPROXY_TEST_TIMEOUT-5s}"
+
+   frontend fe
+   bind "fd@${fe}"
+
+   ### requests
+   http-request set-var(txn.query) query
+   http-response set-header Found %[var(txn.query),param(test)] if { 
var(txn.query),param(test) -m found }
+
+   default_backend be
+
+   backend be
+   server s1 ${s1_addr}:${s1_port}
+} -start
+
+client c1 -connect ${h1_fe_sock} {
+   txreq -url "/foo/?test=1=4"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "1"
+
+   txreq -url "/?a=1=4=34"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "34"
+
+   txreq -url "/?test=bar"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "bar"
+
+   txreq -url "/?a=b=d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == ""
+
+   txreq -url "/?a=b=t=d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "t"
+
+   txreq -url "/?a=b=d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == ""
+
+   txreq -url "/?test="
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == ""
+
+txreq -url "/?a=b"
+rxresp
+expect resp.status == 200
+expect resp.http.found == ""
+
+txreq -url "/?testing=123"
+rxresp
+expect resp.status == 200
+expect resp.http.found == ""
+
+txreq -url "/?testing=123=4"
+rxresp
+expect resp.status == 200
+expect resp.http.found == "4"
+} -run
diff --git a/src/sample.c b/src/sample.c
index 237b88056..b2c80b6c8 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -2582,6 +2582,65 @@ static int sample_conv_word(const struct arg *arg_p, 
struct sample *smp, void *p
return 1;
 }

+static int sample_conv_param_check(struct arg *arg, struct sample_conv *conv,
+   const char *file, int line, char **err)
+{
+   if (arg[1].type == ARGT_STR && arg[1].data.str.data != 1) {
+   memprintf(err, "Delimiter must be exactly 1 character.");
+   return 0;
+   }
+
+   return 1;
+}
+
+static int sample_conv_param(const struct arg *arg_p, struct sample *smp, void 
*private)
+{
+   char *pos, *end, *pend, *equal;
+   char delim = '&';
+   const char *name = arg_p[0].data.str.area;
+   size_t name_l = arg_p[0].data.str.data;
+
+   if (arg_p[1].type == ARGT_STR)
+   delim = *arg_p[1].data.str.area;
+
+   pos = smp->data.u.str.area;
+   end = pos + smp->data.u.str.data;
+   while (pos < end) {
+   equal = pos + name_l;
+   /* Parameter not found */
+   if (equal > end)
+

[PATCH 1/2] CLEANUP/MINOR: sample: factor out common code for setting buffer

2022-06-03 Thread astrothayne
From: Thayne McCombs 

A couple of functions in sample.c have almost identical code for
updating the samples string buffer. This adds a new helper function to
do this and uses it in both places.
---
 include/haproxy/buf.h | 22 ++
 src/sample.c  | 29 ++---
 2 files changed, 24 insertions(+), 27 deletions(-)

diff --git a/include/haproxy/buf.h b/include/haproxy/buf.h
index 4ea4b73f1..303635c39 100644
--- a/include/haproxy/buf.h
+++ b/include/haproxy/buf.h
@@ -940,6 +940,28 @@ static inline int b_peek_varint(struct buffer *b, size_t 
ofs, uint64_t *vptr)
return size;
 }
 
+/*
+ * b_set_area_sub(): Replace the current buffer with a sub-slice of the
+ * current buffer. Sets  to ,  to , and
+ *  to the new size accounting for the change to area. The range from
+ *  to  must be within the range of the current 
buffer.
+ */
+static inline void b_set_area_sub(struct buffer *b, char *new_area, size_t 
new_data)
+{
+   BUG_ON_HOT(new_area < b->area);
+   BUG_ON_HOT(new_area + new_data > b->area + b->data);
+
+   b->data = new_data;
+   /* If buffer is len 0, no need to
+   change pointers or to update size */
+   if (!new_data)
+   return;
+   /* Compute remaining size if needed */
+   if (b->size)
+   b->size -= new_area - b->area;
+   b-> area = new_area;
+}
+
 
 /*
  * Buffer ring management.
diff --git a/src/sample.c b/src/sample.c
index 50ae76b6e..237b88056 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -2490,19 +2490,7 @@ static int sample_conv_field(const struct arg *arg_p, 
struct sample *smp, void *
return 0;
}
 found:
-   smp->data.u.str.data = end - start;
-   /* If ret string is len 0, no need to
-   change pointers or to update size */
-   if (!smp->data.u.str.data)
-   return 1;
-
-   /* Compute remaining size if needed
-   Note: smp->data.u.str.size cannot be set to 0 */
-   if (smp->data.u.str.size)
-   smp->data.u.str.size -= start - smp->data.u.str.area;
-
-   smp->data.u.str.area = start;
-
+   b_set_area_sub(&(smp->data.u.str), start, end - start);
return 1;
 }
 
@@ -2590,20 +2578,7 @@ static int sample_conv_word(const struct arg *arg_p, 
struct sample *smp, void *p
return 1;
}
 found:
-   smp->data.u.str.data = end - start;
-   /* If ret string is len 0, no need to
-   change pointers or to update size */
-   if (!smp->data.u.str.data)
-   return 1;
-
-
-   /* Compute remaining size if needed
-   Note: smp->data.u.str.size cannot be set to 0 */
-   if (smp->data.u.str.size)
-   smp->data.u.str.size -= start - smp->data.u.str.area;
-
-   smp->data.u.str.area = start;
-
+   b_set_area_sub(&(smp->data.u.str), start, end - start);
return 1;
 }
 
-- 
2.36.1




Re: deviceatlas compiler error

2022-06-03 Thread David CARLIER
Hi Amole and thanks for your report.

The C api 2.4.0 version is a major upgrade which comes with additional
dependencies.
Indeed as mentioned by your report

...
> > -- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR)
...
> > -- Could NOT find ZIP
...

Here a sample of the README.Unix.html doc page

 RedHat/CentOS 
CentOS < 8 version
Note: the libzip-devel system package is obsolete, thus the libzip-last-devel
ought to be used instead. A third party repository might need to be enabled.
```shell
% sudo dnf install gcc (or clang) pcre-devel cmake zlib-devel
libzip-last-devel \
  libcurl-devel
```
CentOS >= 8 version
```shell
% sudo dnf install gcc (or clang) pcre-devel cmake zlib-devel libzip-devel \
  libcurl-devel
```

If you have any further question, please let me know.

Kindest regards.

On Fri, 3 Jun 2022 at 07:21, Willy Tarreau  wrote:
>
> Hello Amol,
>
> On Fri, Jun 03, 2022 at 11:09:07AM +0530, Amol Arote wrote:
>
> David, please find the rest of the report below.
>
> Thanks!
> Willy
>
> > ---
> > *Versions*
> > ---
> > HAProxy version 2.4.2-553dee3 2021/07/07
> > cmake version 2.8.12.2
> > CentOS Linux release 7.6.1810 (Core)
> > deviceatlas-enterprise-c-2.4.0.zip
> > ---
> > *deviceatlas compile steps*
> > ---
> > # yum install cmake
> > # unzip deviceatlas-enterprise-c-2.4.0.zip
> > # mv deviceatlas-enterprise-c-2.4.0  deviceatlas
> > # cd /opt/deviceatlas/Src/
> > # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> > # make
> > ---
> > *Error while compiling woth command *
> > * # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas   *
> > ---
> > [root@tt Src]## cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> > -- The C compiler identification is GNU 4.8.5
> > -- The CXX compiler identification is GNU 4.8.5
> > -- Check for working C compiler: /usr/bin/cc
> > -- Check for working C compiler: /usr/bin/cc -- works
> > -- Detecting C compiler ABI info
> > -- Detecting C compiler ABI info - done
> > -- Check for working CXX compiler: /usr/bin/c++
> > -- Check for working CXX compiler: /usr/bin/c++ -- works
> > -- Detecting CXX compiler ABI info
> > -- Detecting CXX compiler ABI info - done
> > -- Found PCRE: /usr/include
> > -- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR)
> > -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
> > -- Performing Test HAVE_BUILTIN__BOOL
> > -- Performing Test HAVE_BUILTIN__BOOL - Success
> > -- Could NOT find ZIP
> > -- Performing Test HAS_STD_ATOMICS
> > -- Performing Test HAS_STD_ATOMICS - Failed
> > -- Performing Test HAS_BUILTINS_ATOMICS
> > -- Performing Test HAS_BUILTINS_ATOMICS - Success
> > -- Performing Test HAS_ATTR_COLD
> > -- Performing Test HAS_ATTR_COLD - Success
> > -- Performing Test HAS_ATTR_ALLOC
> > -- Performing Test HAS_ATTR_ALLOC - Failed
> > -- Performing Test HAS_WIN32_ATOMICS
> > -- Performing Test HAS_WIN32_ATOMICS - Failed
> > -- Performing Test HAS_WIN32_ATTR_ALLOC
> > -- Performing Test HAS_WIN32_ATTR_ALLOC - Failed
> > -- Performing Test HAS_WIN32_UNUSED
> > -- Performing Test HAS_WIN32_UNUSED - Failed
> > --  version
> > -- Configuring done
> > -- Generating done
> > -- Build files have been written to: /opt/deviceatlas/Src
> > [root@tt Src]#  #
> >
> > Request you to please guide us on above matter
> >
> >
> > --
> >
> > Regards,
> >
> >
> >
> > Amol Arote
> >
> > Senior IT Manager
> >
> >
> >
> > *Mobile*: 9773868585 | 8097988585
> >
> > *Phone:*  (022) 61934700 Ext 444
> >
> > *Email:* amol.ar...@naaptol.com
> >
> > *Web:* *https://www.naaptol.com *
> >
> > --
> >



Re: SV: Traffic from HAproxy get error 401 and 500

2022-06-03 Thread Baptiste
Hi Henning,

Please remove this "option http-server-close" from your configuration,
entirely :)

Baptiste


Re: New .NET SPOE Library

2022-06-03 Thread Baptiste
On Thu, Jun 2, 2022 at 10:00 PM Sébastien Crocquesel <
s.crocque...@inulogic.com> wrote:

> Dear all,
>
> I create a .NET Library to build SPOP agent and release it under MIT
> Licence. The library is happily used in production for more than 2 years
> now and serve more than 10K req/s per agent node.
>
> I would be pleased if it can be referenced on the spoe wiki page with
> other current implementations.
>
> You may find more information at
> https://github.com/inulogic/HAProxy.StreamProcessingOffload.AgentFramework
>
>
> Best regards,
> Sebastien
>
>

Hi Sebastien!

Thx a lot for your contribution!
I just updated the wiki page:
https://github.com/haproxy/wiki/wiki/SPOE:-Stream-Processing-Offloading-Engine

Baptiste


Re: deviceatlas compiler error

2022-06-03 Thread Willy Tarreau
Hello Amol,

On Fri, Jun 03, 2022 at 11:09:07AM +0530, Amol Arote wrote:
> We are trying to upgrade deviceatlas for HAProxy version 2.4.2-553dee3, but
> while compiling deviceatlas its showing some error.
> Below are the versions and steps which we perform for the same.

Thanks for the report. Adding David who's the maintainer as I don't know
if he watches the list often.

Amol, please be aware that 235 fixes among which 21 rated as "major"
were applied to the 2.4 branch after your version, as such you're urged
to stop using it and to update it. But that shouldn't be related to your
DA build issue.

David, please find the rest of the report below.

Thanks!
Willy

> ---
> *Versions*
> ---
> HAProxy version 2.4.2-553dee3 2021/07/07
> cmake version 2.8.12.2
> CentOS Linux release 7.6.1810 (Core)
> deviceatlas-enterprise-c-2.4.0.zip
> ---
> *deviceatlas compile steps*
> ---
> # yum install cmake
> # unzip deviceatlas-enterprise-c-2.4.0.zip
> # mv deviceatlas-enterprise-c-2.4.0  deviceatlas
> # cd /opt/deviceatlas/Src/
> # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> # make
> ---
> *Error while compiling woth command *
> * # cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas   *
> ---
> [root@tt Src]## cmake . -DCMAKE_INSTALL_PREFIX=/opt/deviceatlas
> -- The C compiler identification is GNU 4.8.5
> -- The CXX compiler identification is GNU 4.8.5
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Found PCRE: /usr/include
> -- Could NOT find CURL (missing:  CURL_LIBRARY CURL_INCLUDE_DIR)
> -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.7")
> -- Performing Test HAVE_BUILTIN__BOOL
> -- Performing Test HAVE_BUILTIN__BOOL - Success
> -- Could NOT find ZIP
> -- Performing Test HAS_STD_ATOMICS
> -- Performing Test HAS_STD_ATOMICS - Failed
> -- Performing Test HAS_BUILTINS_ATOMICS
> -- Performing Test HAS_BUILTINS_ATOMICS - Success
> -- Performing Test HAS_ATTR_COLD
> -- Performing Test HAS_ATTR_COLD - Success
> -- Performing Test HAS_ATTR_ALLOC
> -- Performing Test HAS_ATTR_ALLOC - Failed
> -- Performing Test HAS_WIN32_ATOMICS
> -- Performing Test HAS_WIN32_ATOMICS - Failed
> -- Performing Test HAS_WIN32_ATTR_ALLOC
> -- Performing Test HAS_WIN32_ATTR_ALLOC - Failed
> -- Performing Test HAS_WIN32_UNUSED
> -- Performing Test HAS_WIN32_UNUSED - Failed
> --  version
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /opt/deviceatlas/Src
> [root@tt Src]#  #
> 
> Request you to please guide us on above matter
> 
> 
> -- 
> 
> Regards,
> 
> 
> 
> Amol Arote
> 
> Senior IT Manager
> 
> 
> 
> *Mobile*: 9773868585 | 8097988585
> 
> *Phone:*  (022) 61934700 Ext 444
> 
> *Email:* amol.ar...@naaptol.com
> 
> *Web:* *https://www.naaptol.com *
> 
> -- 
>