Re: [RFC] setting the backend SNI from the client's authority TLV, when the target address was forwarded

[Willy Tarreau]

Hello Geoff,

On Wed, Aug 21, 2019 at 05:33:18PM +0200, Geoff Simmons wrote:
> Hello to readers of the haproxy list,
> 
> This is my first-ever mail to the list,

Thus welcome to the list :-)

> to propose my first-ever
> contribution to the project, so I apologize in advance if anything here
> runs afoul of your customs. I hope I've come close enough to make it
> worth your while.

No problem, and thanks for the precision.

> What I'm proposing has been coded up here:
> 
> https://github.com/slimhazard/haproxy/commit/2093dc54f0d5f7d0fb30c918c2c7689b8764bec1
> 
> And of course I can send a git-formatted patch to the list (but my
> understanding is that an RFC discussion is preferred first).

If you want to discuss the concept first the patches are not needed
indeed, though when the code exists sometimes it helps understand your
choices. With this said, discussing away from the code sometimes helps
one think again about certain choices and that's not bad.

> The idea is probably best understood from a sample config:
> 
> listen fe
>   # ...
>   server s0 0.0.0.0:0 ssl verify none forward-dst-sni
> 
> This is only relevant for the case of "forwarding" the client's
> destination address, by using 0.0.0.0 or * in the server config. When
> the new option 'forward-dst-sni' is enabled for such a server, then for
> clients who send an authority TLV in the PROXY header, the SNI for the
> backend is set to the value of the TLV.

OK.

> The use case is for clients who want to use haproxy's "address
> forwarding" feature, but also need to communicate with backends via
> haproxy that use SNI for "virtual hosing with SSL". With this feature
> they can do that by using PROXYv2 and TLV.

Just to know a bit more about your use case, thus your client speaks
in clear to haproxy by prepending a PPv2 header and lets haproxy serve
as a TLS "onloader" if I can call it like this, that's it ? If so, this
is pretty close to what is also being done in Varnish to allow it to
use haproxy to gateway to TLS servers :

https://github.com/varnishcache/varnish-cache/pull/2957

> The option is a NOP for servers that don't have the 0.0.0.0/* config.
> When it's enabled for such a server, and when the authority TLV is sent,
> then the authority value overrides any SNI set in the client TLS
> handshake. The rationale is that if a client goes to the trouble of
> sending the TLV, then they want that to be used for the SNI (otherwise
> they can just let the SNI be passed on from the client TLS).

I understand your point, I have an objection but will explain below :-)

> I considered adding a test in reg-tests, but I don't see a way to set a
> PROXY TLV value using VTest and haproxy (happy to be corrected if I'm
> wrong). But we do have some varnishtest VTCs for it here, for a PR that
> my colleague and I are working on for Varnish:
> 
> https://github.com/slimhazard/varnish-cache/blob/proxy_via_authority/bin/varnishtest/tests/c00100.vtc
> 
> https://github.com/slimhazard/varnish-cache/blob/proxy_via_authority/bin/varnishtest/tests/c00101.vtc

I'm not aware of a current solution to use the PROXY proto in vtest either.

> As for the coding, I suspect that my newbie status with haproxy shows
> mostly in the way space is allocated for the authority string, 

It's not dramatic, as long as you're interested in learning to improve,
you will, like about all of us here.

> which the
> patch adds as a new field to struct connection. Since the feature is
> likely to be used only rarely, it doesn't seem sensible to foresee all
> of the space for a host name right in the struct.

Indeed. And in fact it was the main reason why it was not implemented
since the field was specified. But very recently (in 2.1-dev) we moved
a few fields so that addresses are dynamically allocated (for the same
reason which is that allocating 128 bytes for a sockaddr_storage to
store a destination address that nobody uses is a waste of RAM). And I'd
really like that you do the same with the SNI extracted from the PPv2
header, so that it's not wasted when not used.

> In the patch I have it
> as a pointer to struct buffer, which is NULL unless the authority TLV is
> read from the PROXY header.

Well, buffer :-)  Some people (namely those playing with WAFs) can use
huge buffers, 1 MB or more. So as you can see it's not suited, better
create a new pool for this.

> It seemed to make sense to use a memory pool for the allocation, but it
> also felt like overkill to add a new memory pool just for this feature.

No it's not overkill, pools are extremely cheap and they are fused when
they have very close sizes. I just had a quick look and saw that appctx
is 248 bytes si it will end up being fused with your pool, so by adding
a pool, your entries will cost exactly zero. So really, don't be shy on
this.

> So in the patch I'm just using the "trash pool".

Got it, but please, no :-)

> Aside from the fact that the name "trash pool" makes me think that it
> might not be meant 

Re: Support for per-front-end configuration of tune* parameters

[Willy Tarreau]

Hi Richard,

On Wed, Aug 21, 2019 at 09:40:55AM +0100, Richard Mason wrote:
> Hi HAProxiers,
> 
> I'm using HAProxy 1.8 on Centos, and I was wondering if it is possible to
> configure the tune* parameters by front end?
> 
> Specifically, we have a large number of connections, and need to set:
> 
> tune.sndbuf.client 2048000
> 
> We have a few hundred ports to proxy, but not all require such a large
> sendbuffer, and the difference is a meaningful quantity of memory. Ideally
> it would only be set on a portion of our proxied ports.

I can easily understand, and thought about it when doing it, but
preferred to keep the config simple until the need arises. The
proper way to do it is by adding a set of new "bind" keywords
(the keywords you have on the "bind" lines), so that they are
applied to incoming connections, let's call them "rcvbuf" and
"sndbuf" which are pretty explicit for this line.

It's not particularly hard to do if you're interested in looking
at it, just check how "mss" is handled (it sets a maxseg value on
the listeners). The keyword is registered in proto_tcp.c. In
session.c, where rcvbuf and sndbuf are checked and applied, you'd
just have to first check if a value is set on your listener (as is
done just above for maxseg), and apply it, otherwise fall back to
the global settings.

Just give it a try, and please update the doc :-)

Cheers,
Willy

[RFC] setting the backend SNI from the client's authority TLV, when the target address was forwarded

[Geoff Simmons]

Hello to readers of the haproxy list,

This is my first-ever mail to the list, to propose my first-ever
contribution to the project, so I apologize in advance if anything here
runs afoul of your customs. I hope I've come close enough to make it
worth your while.

What I'm proposing has been coded up here:

https://github.com/slimhazard/haproxy/commit/2093dc54f0d5f7d0fb30c918c2c7689b8764bec1

And of course I can send a git-formatted patch to the list (but my
understanding is that an RFC discussion is preferred first).

The idea is probably best understood from a sample config:

listen fe
# ...
server s0 0.0.0.0:0 ssl verify none forward-dst-sni

This is only relevant for the case of "forwarding" the client's
destination address, by using 0.0.0.0 or * in the server config. When
the new option 'forward-dst-sni' is enabled for such a server, then for
clients who send an authority TLV in the PROXY header, the SNI for the
backend is set to the value of the TLV.

The use case is for clients who want to use haproxy's "address
forwarding" feature, but also need to communicate with backends via
haproxy that use SNI for "virtual hosing with SSL". With this feature
they can do that by using PROXYv2 and TLV.

The option is a NOP for servers that don't have the 0.0.0.0/* config.
When it's enabled for such a server, and when the authority TLV is sent,
then the authority value overrides any SNI set in the client TLS
handshake. The rationale is that if a client goes to the trouble of
sending the TLV, then they want that to be used for the SNI (otherwise
they can just let the SNI be passed on from the client TLS).

I considered adding a test in reg-tests, but I don't see a way to set a
PROXY TLV value using VTest and haproxy (happy to be corrected if I'm
wrong). But we do have some varnishtest VTCs for it here, for a PR that
my colleague and I are working on for Varnish:

https://github.com/slimhazard/varnish-cache/blob/proxy_via_authority/bin/varnishtest/tests/c00100.vtc

https://github.com/slimhazard/varnish-cache/blob/proxy_via_authority/bin/varnishtest/tests/c00101.vtc

As for the coding, I suspect that my newbie status with haproxy shows
mostly in the way space is allocated for the authority string, which the
patch adds as a new field to struct connection. Since the feature is
likely to be used only rarely, it doesn't seem sensible to foresee all
of the space for a host name right in the struct. In the patch I have it
as a pointer to struct buffer, which is NULL unless the authority TLV is
read from the PROXY header.

It seemed to make sense to use a memory pool for the allocation, but it
also felt like overkill to add a new memory pool just for this feature.
So in the patch I'm just using the "trash pool".

Aside from the fact that the name "trash pool" makes me think that it
might not be meant to be used this way, if I understand correctly the
default allocation size is 16KiB, and users are advised not to reduce
it. That's far too large for a host name (usually max 255 bytes). So I
suspect that this may not be best practice for haproxy. Maybe define a
dedicated pool after all?

I'd be grateful for any feedback about the feature and the code (or
anything else). And of course I can send a patch to the list if you're
interested.


Thanks,
Geoff Simmons
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de



signature.asc
Description: OpenPGP digital signature

Hello

[Victoria Sandoval]

Have you removed push notification from your website www.formilux.org

Regards,
Victoria

Re: HAProxy Dataplane API Runtime Configuration Error

[Marko Juraga]

On 21. 08. 2019. 13:59, void _in wrote:

> We have configured the new dataplane API in order to perform runtime
> configuration changes on our HAProxy. It is working perfectly fine on
> one server. However, on the other server, it is generating the
> error: level=warning msg="Runtime API not configured, not using it".
> Both servers are CentOS 7 and both have the HAProxy version HA-Proxy
> version 2.0.5 2019/08/16 - https://haproxy.org/. 

Hi,

This error message means that the dataplaneapi process cannot connect to
the configured stats socket and that the dataplaneapi will work but all
endpoints requiring access to stats socket won't work. Can you check
that the stats socket is configured correctly and that the dataplaneapi
can access it?

Regards,
Marko

HAProxy Dataplane API Runtime Configuration Error

[void _in]

Hi

We have configured the new dataplane API in order to perform runtime
configuration changes on our HAProxy. It is working perfectly fine on one
server. However, on the other server, it is generating the
error: level=warning msg="Runtime API not configured, not using it". Both
servers are CentOS 7 and both have the HAProxy version HA-Proxy version
2.0.5 2019/08/16 - https://haproxy.org/.

Running HAProxy -vv on both the servers seems exactly the same as well.
Below is the output of HAProxy -vv on the server where the dataplaneapi is
not working:

HA-Proxy version 2.0.5 2019/08/16 - https://haproxy.org/
Build options :
  TARGET  = linux-glibc
  CPU = generic
  CC  = gcc
  CFLAGS  = -O2 -g -fno-strict-aliasing -DTCP_USER_TIMEOUT=18
  OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_THREAD=1 USE_OPENSSL=1 USE_ZLIB=1
USE_SYSTEMD=1

Feature list : +EPOLL -KQUEUE -MY_EPOLL -MY_SPLICE +NETFILTER +PCRE
+PCRE_JIT -PCRE2 -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED
-REGPARM -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE
+LIBCRYPT +CRYPT_H -VSYSCALL +GETADDRINFO +OPENSSL -LUA +FUTEX +ACCEPT4
-MY_ACCEPT4 +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS
-51DEGREES -WURFL +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=56).
Built with OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
Running on OpenSSL version : OpenSSL 1.0.2k-fips  26 Jan 2017
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with network namespace support.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT
IP_FREEBIND
Built with zlib version : 1.2.7
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"),
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with PCRE version : 8.32 2012-11-30
Running on PCRE version : 8.32 2012-11-30
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as  cannot be specified using 'proto' keyword)
  h2 : mode=HTXside=FE|BE mux=H2
  h2 : mode=HTTP   side=FEmux=H2
: mode=HTXside=FE|BE mux=H1
: mode=TCP|HTTP   side=FE|BE mux=PASS

Available services : none

Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace

The dataplaneapi was compiled exactly like the example given on its github
page. Any help would be really appreciated.

Thanks

Re: Support for per-front-end configuration of tune* parameters

[Aleksandar Lazic]

Hi Richard.

Wed Aug 21 10:41:36 GMT+02:00 2019 Richard Mason :

> Hi HAProxiers,
 > I'm using HAProxy 1.8 on Centos, and I was wondering if it is possible to 
 > configure the tune* parameters by front end?

In general I suggest to use the latest 2.0, especially for your setup.

https://github.com/AxisNL/haproxy-rpmbuild

> Specifically, we have a large number of connections, and need to set:
 > tune.sndbuf.client 2048000
 > We have a few hundred ports to proxy, but not all require such a large 
 > sendbuffer, and the difference is a meaningful quantity of memory. Ideally 
 > it would only be set on a portion of our proxied ports.
 > At present, the way I can get this to work is by including the tune 
 > parameter in the global settings.
 > When I move this to the relevant frontend, then HAProxy refuses to start 
 > with the following error:
 > Aug 21 04:36:17 xxx haproxy: [ALERT] 232/043617 (84969) : parsing 
 > [/etc/haproxy/haproxy.cfg:62] : unknown keyword 'tune.sndbuf.client' in 
 > 'frontend' section
 > Is there a way to achieve this per front-end so I can narrow the ports it 
 > will be applied to?

Well, as the source say, it's for now only a global variable.

http://git.haproxy.org/?p=haproxy-2.0.git;a=blob;f=src/session.c;hb=d905f49a0dbf78b69f24ee69df10b873680b92ea#l251

I you like you can create a feature request on github for your demand.

https://github.com/haproxy/haproxy/issues

> Thanks in advance for your time and for an excellent proxy!
 >
 > Regards,
 > Richard

Best regards
 Aleks

Are you interested for Revamp your haproxy.com?

[Adam Gibbs]

Hello, *haproxy.com  Team*,



We have a bunch of expert developers who can customize and create a site
from scratch as per your needs. Our WordPress services use the best host on
the internet. We also provide premium themes from various WordPress theme
makers and create your site at an *affordable price.*



*We specialize in lots of WordPress related services that include:*



Setting up your first professional WordPress blog.

Moving from BlogSpot to word press.

Moving from *wordpress.com*

to
self-hosted word press blog.

Creating and Customizing Word Press themes.

Wordpress SEO.

Wordpress Maintenance.

Wordpress Consultancy.

Wordpress integration with social media.

Fixing hacked websites and hardening security.

Integration with email marketing services.



*We also specialize in:*



Graphic design.

E-commerce and M-commerce.

Responsive web design.

Web Branding.

Logo design.

Advertising marketing online.

Registration domain, hosting server.

Google AdWords, SEO website, Facebook advertising.



*If are you interested, please contact us for more information.*



Thanks & Regards,

*Adam Gibbs*
*Business Development Manager*
[image: beacon]

Re: HA Proxy Support for RedHat 8 Enquiries

[Bruno Henc]

The RHEL7 package for HAProxy Enterprise is fully compatible with RHEL8, 
and there's also a build against openssl 1.1.1 , so for all intents and 
purposes one can start using it on RHEL8.



Direct RHEL8 support should arrive with the release of HAProxy 
Enterprise 2.0 which should arrive at the end of Q3 or at the start of 
Q4.  We can expedite the process if needed.



If you have any further questions regarding the enterprise version feel 
free to reach out at supp...@haproxy.com or sa...@haproxy.com, the 
mailing list is oriented towards questions regarding open source 
development of the community edition.


On 8/21/19 9:42 AM, Eng, Lijwee wrote:


Hi HA Proxy Team,

Would like to check is HA Proxy compatible with RHEL 8, from the 
current compatibility , based on the current documentation, 1-9r1 
supports up to RHEL 7.


Will RHEL 8 be supported as well ?

https://www.haproxy.com/documentation/hapee/1-9r1/getting-started/os-hardware/

Please advise, thank you!

Regards

*LiJwee Eng*

Systems Engineer

*Dell Technologies**| *Data Protection Solutions

Mobile +65 97516931

lijwee@dell.com **


--
Bruno Henc
Support Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | www.haproxy.com 

Support for per-front-end configuration of tune* parameters

[Richard Mason]

Hi HAProxiers,

I'm using HAProxy 1.8 on Centos, and I was wondering if it is possible to
configure the tune* parameters by front end?

Specifically, we have a large number of connections, and need to set:

tune.sndbuf.client 2048000

We have a few hundred ports to proxy, but not all require such a large
sendbuffer, and the difference is a meaningful quantity of memory. Ideally
it would only be set on a portion of our proxied ports.

At present, the way I can get this to work is by including the tune
parameter in the global settings.

When I move this to the relevant frontend, then HAProxy refuses to start
with the following error:

Aug 21 04:36:17 xxx haproxy: [ALERT] 232/043617 (84969) : parsing
[/etc/haproxy/haproxy.cfg:62] : unknown keyword 'tune.sndbuf.client' in
'frontend' section

Is there a way to achieve this per front-end so I can narrow the ports it
will be applied to?

Thanks in advance for your time and for an excellent proxy!

Regards,
Richard

HA Proxy Support for RedHat 8 Enquiries

[Eng, Lijwee]

Hi HA Proxy Team,

Would like to check is HA Proxy compatible with RHEL 8, from the current 
compatibility , based on the current documentation, 1-9r1 supports up to RHEL 7.
Will RHEL 8 be supported as well ?
https://www.haproxy.com/documentation/hapee/1-9r1/getting-started/os-hardware/

Please advise, thank you!

Regards
LiJwee Eng
Systems Engineer
Dell Technologies | Data Protection Solutions
Mobile +65 97516931<+65%209751%206931>
lijwee@dell.com