Re: HttpClient in Lua
I don’t mind the idea, it would reduce having a separate service/ proxy. If creating it inside HAProxy then wouldn’t that mess with the threading and it blocking? > On 20 Jun 2022, at 9:47 pm, William Lallemand wrote: > > On Mon, Jun 20, 2022 at 08:27:22PM +1000, Philip Young wrote: >> Thanks for the answer William, it is very much appreciated. It is good to >> get some clarification and will stop me continuing to spend time trying to >> get it to work. >> >> In the meantime, I am working around the problem by calling out to a local >> python service running on the same machine as HAProxy over http, which is >> then making the authorisation request with a client certificate. Not ideal, >> but will switch out the hack once it is supported in HAProxy. >> >> Thanks again >> Phil >> > If you want to take the hackish road, you can just simply create a proxy > in your haproxy which does this, with an SSL server and a crt. This way > you can still use the httpclient or the socket API directly with this > proxy. > > -- > William Lallemand
Re: [PATCH 1/1]: MINOR __builtin_memcpy_inline usage introduction
On Mon, 20 Jun 2022 at 07:27, Willy Tarreau wrote:
>
> Hi David,
>
> On Sat, Jun 18, 2022 at 12:52:23PM +0100, David CARLIER wrote:
> > From 9d7b6448a2407451c3115b701c51f97ab2bf6a59 Mon Sep 17 00:00:00 2001
> > From: David Carlier
> > Date: Sat, 18 Jun 2022 12:41:11 +0100
> > Subject: [PATCH] MINOR: compiler __builtin_memcpy_inline usage introduction.
> >
> > Optimised version of the existing __builtin_memcpy builtin, differences
> > reside by the fact it works only with constant time sizes and does
> > generate extra calls.
> > At the moment, is clang exclusive even tough GCC does not seem to
> > want to implement it, the comments try not to reflect this current
> > state.
> > Usage can be expanded, used purposely only in few places for starter.
> > ---
> > include/haproxy/compiler.h | 12
> > src/haproxy.c | 4 ++--
> > src/log.c | 2 +-
> > src/proxy.c| 2 +-
> > 4 files changed, 16 insertions(+), 4 deletions(-)
> >
> > diff --git a/include/haproxy/compiler.h b/include/haproxy/compiler.h
> > index 66b5f5835..dca9f6aef 100644
> > --- a/include/haproxy/compiler.h
> > +++ b/include/haproxy/compiler.h
> > @@ -192,6 +192,18 @@
> > #endif
> > #endif
> >
> > +/*
> > + * This builtin works only with compile time lengths unlike
> > __builtin_memcpy.
> > + * Also guarantees no external call is generated.
> > + * We could `replicate` the aforementioned constraint with a
> > + * _Static_assert/__builtin_constant_p theoretically, but that might be
> > + * too much trouble to make it happens (C11 min)
> > + * so here we trust the proper usage with other compilers (and the CI
> > infrastructure).
> > + */
> > +#if !defined(__clang__) || __clang_major__ < 11
> > +#define __builtin_memcpy_inline(x, y, s) memcpy(x, y, s)
> > +#endif
> > +
>
> Hmmm please no, this is not the right way to do it for two reasons:
> - it will encourage use of __builtin_memcpy_inline() making one believe
> it's the expected one when it isn't ;
>
> - developing on non-clang systems will not report the problem with the
> non-constant size that __builtin_memcpy_inline() expects, resulting
> in breakage from time to time.
>
> I'd suggest that instead you create a macro named ha_memcpy_inline() that
> maps to __builtin_memcpy_inline() when present and s is a builtin constant,
> or maps to __builtin_memcpy() for the rest. Please note, by the way, that
> gcc since at least 3.4 has been emitting the same instructions (rep movsb)
> as clang's __builtin_memcpy_inline(), but only when optimizing for size.
> When optimizing for anything else, it can emit ton of inlined garbage^Wvector
> instructions.
>
> Thus I suspect we could achieve the same result by doing a clever
> arrangement of #pragma GCC push_options / #pragma GCC optimize("Os,inline")
> around an inline function definition that does the memcpy, and that is
> called by the macro. I have not tried, but probably something like this
> would do it:
>
> #pragma GCC push_options
> #pragma GCC optimize("Os,inline")
> static inline void _ha_memcpy_inline(void *restrict dest, const void
> *restrict src, size_t n)
> {
> __builtin_memcpy(dest, src, n);
> }
> #pragma GCC pop_options
>
> #if defined(clang...)
> #define ha_memcpy_inline(d,s,n) do { if (__builtin_constant_p(n))
> __builtin_memcpy_inline(d,s,n); else _ha_memcpy_inline(d,s,n); } while (0)
> #else
> #define ha_memcpy_inline(d,s,n) _ha_memcpy_inline(d,s,n)
> #endif
>
> That's not even compile-tested and I haven't looked at the result, but
> you get the idea.
Alright thanks here a newer version.
>
> Now regarding where to use it... I'd say it depends on a lot of factors,
> size, speed, undesired dependency on external libs. I think that each and
> every call place does warrant an individual commit with a justification
> and a check that the benefit matches expectations. There could be some
> value in using this in various low-level protocol layers (QUIC, HTX, SPOE).
Exactly.
> Maybe more, I don't know.
Over time its value will display more.
>
> Thanks,
> Willy
From 90c951d29c32a345cf38d6c0b43ee904f12b8ac7 Mon Sep 17 00:00:00 2001
From: David Carlier
Date: Sat, 18 Jun 2022 12:41:11 +0100
Subject: [PATCH] MINOR: compiler __builtin_memcpy_inline usage introduction.
Optimised version of the existing __builtin_memcpy builtin, differences
reside by the fact it works only with constant time sizes and does
not generate extra calls.
At the moment, is clang exclusive even tough GCC does not seem to
want to implement it, the comments try not to reflect this current
state.
Usage can be expanded, used purposely only in few places for starter.
---
include/haproxy/compiler.h | 32
src/haproxy.c | 4 ++--
src/log.c | 2 +-
src/proxy.c| 2 +-
4 files changed, 36 insertions(+), 4 deletions(-)
diff --git a/include/haproxy/compiler.h b/include/haproxy/compiler.h
index 66b5f5835..95
AW: rate-limiting and retry-after header
Hello Jérôme, On 2022-06-20 13:43, Jérôme Magnin wrote: You can with an http-after-response rule. See https://cbonte.github.io/haproxy-dconv/2.5/configuration.html#http-after-response Or you can use an http-request return rule instead of deny/deny_status to set the status code and header with the same rule. https://cbonte.github.io/haproxy-dconv/2.5/configuration.html#http-request%20return Thank you for your quick response. But how can I get/ calculate the value of the retry-after header? Cheers Corin
Re: HttpClient in Lua
On Mon, Jun 20, 2022 at 08:27:22PM +1000, Philip Young wrote: > Thanks for the answer William, it is very much appreciated. It is good to get > some clarification and will stop me continuing to spend time trying to get it > to work. > > In the meantime, I am working around the problem by calling out to a local > python service running on the same machine as HAProxy over http, which is > then making the authorisation request with a client certificate. Not ideal, > but will switch out the hack once it is supported in HAProxy. > > Thanks again > Phil > If you want to take the hackish road, you can just simply create a proxy in your haproxy which does this, with an SSL server and a crt. This way you can still use the httpclient or the socket API directly with this proxy. -- William Lallemand
Re: rate-limiting and retry-after header
Hello Corin, On 2022-06-20 13:18, Corin Langosch wrote: Hi guys, I'm using haproxy 2.5 and have some basic rate limiting configured like this (the actual configuration contains more rules for different urls): backend test acl rate_limit_by_ip_exceeds_limit src,table_http_req_rate(rate_limit_by_ip) gt 100 http-request deny deny_status 429 if rate_limit_by_ip_exceeds_limit http-request track-sc0 src table rate_limit_by_ip ... backend rate_limit_by_ip stick-table type ipv6 size 1m expire 24h store http_req_rate(5m) Is there any way to include a "retry-after" header in case the rate limit is exceeded? You can with an http-after-response rule. See https://cbonte.github.io/haproxy-dconv/2.5/configuration.html#http-after-response Or you can use an http-request return rule instead of deny/deny_status to set the status code and header with the same rule. https://cbonte.github.io/haproxy-dconv/2.5/configuration.html#http-request%20return -- Jérôme
rate-limiting and retry-after header
Hi guys, I'm using haproxy 2.5 and have some basic rate limiting configured like this (the actual configuration contains more rules for different urls): backend test acl rate_limit_by_ip_exceeds_limit src,table_http_req_rate(rate_limit_by_ip) gt 100 http-request deny deny_status 429 if rate_limit_by_ip_exceeds_limit http-request track-sc0 src table rate_limit_by_ip ... backend rate_limit_by_ip stick-table type ipv6 size 1m expire 24h store http_req_rate(5m) Is there any way to include a "retry-after" header in case the rate limit is exceeded? Cheers Corin
Re: HttpClient in Lua
Thanks for the answer William, it is very much appreciated. It is good to get some clarification and will stop me continuing to spend time trying to get it to work. In the meantime, I am working around the problem by calling out to a local python service running on the same machine as HAProxy over http, which is then making the authorisation request with a client certificate. Not ideal, but will switch out the hack once it is supported in HAProxy. Thanks again Phil > On 20 Jun 2022, at 7:38 pm, William Lallemand wrote: > > On Wed, Jun 15, 2022 at 11:33:27PM +1000, Philip Young wrote: >> Hi >> I am currently writing a LUA module to make authorisation decisions on >> whether a request is allowed, by calling out to another service to make the >> authorisation decision. >> In the Lua module, I am using Socket.connect_ssl() to connect to the >> authorisation service but I am struggling to work out how to set the path to >> the certificate I want to use to connect to the authorisation service. >> Does anybody know how to set the path to the certificate that is used when >> using Socket.connect_ssl() >> Is it possible to do this using the httpclient? >> I have tried asking the Slack chat channel and on the commercial site but no >> one knows. >> >> Cheers Phil > > Hello Phil, > > Unfortunately the Socket and the HTTPClient lua class are not able to > use a client certificate right now. This should evolve in the future but > the current architecture is not able to do it. > > -- > William Lallemand
Re: HttpClient in Lua
On Wed, Jun 15, 2022 at 11:33:27PM +1000, Philip Young wrote: > Hi > I am currently writing a LUA module to make authorisation decisions on > whether a request is allowed, by calling out to another service to make the > authorisation decision. > In the Lua module, I am using Socket.connect_ssl() to connect to the > authorisation service but I am struggling to work out how to set the path to > the certificate I want to use to connect to the authorisation service. > Does anybody know how to set the path to the certificate that is used when > using Socket.connect_ssl() > Is it possible to do this using the httpclient? > I have tried asking the Slack chat channel and on the commercial site but no > one knows. > > Cheers Phil Hello Phil, Unfortunately the Socket and the HTTPClient lua class are not able to use a client certificate right now. This should evolve in the future but the current architecture is not able to do it. -- William Lallemand
Re: To upgrade from 2.5 to 2.6 on ubuntu
On Fri, Jun 17, 2022 at 10:55:19PM +, Henning Svane wrote: > Hi > To upgrade to HAproxy 2.6 will I have to add this repository? > > sudo add-apt-repository ppa:vbernat/haproxy-2.6 > Or is there another way I should use? > > I use Ubuntu 20.04 and would also like to upgrade to 22.04. Any know problems > with 22.04? > > Regards > Henning > Probably the best way to do it indeed. I don't know any problem regarding 22.04 at the moment, there are some changes with the algorithms used since the migration to openssl 3.0, if you need old algorithms you might want to reenable them. -- William Lallemand
