OCSP renewal with 2.8
Dear all, I just saw in the release notes for 2.8 that an automatic OCSP renewal is now included and I would like to get rid of my manual scripts that are currently injecting the OCSP information. I checked a little bit the documentation here: https://docs.haproxy.org/2.8/configuration.html#ocsp-update https://docs.haproxy.org/2.8/configuration.html#5.1-crt-list and if I understood it correctly it only works if used together with a crt-list line. I currently use the crt definition on a bind line like: frontend www-https mode tcp option tcplog bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/certs/ alpn h2,http/1.1 bind :::443 ssl crt /usr/local/etc/haproxy/certs/ alpn h2,http/1.1 Could you please help me, how I need to configure haproxy to use ocsp renewal. It is not my intent to list all certificates in the haproxy configuration as that would make it unnecessarily complicated. Thanks a lot. Gruß Matthias -- "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook
Re: [ANNOUNCE] haproxy-2.8.0
On Wed, May 31, 2023 at 04:54:57PM +, Tristan wrote: > Congratulations to the team at large for the release! > There's definitely been more improvements and fixes than meets the eye from > the release notes alone! > > On 31/05/2023 16:14, Willy Tarreau wrote: > > > - QUIC: it has been running almost flawlessly for a year on haproxy.org, > >and totally flawlessly over the last 6 months [...] >we now consider > > it production-ready [...] > All I can say is that this was a long time coming since the early days of > 2.6.0, and that the amount of bug fixes and improvements Amaury and Fred > provided to make this happen has been nothing short of amazing. > > A big thank you to them for their patience with my reports and not giving up > on even the most obscure issues! > > As a parting gift, I don't have a CSS nit to report like Tim, but I do have > this prophetic comment from Willy almost a year ago on the matter: > https://github.com/haproxy/haproxy/issues/1808#issuecomment-1230660870 :) Hehe indeed. That was not prophetic, that's experience. Over time you learn to detest what you did in the past :-) Willy
Re: [ANNOUNCE] haproxy-2.8.0
Congratulations to the team at large for the release! There's definitely been more improvements and fixes than meets the eye from the release notes alone! On 31/05/2023 16:14, Willy Tarreau wrote: - QUIC: it has been running almost flawlessly for a year on haproxy.org, and totally flawlessly over the last 6 months [...] >we now consider it production-ready [...] All I can say is that this was a long time coming since the early days of 2.6.0, and that the amount of bug fixes and improvements Amaury and Fred provided to make this happen has been nothing short of amazing. A big thank you to them for their patience with my reports and not giving up on even the most obscure issues! As a parting gift, I don't have a CSS nit to report like Tim, but I do have this prophetic comment from Willy almost a year ago on the matter: https://github.com/haproxy/haproxy/issues/1808#issuecomment-1230660870 :) Tristan
Re: [ANNOUNCE] haproxy-2.8.0
On Wed, May 31, 2023 at 06:10:37PM +0200, Tim Düsterhus wrote: > Willy, > > On 5/31/23 17:14, Willy Tarreau wrote: > > HAProxy 2.8.0 was released on 2023/05/31. It added 27 new commits > > after version 2.8-dev13. > > Congratulations! Enjoy the release party :-) Thanks ;-) > Best regards > Tim Düsterhus > > PS: Wouldn't be a "Tim email" without some minor nit. Yeah, after sending the message I said to the coworkers here "now let's wait for Tim's message ;-)" > Just FYI: I made a > small adjustment to the docs repository: > https://github.com/haproxy/docs/commit/7797f2d18aa1beafd73f9cec2a9222cff1b7cd1c Ah, nice catch, thank you! I'm never much at ease when I touch these, though they're not too difficult to deal with when you compare them. Cheers, Willy
Re: [ANNOUNCE] haproxy-2.8.0
Willy, On 5/31/23 17:14, Willy Tarreau wrote: HAProxy 2.8.0 was released on 2023/05/31. It added 27 new commits after version 2.8-dev13. Congratulations! Enjoy the release party :-) Best regards Tim Düsterhus PS: Wouldn't be a "Tim email" without some minor nit. Just FYI: I made a small adjustment to the docs repository: https://github.com/haproxy/docs/commit/7797f2d18aa1beafd73f9cec2a9222cff1b7cd1c
[ANNOUNCE] haproxy-2.8.0
Hi, HAProxy 2.8.0 was released on 2023/05/31. It added 27 new commits after version 2.8-dev13. Only a small minor issues were addressed this time, the rest was mostly doc polishing and cleanups. 2.8 is entering LTS status and will be supported till 2028-Q2, and 2.9-dev0 was just created to pursue the development, with an expected release around end of November this year. Let's try to summarize the changes from 37 participants in the 1382 commits that were merged since 2.7.0 from a high level perspective: - Lua/Mailers: there's now a full-Lua implementations of the mailers subsystem. It's provided as a Lua script (examples/lua/mailers.lua) which relies on the new internal event notification API. As such the script subscribes to server state change events and emits mails when the defined criteria are matched. It continues to rely on the "mailers" section, but being a Lua script, it's totally customizable. You can imagine to change the contents, change the notification conditions, send to multiple destinations etc. With this change, the internal Lua view of the servers was made fully dynamic so that added or removed servers are always seen in their current state. In fact the new event notification API goes way beyond this but better read the Lua API documentation to know more. The next step will be to completely deprecate the old Mailers subsystem in 2.9 and 3.0 and to remove it in 3.1. - HTTP/2 is advertised by default in ALPN on TLS listeners. It was about time, 5 years have passed since it was introduced, it's been enabled by default in clear text as an HTTP/1 upgrade for 4 years, yet some users do not know how to enable it. From now on, ALPN defaults to "h2,http/1.1" on TCP and "h3" on QUIC so that these protocol versions work by default. It's still possible to set/reset the ALPN to disable them of course. The old concern some users were having about window sizes was addressed by having a setting for each side (front vs back). - Threading: thread groups are now usable by default by "bind" lines without requiring to replicate these lines once per thread group. This means that by default a bind line is bound to all threads, regardless of the number of groups (up to 64 groups of 64 threads or 4096 threads total). As such it becomes possible to enable multiple groups on a large system to benefit from all the processing power available if you're running heavy rules, Lua, compression, SSL or whatever. We still default to a single NUMA node because the cases where it brings solid benefits are not frequent enough, compared to the cost of having more listening sockets. Note that on systems with non-uniform L3 caches like AMD EPYC, this can bring important performance gains with only one setting in the config. We noticed a doubling of the request rate on a 24-core EPYC 74F3 by enabling 8 groups instead of the default 1, to map to the L3 cache topology. The maximum tested so far was 224 threads with 4 & 8 groups on a dual-socket intel Sapphire Rapids system. That was blazingly fast :-) - SSL: there are quite a bunch of updates on the SSL front in this release: - it's possible to adjust the signature algorithms to improve interoperability with some other TLSv1.2/1.3 clients. These algorithms are used to sign the ephemeral keys used during the handshake. Changing these algorithms are useful for buggy clients that negociate algorithms they don't support. Though the usage is very specific. It's also possible to adjust this parameter for Client Authentication. - SSL hanshake failure logs now dump the OpenSSL error string by default. No need to configure an error-log-format anymore to show details on the handshake error. It can be helpful to debug SSL problems (e.g. you'll now see "tlsv1 alert unknown ca" instead of just "SSL handshake failure"). - OCSP: in 2.8 the OCSP responses for certificates can be automatically updated by a background task (by default every 5 minutes) so that it is no longer necessary to feed them over the CLI from an external script. Of course, this requires that your load balancers have outgoing HTTP access. This is enabled in crt-list files by adding "ocsp-update on" on the certificate's line. All this is observable on the CLI via "show ssl-ocsp-update" and "show ssl-ocsp-response". - LetsEncrypt: there's an acme.sh script in admin/acme.sh that can be used with your existing deployments (pull request for upstream still pending). It will permit to handle the renewal of LE certificates in stateless mode with no hassle (no need to proxy to a local port anymore). - OpenSSL: version 3.1 is now supported. It's less slow than 3.0 but still significantly slower than 1.1.1, but might be usable for most users with a low enough traffic. - wolfSSL: we've worked quite a bit with the wolfSSL team to make sure their latest version
Re: haproxy -dKcnv output
Is it? In all the programming languages I use, the colon is followed by
the return type (which for iif is str).
my claim of mainstream-ness, was mainly meaning the ": in => out" order
(one example would be most ML languages, Typescript, Java...) as opposed
to ": out <= in" which I haven't seen being used so far
Of course the '=>' also implies
return type and now I'm seeing two return types there.
[...]
I think it's pretty clear if you look at the usage in a config. Example
from my production config:
ssl_fc,iif(https,http)
That works like the pipeline operator in shell or various programming
languages. Effectively the 'ssl_fc' result acts as the first parameter,
it's just written a little differently.
I think we're just seeing it in a different way
I see converters (and shell pipe operations for that matter) as
functions being chained, so their individual types must be those of a
function (ie: in => out) and cannot be "str", since that is the type of
a value
Running with this on the example of shell pipes, where the fact that
things usually also support these forms:
tool [...args] file
tool [...args] -
is interpretable as a convenience parenthesis-less form of:
(tool [...args)(file)
(tool [...args])(-)
And the left-side portion of the pipe (not sure if that has a precise
name?) is not an implicit first argument to the right side, but rather
the only argument of a nameless function defined on the right side
In that view then
> var x = iif("foo","bar")
here x is a a function, of type bool => str
> var y = x(true)
and y is the result of giving a bool to a bool => str, and thus an str
> ssl_fc,iif(https,http)
is then pretty much akin to iif(https,http)(ssl_fc)
and that is why iif(str,str) being "bool => str" seems sensible to me
Now one can be pedantic and then argue to write it like this:
iif: (str,str) => (bool => str)
which is what many ML languages go with, and has some elegance, but is
terrible for readability and that I wouldn't defend
iif(str,str): bool => str
hides the type of the "important" parameter in the middle (and has the
colon = return type problem).
That's a fair point admittedly; maybe I've just grown too used to that
downside
iif(str,str): str <= bool
at least has the parameter on an outer position, but the order is still
reversed from the config.
fwiw while I said it seems to be an unusual notation order, it does have
the benefit of being clear, indeed
iif(bool,str,str): str
is reasonably similar to the config, but not identical
The similarity is specifically what I dislike about it, as it does seem
a lot like this
http-request set-var(txn.proto) iif(ssl_fc,https,http)
should be a valid config line
or at least I think that wouldn't be entirely unreasonable to think that
at first glance
bool,iif(str,str): str
which I didn't mention before might possibly be the best choice?
I do quite like that
Best regards, and thanks for the bike-shedding exercise :D
Tristan
Re: haproxy -dKcnv output
Tristan, On 5/31/23 12:28, Tristan wrote: If fetches already have the output type after the colon, then the converter should not have the input type after the colon, i.e. iif(str,str): bool => str is confusing, because it looks like it returns a bool, ... I guess? While this is mainly a feelings thing, I'd say that it is more widespread (if not nowadays near-mainstream? not that this is necessarily a good argument of course) to write it that way Is it? In all the programming languages I use, the colon is followed by the return type (which for iif is str). Of course the '=>' also implies return type and now I'm seeing two return types there. Another option might be listing the "implicit" parameter as a real parameter: iif(bool,str,str): str This one feels quite unnatural to me; unless you're familiar with the underlying implementation of many OOP systems and see this as some kind of extension method definition, the fact that the first parameter is implicitly passed seems (to me) like it's coming out of nowhere I think it's pretty clear if you look at the usage in a config. Example from my production config: ssl_fc,iif(https,http) That works like the pipeline operator in shell or various programming languages. Effectively the 'ssl_fc' result acts as the first parameter, it's just written a little differently. I also think it's the most important parameter, because it represents the actual data that the converter works with. iif(str,str): bool => str hides the type of the "important" parameter in the middle (and has the colon = return type problem). iif(str,str): str <= bool at least has the parameter on an outer position, but the order is still reversed from the config. iif(bool,str,str): str is reasonably similar to the config, but not identical bool,iif(str,str): str which I didn't mention before might possibly be the best choice? Best regards Tim Düsterhus
Re: haproxy -dKcnv output
If fetches already have the output type after the colon, then the converter should not have the input type after the colon, i.e. iif(str,str): bool => str is confusing, because it looks like it returns a bool, ... I guess? While this is mainly a feelings thing, I'd say that it is more widespread (if not nowadays near-mainstream? not that this is necessarily a good argument of course) to write it that way iif(str,str): str <= bool Though I guess this one is clear enough, if unusual (to me anyway) Another option might be listing the "implicit" parameter as a real parameter: iif(bool,str,str): str This one feels quite unnatural to me; unless you're familiar with the underlying implementation of many OOP systems and see this as some kind of extension method definition, the fact that the first parameter is implicitly passed seems (to me) like it's coming out of nowhere Tristan
Re: haproxy -dKcnv output
Hi all, On Wed, May 31, 2023 at 10:02:45AM +0200, Tim Düsterhus wrote: > Aurelien, > > On 5/31/23 09:57, Aurelien DARRAGON wrote: > > would not fit properly with existing representation for converters > > within the doc > > > > > iif(str,str): str <= bool > > > > and > > > > > iif(str,str): bool => str > > > > could be good candidates (fetches are already represented using > > "name(arg) : out"), although dconv would have to be patched anyway for > > If fetches already have the output type after the colon, then the converter > should not have the input type after the colon, i.e. > > iif(str,str): bool => str > > is confusing, because it looks like it returns a bool, ... I guess? > > iif(str,str): str <= bool > > is better, because the colon is followed by the output type, but the order > of "reading" is a little funky with the actual input on the very right side. > > Another option might be listing the "implicit" parameter as a real > parameter: > > iif(bool,str,str): str Alright, at least it means we have something non-trivial to improve here, so better not change it for 2.8 so that we have plenty of time to rethink all this (including the way we present them in the doc) past the release. Thank you, that's exactly what I needed ;-) Willy
Re: haproxy -dKcnv output
Aurelien, On 5/31/23 09:57, Aurelien DARRAGON wrote: would not fit properly with existing representation for converters within the doc iif(str,str): str <= bool and iif(str,str): bool => str could be good candidates (fetches are already represented using "name(arg) : out"), although dconv would have to be patched anyway for If fetches already have the output type after the colon, then the converter should not have the input type after the colon, i.e. iif(str,str): bool => str is confusing, because it looks like it returns a bool, ... I guess? iif(str,str): str <= bool is better, because the colon is followed by the output type, but the order of "reading" is a little funky with the actual input on the very right side. Another option might be listing the "implicit" parameter as a real parameter: iif(bool,str,str): str Best regards Tim Düsterhus
Re: haproxy -dKcnv output
> What I would find clear: > > bool => iif(str,str) => str You're right Tim But in the long term it could be great to share a common output format with the doc as well (to find all relevant info from -dKcnv in the doc, and vice versa) While > bool => iif(str,str) => str and > bool | iif(str,str) => str would not fit properly with existing representation for converters within the doc > iif(str,str): str <= bool and > iif(str,str): bool => str could be good candidates (fetches are already represented using "name(arg) : out"), although dconv would have to be patched anyway for this to work since it does not allow '=' nor '>' or '<' characters after the ':' because of this regexp: https://github.com/cbonte/haproxy-dconv/blame/4decf0b72eff0888195c2353ea149cd969110fe5/parser/keyword.py#L31
Re: haproxy -dKcnv output
Hi On 5/30/23 22:09, Aurelien DARRAGON wrote: $> haproxy -f test.conf -dKcnv | grep nbsrv iif(string,string): str => bool iif(string,string): bool => str I don't rely on it, but frankly I find both variants confusing, because it does not follow the logical processing order. What I would find clear: bool => iif(str,str) => str or bool | iif(str,str) => str or perhaps iif(str,str): str <= bool Best regards Tim Düsterhus
