[ANNOUNCE] haproxy-2.2-dev2
Hi, HAProxy 2.2-dev2 was released on 2020/02/07. It added 115 new commits after version 2.2-dev1. Over the last two weeks, 36 bugs and a few build warnings were addressed. On the features front, I mainly noted these : - CLI: Adis added support for multiple filters on the "show table" output. I haven't tried it yet but it looks promising as I guess I'll be able to simplify a few anti-abuse scripts here and there that heavily rely on grep. - connection: the ongoing code cleanup work was continued. It should normally be harmless (famous last words). Once this is complete, I intend to write some doc about it so that we know this tricky area better in the future. - HTTP: Christopher implemented about all the things we've been debating over the last 5 years around the "return" directive. Now we can forge a response at any instant based on a status code, an error file, a body from an argument string, a body from a log-format argument, a body from a file, a body processed from a log-format stored in a file (effectively making it a response template), a set of headers, and I think that's about all. We may have to remind people more often that haproxy is not a file server, but at least it will be extremely convenient for some to be able to return rich reject pages or tailored sets of headers for rate shaping based on 503+retry for example. This should allow some of us to finally remove the dirty hacks consisting in using an backend with a specific 503 error file, or something similar based on a deny rule, just for the sake of delivering a robots, favicon, or any ".well-known" content. Responses remain limited to the size of a buffer, and I'm not willing to see this change for now. - HTTP: Christopher also added a new "http-after-response" ruleset to manipulate headers after the final response is sent. This is mainly used to append headers after redirects or haproxy's error responses. The main use case definitely concerns HSTS, but given that all regular actions were implemented, one could also think about using this to delete some Server headers for example. - Lua: it is now possible to directly build a response to be injected from an HTTP action by passing a reply object to txn:done(). In the past it used to be only possible from services. This means that some new HTTP actions could first be implemented in Lua for the time it takes to get a broad consensus on them, before doing them natively. - SSL: William brought significant startup time savings when using large amounts of certificates thanks to a new option indicating what extensions to look for. By default, for backwards compatibility we look for ".ocsp", ".sctl", ".issuer" and all cert types extensions. But when you know exactly what you're using and know it's pointless to check for the ones above, you can now explicitly tell haproxy not to look them up, and all these extra syscalls start to account for real when you have 100k certs. - Lua: Tim added options to prepend the lookup path for Lua modules. - a bunch of dead code cleanups and/or minor fixes by Ilya and William Dauchy (I noticed a few other ones arrived since the release). - splicing: a thread-local pool of recently used pipes was added to improve cache locality and eliminate locking on allocation, resulting in ~5-6% performance increase on spliced traffic. - scheduler: the scheduler now becomes latency aware. I was particularly irritated by seeing some pathological cases in which a "show info" on the CLI could take tens of seconds on a machine saturated under high traffic rates just because I/O tasks requeue themselves and find some new data available. Now we have 3 latency classes and tasks are placed there based on their behavior. The result is that now even on a machine saturating 16 threads at 100% forwarding 90 Gbps of traffic, the CLI responds in 70ms and not one minute anymore. And the small objects now experience a much lower latency on mixed traffic. Last, a few potentially user-visible changes: - you'll now get an error if an ACL is called "or" since you'll never be able to match it, as the "or" word is taken by the expression parser. It was backported but will only warn in stable versions. - too large error files that fill the rewrite reserved area will warn you at load time, as they are potentially incompatible with http-after-reponse rules. - the number of connections reported in the logs output of a quitting proxy now clearly indicates that the value is the cumulated conns and not the active conns. I wouldn't be surprised to learn that some people parse this output and got it wrong as I did a few times :-) This branch is starting to be interesting, I'll deploy it on haproxy.org and see if it allows me to simplify some of the
unsubscribe
unsubscribe haproxy -- Regards, Yi Yang @ Beijing, China
[PATCH] BUG/MINOR: acl: Fix type of log message when an acl is named 'or'
The patch adding this check initially only issued a warning, instead of
being fatal. It was changed before committing. However when making this
change the type of the log message was not changed from `ha_warning` to
`ha-alert`. This patch makes this forgotten adjustment.
see 0cf811a5f941261176b67046dbc542d0479ff4a7
No backport needed. The initial patch was backported as a warning, thus
the log message type is correct.
---
src/cfgparse-listen.c | 2 +-
src/fcgi-app.c| 2 +-
src/flt_spoe.c| 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/cfgparse-listen.c b/src/cfgparse-listen.c
index 77a5d5b8f..6d9c1f52e 100644
--- a/src/cfgparse-listen.c
+++ b/src/cfgparse-listen.c
@@ -808,7 +808,7 @@ int cfg_parse_listen(const char *file, int linenum, char
**args, int kwm)
}
if (strcasecmp(args[1], "or") == 0) {
- ha_warning("parsing [%s:%d] : acl name '%s' will never
match. 'or' is used to express a "
+ ha_alert("parsing [%s:%d] : acl name '%s' will never
match. 'or' is used to express a "
"logical disjunction within a condition.\n",
file, linenum, args[1]);
err_code |= ERR_ALERT | ERR_FATAL;
diff --git a/src/fcgi-app.c b/src/fcgi-app.c
index 7b28e3a9c..a9466a577 100644
--- a/src/fcgi-app.c
+++ b/src/fcgi-app.c
@@ -888,7 +888,7 @@ static int cfg_parse_fcgi_app(const char *file, int
linenum, char **args, int kw
goto out;
}
if (strcasecmp(args[1], "or") == 0) {
- ha_warning("parsing [%s:%d] : acl name '%s' will never
match. 'or' is used to express a "
+ ha_alert("parsing [%s:%d] : acl name '%s' will never
match. 'or' is used to express a "
"logical disjunction within a condition.\n",
file, linenum, args[1]);
err_code |= ERR_ALERT | ERR_FATAL;
diff --git a/src/flt_spoe.c b/src/flt_spoe.c
index 06c70d24f..4e448dd11 100644
--- a/src/flt_spoe.c
+++ b/src/flt_spoe.c
@@ -3992,7 +3992,7 @@ cfg_parse_spoe_message(const char *file, int linenum,
char **args, int kwm)
goto out;
}
if (strcasecmp(args[1], "or") == 0) {
- ha_warning("parsing [%s:%d] : acl name '%s' will never
match. 'or' is used to express a "
+ ha_alert("parsing [%s:%d] : acl name '%s' will never
match. 'or' is used to express a "
"logical disjunction within a condition.\n",
file, linenum, args[1]);
err_code |= ERR_ALERT | ERR_FATAL;
--
2.25.0
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
Willy, Am 06.02.20 um 16:37 schrieb Willy Tarreau: > Maybe you didn't reload the page or ended on an older tab ? > Apparently it was my browser cache. Hitting reload several times made it show up. All good now! Best regards Tim Düsterhus
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
On Thu, Feb 06, 2020 at 04:31:36PM +0100, Tim Düsterhus wrote: > The committed patch is looking good. I'm not seeing anything in 2.1 > though, did you forget to push that branch? No here it's pushed: commit 9007a8d65d56e670aad97f60b66c36a52aab22c5 (HEAD -> master, flx04/master) Author: Tim Duesterhus Date: Wed Feb 5 21:00:50 2020 +0100 MINOR: acl: Warn when an ACL is named 'or' (flx04 is the remote's name). And I'm seeing it at the top here: http://git.haproxy.org/?p=haproxy-2.1.git;a=summary Maybe you didn't reload the page or ended on an older tab ? Willy
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
Willy, Am 06.02.20 um 16:19 schrieb Willy Tarreau: > On Thu, Feb 06, 2020 at 03:42:17PM +0100, Willy Tarreau wrote: >>> Feel free to either modify the patch yourself to make the adjustments >>> you want to see or let me know if you want me to make them. It might >>> require a bit more back and forth, though :-) >> >> OK I guess I can do them, sure. > > I did what I mentioned and backported it as far as 1.8 (with warnings > for stable releases). The committed patch is looking good. I'm not seeing anything in 2.1 though, did you forget to push that branch? Best regards Tim Düsterhus
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
On Thu, Feb 06, 2020 at 03:42:17PM +0100, Willy Tarreau wrote: > > Feel free to either modify the patch yourself to make the adjustments > > you want to see or let me know if you want me to make them. It might > > require a bit more back and forth, though :-) > > OK I guess I can do them, sure. I did what I mentioned and backported it as far as 1.8 (with warnings for stable releases). Thanks, Willy
Re: [PATCH] AIX 7.2 support
On Thu, Feb 06, 2020 at 03:36:06PM +0100, Willy Tarreau wrote: > Just two small extra requests : > - please rebase it on top of the development branch. If you really > need it in 2.1, just indicate it. Given that it's well isolated, > I'm fine with having it backported. > > - please have a look at CONTRIBUTING to get guidance to write a > subject and a commit message. It doesn't need to be very long > but at least indicating what options you chosed to enable/disable > and on what system you tested it will be enough if we need to > reconsider parts of it later. And a last one :-) Do you have a permanent and durable access to this machine, with the ability to occasionally re-run a build test in case we ask you (likely no more than 2-3 times a year in the worst case) ? I'm asking because I'm still keeping a very old IBM server running 5.2 on a Power3 just for the sake of revalidating new releases once in a while. Given that I could not upgrade it to latest OpenSSL, it cannot even be used to provide complete binaries to those needing them, so it's getting very obsolete and knowing that there's a better solution somewhere would allow me to get rid of it. Thanks, Willy
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
On Thu, Feb 06, 2020 at 03:33:45PM +0100, Tim Düsterhus wrote: > Willy, > > Am 06.02.20 um 15:24 schrieb Willy Tarreau: > > I was surprized because I discovered that we've always accepted empty > > expressions thus"if" without even a condition or "if or" are valid, which > > adds to the confusion. So your patch does indeed make sense. However we > > don't need to test for "||" because it's already invalid as an ACL name > > and the existing message is more precise about it (invalid chars in the > > name). > > I added the test for '||' for completeness in case there are adjustments > to valid characters later on. I am aware that it will never trigger and > being only run on configuration parsing the performance impact should be > a non-issue. It's not a matter of performance impact but that the message is misleading in this case. It says that "||" is forbidden as an ACL name, which sort of implies a lot of other ones are valid. It's not "||" that's forbidden, it's anything not made exclusively of "[0-9a-zA-Z_.:-]", and actually your message scared me and made me recheck the doc to be sure. > > Also I'd rather have the "goto out" statement after setting the error, > > otherwise it becomes confusing and will certainly trap someone adding > > yet another test later. > > So you want to make using 'or' as an ACL name fatal instead of a warning? Ah silly me! I misread it, I read only the first one as a warning and the second one as an error. However you're making me think now :-) We should likely make it an error in 2.2 and a warning in existing branches. We know that people don't read warnings, so if we keep it forever, such a config might last. But we must not break an existing config in a stable branch otherwise users roll back by lack of time to fix the conf. > Feel free to either modify the patch yourself to make the adjustments > you want to see or let me know if you want me to make them. It might > require a bit more back and forth, though :-) OK I guess I can do them, sure. Thanks! Willy
Re: [PATCH] AIX 7.2 support
Hello Christian,
On Mon, Feb 03, 2020 at 12:09:46PM +0100, Chris wrote:
> Hello everybody,
>
> I spent some time making haproxy compile and run successfully on AIX
> 7.2 using GCC 8.3 and wanted to contribute my patch in the hope that
> it could be merged. The patch is based on the current haproxy 2.1 head
> revision. I can make one for the development branch too - but it
> should be basically identical.
Thank you for doing this work. I'm having a few questions below.
> +# AIX 7.2 and above
> +ifeq ($(TARGET),aix72-gcc)
> + set_target_defaults = $(call default_opts, \
> +USE_POLL USE_THREAD USE_LIBCRYPT USE_OBSOLETE_LINKER
> USE_GETADDRINFO USE_TFO)
Are you really really sure about USE_TFO ? I suspect you might have
accidently borrowed it from another line. It stands for TCP FastOpen
and is really not broadly deployed, I was only aware of Linux and
FreeBSD, but there are likely a few other ones. And the fact that it
looks OK is possibly just that it's causing a different TCP socket
option to be set on the connection, so if you're not certain we'd
rather avoid it.
> The patch implements a new TARGET called aix72-gcc and also adds 2
> CPUs (power8 and power9). Here is my proof-of-work:
(...)
> src/hlua.c: In function 'hlua_panic_ljmp':
> src/hlua.c:128:1: warning: no return statement in function returning
> non-void [-Wreturn-type]
> static int hlua_panic_ljmp(lua_State *L) { longjmp(safe_ljmp_env, 1); }
> ^~
Don't worry about this one, I'll handle it. I suspect that on linux
platforms the longjmp() function prototype is decorated with
__attribute__((noreturn)) which makes the compiler happy, but that's
likely not the case on any system not relying on a gcc-compatible
compiler by default.
> -bash-4.4$ ./haproxy -vv
> HA-Proxy version 2.1.2 2019/12/21 - https://haproxy.org/
> Status: stable branch - will stop receiving fixes around Q1 2021.
(...)
Looks good!
> If you have any questions feel free to ask and please keep me on CC
> for this topic!
Just two small extra requests :
- please rebase it on top of the development branch. If you really
need it in 2.1, just indicate it. Given that it's well isolated,
I'm fine with having it backported.
- please have a look at CONTRIBUTING to get guidance to write a
subject and a commit message. It doesn't need to be very long
but at least indicating what options you chosed to enable/disable
and on what system you tested it will be enough if we need to
reconsider parts of it later.
thanks!
Willy
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
Willy, Am 06.02.20 um 15:24 schrieb Willy Tarreau: > I was surprized because I discovered that we've always accepted empty > expressions thus"if" without even a condition or "if or" are valid, which > adds to the confusion. So your patch does indeed make sense. However we > don't need to test for "||" because it's already invalid as an ACL name > and the existing message is more precise about it (invalid chars in the > name). I added the test for '||' for completeness in case there are adjustments to valid characters later on. I am aware that it will never trigger and being only run on configuration parsing the performance impact should be a non-issue. > Also I'd rather have the "goto out" statement after setting the error, > otherwise it becomes confusing and will certainly trap someone adding > yet another test later. So you want to make using 'or' as an ACL name fatal instead of a warning? Feel free to either modify the patch yourself to make the adjustments you want to see or let me know if you want me to make them. It might require a bit more back and forth, though :-) Best regards Tim Düsterhus
Re: [PATCH] MINOR: acl: Warn when an ACL is named 'or' or '||'
Hi Tim, On Wed, Feb 05, 2020 at 09:00:50PM +0100, Tim Duesterhus wrote: (...) > Consider a configuration like this: > > > acl t always_true > > acl or always_false > > > > http-response set-header Foo Bar if t or t > > The 'or' within the condition will be treated as a logical disjunction > and the header will be set, despite the ACL 'or' being falsy. I was surprized because I discovered that we've always accepted empty expressions thus"if" without even a condition or "if or" are valid, which adds to the confusion. So your patch does indeed make sense. However we don't need to test for "||" because it's already invalid as an ACL name and the existing message is more precise about it (invalid chars in the name). Also I'd rather have the "goto out" statement after setting the error, otherwise it becomes confusing and will certainly trap someone adding yet another test later. > May be backported to older branches, it should not break anything > and might improve the users' lifes. I also think we should, indeed. Willy
Improve your website traffic - Haproxy.com
Dear Owner, Wish you a very Happy New Year 2020 I was on your Haproxy.com. What I think, there are a few things you could implement pretty quickly that would help boost your Google rankings, natural traffic, conversion. Major part Enhancing social presence Enhance Facebook likes, Speed up twitter followers, YouTube viewers etc. Is this of interest? Kiley Pearson. (301)489-2269
HAProxy Sticky Sessions Within Request Body / Packet
Hi, Is it possible to decode a guid or session identifier that exists within a request packet's body?For example, if I would like to get the sessionID out of the body below? 12345 Kind Regards, Rohan
Re: Configuring HAProxy
Hi. On 06.02.20 07:08, Akshay Mangla wrote: Hi HAProxy Team, I have been trying to install HAProxy on my vm machine and facing some difficulties in doing so. Following is the HAProxy config file that we have currently. #- # Example configuration for a possible web application. See the # full configuration options online. # # http://haproxy.1wt.eu/download/1.4/doc/configuration.txt # #- #- # Global settings #- global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the '-r' option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log 127.0.0.1 local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon # turn on stats unix socket stats socket /var/lib/haproxy/stats #- # common defaults that all the 'listen' and 'backend' sections will # use if not designated in their block #- defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 3000 #- # main frontend which proxys to the backends #- frontend main *:5000 acl url_static path_beg -i /static /images /javascript /stylesheets acl url_static path_end -i .jpg .gif .png .css .js use_backend static if url_static default_backend app #- # static backend for serving up images, stylesheets and such #- backend static balance roundrobin server static 127.0.0.1:4331 check #- # round robin balancing between the various backends #- backend app balance roundrobin server app1 127.0.0.1:5001 check server app2 127.0.0.1:5002 check server app3 127.0.0.1:5003 check server app4 127.0.0.1:5004 check frontend haproxy_inbound bind *:7068 default_backend haproxy_httpd backend haproxy_httpd balance roundrobin mode http #(NOT NEEDED IF DEFINED IN DEFAULTS) option httpchk server lxapp14058.dc.corp.telstra.com 10.195.77.21:7068 check server lxapp14059.dc.corp.telstra.com 10.195.77.22:7068 check I have added the lines at the end which are colored and ran the command ---> */haproxy -c -f /etc/haproxy/haproxy.cfg/* which gave me an output that /*configuration file is valid*/. When i tried to start it manually (in foreground, to test) with ---> */haproxy -db -f /etc/haproxy/haproxy.cfg/* it started giving me an error image.png I love screenshots, it's so easy to copy some text out of them ;-). My suggestion would be to copy the text from the console to the mail instead the screenshot. Can you help me resolve this issue as I am stuck on this. Any suggestions would be appreciated. I would assume that the backend is not a http backend as the httpchk fails. What do you get when you execute the follwoing command from haproxy maschine? curl -v --max-time 30 127.0.0.1:5001 curl -v --max-time 30 http://10.195.77.21:7068 curl -v --max-time 30 http://10.195.77.22:7068 Do let me know if you need any further information on this. Which haproxy version do you use? haproxy -vv Regards, Akshay Regards Aleks
