Re: Lua: forcing garbage collector after socket i/o
On Wed, Jan 22, 2020 at 05:54:29PM -0600, Dave Chiluk wrote: > We are running this patch on top of 1.9.13 where it is needed. I will > report back if/when we have anything to add. Untill then consider no > news as good news in regards to this. OK, thank you Dave for your feedback! Willy
Recommendations for deleting headers by regexp in 2.x?
We're upgrading from 1.8 to 2.x and one of the things I've noticed is that reqidel and rspidel seem to be totally gone in 2.1... What's the new recommendation to delete headers from request/response based on a regular expression? Do I have to write a Lua action to do this now? I read through the documentation for http-request and http-response and there doesn't seem to be an `http-request del-header-by-regex`... Our use case is that we have dozens of different internal headers behind a prefix, and we promise that we'll strip them all for incoming requests and outgoing responses at the edge load balancer. That is harder to do if we can't delete all headers matching a certain regex... -- James Brown Engineer
Re: Lua: forcing garbage collector after socket i/o
We are running this patch on top of 1.9.13 where it is needed. I will report back if/when we have anything to add. Untill then consider no news as good news in regards to this. Dave. On Tue, Jan 14, 2020 at 9:37 AM Willy Tarreau wrote: > > On Tue, Jan 14, 2020 at 09:31:07AM -0600, Dave Chiluk wrote: > > Can we get this backported onto the 2.0 and 1.9 stable streams? It > > looks like it mostly cleanly patches. *(aside from line numbers). > > Given that the risk of regression is far from zero (hence the tag "medium"), > I'd rather avoid for a while and observe instead. Very few users will notice > an improvement, maybe only two, but every Lua user would have to accept the > risk of a possible regression, so care is mandatory. We'd do it to 2.1 first, > and after a few releases possibly to 2.0 if there is compelling demand for > this. By then 1.9 will likely be dead anyway. > > If you're facing a high enough Lua-based connection rate that would make this > a nice improvement to the point where you'd be taking the risk to use the > backport, I think everyone would very much appreciate that you run with this > patch for a while to help confirm it doesn't break anything. > > Thanks, > Willy
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
чт, 23 янв. 2020 г. в 03:10, Tim Düsterhus : > Ilya, > > Am 22.01.20 um 23:04 schrieb Илья Шипицин: > >> Yes, that's my understanding of GitHub actions as well. However I > >> dislike having three types of CI (Travis, Cirrus and GitHub Actions). > >> Can Travis be replaced with GitHub Actions for our use case? I guess > >> Cirrus can't, because FreeBSD? > > > > both travis and github actions do offer 4 parallel builds, while cirrus > and > > app veyor offer 1 parallel build. > > Parallel Builds just improve test speed. I don't consider that an > important selling point for us. The development process is fairly > asynchronous anyway and the important thing is that there are results > for more obscure configurations, not that there results within 1 minute. > However ... > > > travis-ci offers ARM64, ppc64le and s390x (not available on github > actions). > > ... that's a good argument to keep Travis-CI. Too bad, I like the GitHub > Actions integration better. > > me too :) travis is not comfortable for choosing custom images (for example, if you wish to build on Fedora or Arch). > >>> +- name: fake step > >> > >> Give a proper name to that step. "Show pwd" is fine. > >> > > > > > > there should be no such step. > > however, without that step cygwin fails for no visible reason. > > Then it's even more important to give a good name (or comment). > Otherwise you might risk that someone removes it accidentally! > > >> 2. Split the ./haproxy -vv into a separate step, if that's possible. > >> > > sure, it's possible > > > > Perfect. I wasn't sure whether the environment was somehow cleaned up in > between the steps. > > Best regards > Tim Düsterhus >
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
Ilya, Am 22.01.20 um 23:04 schrieb Илья Шипицин: >> Yes, that's my understanding of GitHub actions as well. However I >> dislike having three types of CI (Travis, Cirrus and GitHub Actions). >> Can Travis be replaced with GitHub Actions for our use case? I guess >> Cirrus can't, because FreeBSD? > > both travis and github actions do offer 4 parallel builds, while cirrus and > app veyor offer 1 parallel build. Parallel Builds just improve test speed. I don't consider that an important selling point for us. The development process is fairly asynchronous anyway and the important thing is that there are results for more obscure configurations, not that there results within 1 minute. However ... > travis-ci offers ARM64, ppc64le and s390x (not available on github actions). ... that's a good argument to keep Travis-CI. Too bad, I like the GitHub Actions integration better. >>> +- name: fake step >> >> Give a proper name to that step. "Show pwd" is fine. >> > > > there should be no such step. > however, without that step cygwin fails for no visible reason. Then it's even more important to give a good name (or comment). Otherwise you might risk that someone removes it accidentally! >> 2. Split the ./haproxy -vv into a separate step, if that's possible. >> > sure, it's possible > Perfect. I wasn't sure whether the environment was somehow cleaned up in between the steps. Best regards Tim Düsterhus
Re: mcli vtest broken by commit.?. MEDIUM: connections: Get ride of the xprt_done callback.
Hi Pieter, On Wed, Jan 22, 2020 at 09:57:31PM +0100, PiBa-NL wrote: > Hi Olivier, > > Just to let you know, seems this commit has broken a few regtests: > http://git.haproxy.org/?p=haproxy.git;a=commit;h=477902bd2e8c1e978ad43d22dba1f28525bb797a > > https://api.cirrus-ci.com/v1/task/5885732300521472/logs/main.log > Testing with haproxy version: 2.2-dev1 > #top TEST reg-tests/mcli/mcli_show_info.vtc TIMED OUT (kill -9) > #top TEST reg-tests/mcli/mcli_show_info.vtc FAILED (10.044) signal=9 > #top TEST reg-tests/mcli/mcli_start_progs.vtc TIMED OUT (kill -9) > #top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (10.019) signal=9 > > Can reproduce it on my own FreeBSD machine as well, the testcase just sits > and waits.. until the vtest-timeout strikes. > > Do you need more info? If so what can i provide.? We noticed it this evening, it just happens that these tests already used to fail on his machine so there was nothing alarming, but when tested on mine the same issues popped up and was new. Strangely, running the test by hand using socat works perfectly fine, and it seems that it only fails when run by the vtest CLI client. I can't grasp the difference between both at this point. I also noticed that after 10 connections to the master we can't create new ones, just as if something was not properly released and the connection wouldn't die... which is totally odd since it's only the connection setup which was cleaned up. Both of us gave up for today but we're still interested in a reliable reproducer outside of vtest in case you happen to stumble upon one (don't waste your time searching however). I suspect we've accidently uncovered a nasty corner case somewhere, which is good since the purpose of the code cleanup was precisely to make some complex bugs easier to spot and analyze :-) Thanks for your quick report! I hoped nobody else would notice in the mean time, and I was wrong :-) Cheers, Willy
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
чт, 23 янв. 2020 г. в 02:59, Tim Düsterhus : > Ilya, > > Am 22.01.20 um 22:41 schrieb Илья Шипицин: > > let us move it to Github Actions. > > as far as I tried no extra step are required for enabling, just proper > file > > in > > > > .github/workflows > > Yes, that's my understanding of GitHub actions as well. However I > dislike having three types of CI (Travis, Cirrus and GitHub Actions). > Can Travis be replaced with GitHub Actions for our use case? I guess > Cirrus can't, because FreeBSD? > both travis and github actions do offer 4 parallel builds, while cirrus and app veyor offer 1 parallel build. travis-ci offers ARM64, ppc64le and s390x (not available on github actions). > > To the patch: > > > From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001 > > From: Ilya Shipitsin > > Date: Thu, 23 Jan 2020 02:33:38 +0500 > > Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions > > > > builds on travis-ci fail because of > > > https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359 > > unfortunately, documentation does not help. > > > > so, let us move builds to Github Actions. > > > > also, remove deprecated "sudo" directive from .travis.yml > > Could you please create separate patches for several unrelated changes? > If there's an "also" in the commit message then that's an indication > that the patch should be split. > > Single-line patches are perfectly acceptable. I send series containing > multiple single-line patches to the list on a regular basis (as distinct > emails even, because I use git-send-email). > > > +steps: > > +- uses: actions/checkout@v1 > > +- name: install prerequisites > > + run: choco install bash make libssl-devel cygwin-devel gcc-core > libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin > > +- name: fake step > > Give a proper name to that step. "Show pwd" is fine. > there should be no such step. however, without that step cygwin fails for no visible reason. I'll remove this step after I'll find it out. > > > + run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd' > > +- name: build > > + run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make > TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 > USE_THREAD=1 && ./haproxy -vv' > > 1. Does GitHub Actions support variables for the build flags, similar to > Travis? That would make things more readable. > yes. > > 2. Split the ./haproxy -vv into a separate step, if that's possible. > sure, it's possible > > Best regards > Tim Düsterhus >
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
Willy, Ilya, Am 22.01.20 um 22:52 schrieb Willy Tarreau: > OK, why not, let's give it a try. Now applied, thanks! > Willy > Appears to work fine: https://github.com/haproxy/haproxy/runs/403976092 However see my sibling email, please. Best regards Tim Düsterhus
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
Ilya, Am 22.01.20 um 22:41 schrieb Илья Шипицин: > let us move it to Github Actions. > as far as I tried no extra step are required for enabling, just proper file > in > > .github/workflows Yes, that's my understanding of GitHub actions as well. However I dislike having three types of CI (Travis, Cirrus and GitHub Actions). Can Travis be replaced with GitHub Actions for our use case? I guess Cirrus can't, because FreeBSD? To the patch: > From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001 > From: Ilya Shipitsin > Date: Thu, 23 Jan 2020 02:33:38 +0500 > Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions > > builds on travis-ci fail because of > https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359 > unfortunately, documentation does not help. > > so, let us move builds to Github Actions. > > also, remove deprecated "sudo" directive from .travis.yml Could you please create separate patches for several unrelated changes? If there's an "also" in the commit message then that's an indication that the patch should be split. Single-line patches are perfectly acceptable. I send series containing multiple single-line patches to the list on a regular basis (as distinct emails even, because I use git-send-email). > +steps: > +- uses: actions/checkout@v1 > +- name: install prerequisites > + run: choco install bash make libssl-devel cygwin-devel gcc-core > libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin > +- name: fake step Give a proper name to that step. "Show pwd" is fine. > + run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd' > +- name: build > + run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make > TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 > USE_THREAD=1 && ./haproxy -vv' 1. Does GitHub Actions support variables for the build flags, similar to Travis? That would make things more readable. 2. Split the ./haproxy -vv into a separate step, if that's possible. Best regards Tim Düsterhus
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
On Wed, Jan 22, 2020 at 10:52:28PM +0100, Willy Tarreau wrote: > Hi Ilya, > > On Thu, Jan 23, 2020 at 02:41:05AM +0500, ??? wrote: > > Hello, > > > > I spent lots of time trying to get cygwin on travis working. > > no luck. > > > > let us move it to Github Actions. > > as far as I tried no extra step are required for enabling, just proper file > > in > > > > .github/workflows > > > > (attached in patch). > > OK, why not, let's give it a try. Now applied, thanks! Looks like it works: https://github.com/haproxy/haproxy/runs/403976092 Great, we're increasing coverage! Willy
Re: [PATCH[ re-enable cygwin CI builds (Github Actions)
Hi Ilya, On Thu, Jan 23, 2020 at 02:41:05AM +0500, ??? wrote: > Hello, > > I spent lots of time trying to get cygwin on travis working. > no luck. > > let us move it to Github Actions. > as far as I tried no extra step are required for enabling, just proper file > in > > .github/workflows > > (attached in patch). OK, why not, let's give it a try. Now applied, thanks! Willy
[PATCH[ re-enable cygwin CI builds (Github Actions)
Hello, I spent lots of time trying to get cygwin on travis working. no luck. let us move it to Github Actions. as far as I tried no extra step are required for enabling, just proper file in .github/workflows (attached in patch). Cheers, Ilya Shipitcin From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Thu, 23 Jan 2020 02:33:38 +0500 Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions builds on travis-ci fail because of https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359 unfortunately, documentation does not help. so, let us move builds to Github Actions. also, remove deprecated "sudo" directive from .travis.yml --- .github/workflows/windows-latest.yml | 20 .travis.yml | 7 --- 2 files changed, 20 insertions(+), 7 deletions(-) create mode 100644 .github/workflows/windows-latest.yml diff --git a/.github/workflows/windows-latest.yml b/.github/workflows/windows-latest.yml new file mode 100644 index 0..27b16eec6 --- /dev/null +++ b/.github/workflows/windows-latest.yml @@ -0,0 +1,20 @@ +# build status appears on https://github.com/haproxy/haproxy/actions + +name: windows-latest + +on: [push] + +jobs: + cygwin: + +runs-on: windows-latest + +steps: +- uses: actions/checkout@v1 +- name: install prerequisites + run: choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin +- name: fake step + run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd' +- name: build + run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv' + diff --git a/.travis.yml b/.travis.yml index a0d9502b4..bf4b82aa9 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,6 +1,5 @@ # build status appears on https://travis-ci.com/haproxy/haproxy -sudo: required dist: bionic language: c @@ -86,12 +85,6 @@ matrix: if: type != cron compiler: clang env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1d -# - os: windows -#if: type == cron -#install: -# - choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin -#script: -# - C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv' - os: linux if: type == cron compiler: clang -- 2.24.1
mcli vtest broken by commit.?. MEDIUM: connections: Get ride of the xprt_done callback.
Hi Olivier, Just to let you know, seems this commit has broken a few regtests: http://git.haproxy.org/?p=haproxy.git;a=commit;h=477902bd2e8c1e978ad43d22dba1f28525bb797a https://api.cirrus-ci.com/v1/task/5885732300521472/logs/main.log Testing with haproxy version: 2.2-dev1 #top TEST reg-tests/mcli/mcli_show_info.vtc TIMED OUT (kill -9) #top TEST reg-tests/mcli/mcli_show_info.vtc FAILED (10.044) signal=9 #top TEST reg-tests/mcli/mcli_start_progs.vtc TIMED OUT (kill -9) #top TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (10.019) signal=9 Can reproduce it on my own FreeBSD machine as well, the testcase just sits and waits.. until the vtest-timeout strikes. Do you need more info? If so what can i provide.? Regards, Pieter
Re: [PATCH] MEDIUM: cli: Allow multiple filter entries for show table
Adis, On Wed, Jan 22, 2020 at 04:57:10PM +0100, Adis Nezirovic wrote: > Hello, > > Here it is, I've introduced a bug in last patch (since it is recent I've put > "MINOR", even if it might have bigger impact in prod. Strangely, while I was certain I had build-tested the original one, apparently I failed as it didn't build for two errors, which I fixed in a subsequent patch. I'm seeing that your patches still seem to rely on this bug (data_op < 0) which normally cannot compile so I find this surprising. I still merged the first one of these two because I think it's OK, however, could you please add a bit of commit message to the second one ? As a rule of thumb, the commit message should be used to "sell" me your patch and to sell it to whoever would be hesitating in backporting it or not. Thus I think that here the benefits and/or impacts are not obvious from this one-liner : > Subject: [PATCH 2/2] MINOR: cli: Report location of error or any extra data > for "show table" Thanks! Willy
Re: [PATCH] BUG/MINOR: ssl: ssl_sock_load_pem_into_ckch is not consistent
> Le 22 janv. 2020 à 15:56, William Lallemand a écrit : > > On Mon, Jan 20, 2020 at 05:13:13PM +0100, Emmanuel Hocdet wrote: >> >> Hi, >> >> Proposal to fix the issue. >> > > The purpose at the beginning was to be able to keep a .dh / .ocsp etc. But > that > probably does not make sense once you changed the private key, we should > probably remove everything in a ckch once we load a new private key. > Indeed, and the case of ckch->ocsp_issuer is also problematic. > Thanks, merged! > Thanks Manu
Re: [PATCH] MEDIUM: cli: Allow multiple filter entries for show table
Hello,
Here it is, I've introduced a bug in last patch (since it is recent I've
put "MINOR", even if it might have bigger impact in prod.
Also, a patch for better error diagnostics, and to report any extra data
in the filter.
Best regards,
--
Adis Nezirovic
Software Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | https://www.haproxy.com
>From 030ee62e99d0ede6ea2e0bae16d4621cdd77da19 Mon Sep 17 00:00:00 2001
From: Adis Nezirovic
Date: Wed, 22 Jan 2020 16:16:48 +0100
Subject: [PATCH 1/2] BUG/MINOR: cli: Missing arg offset for filter data
values.
We don't properly check for missing data values for additional filter
entries, passing out of bounds index to args[], then passing to strlen.
Introduced in commit 1a693fc2: (MEDIUM: cli: Allow multiple filter
entries for "show table")
---
src/stick_table.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/stick_table.c b/src/stick_table.c
index 1393b1ff3..beec279c2 100644
--- a/src/stick_table.c
+++ b/src/stick_table.c
@@ -3620,7 +3620,7 @@ static int table_prepare_data_request(struct appctx *appctx, char **args)
if (appctx->ctx.table.data_op < 0)
return cli_err(appctx, "Require and operator among \"eq\", \"ne\", \"le\", \"ge\", \"lt\", \"gt\"\n");
- if (!*args[5] || strl2llrc(args[5+3*i], strlen(args[5+3*i]), >ctx.table.value[i]) != 0)
+ if (!*args[5+3*i] || strl2llrc(args[5+3*i], strlen(args[5+3*i]), >ctx.table.value[i]) != 0)
return cli_err(appctx, "Require a valid integer value to compare against\n");
}
--
2.25.0
>From d193f62ff73f01deb1eae77c0d3f319af50858ad Mon Sep 17 00:00:00 2001
From: Adis Nezirovic
Date: Wed, 22 Jan 2020 16:50:27 +0100
Subject: [PATCH 2/2] MINOR: cli: Report location of error or any extra data
for "show table"
---
src/stick_table.c | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/src/stick_table.c b/src/stick_table.c
index beec279c2..b7d5d145b 100644
--- a/src/stick_table.c
+++ b/src/stick_table.c
@@ -3601,6 +3601,7 @@ static int table_process_entry_per_key(struct appctx *appctx, char **args)
static int table_prepare_data_request(struct appctx *appctx, char **args)
{
int i;
+ char *err = NULL;
if (appctx->ctx.table.action != STK_CLI_ACT_SHOW && appctx->ctx.table.action != STK_CLI_ACT_CLR)
return cli_err(appctx, "content-based lookup is only supported with the \"show\" and \"clear\" actions\n");
@@ -3611,17 +3612,21 @@ static int table_prepare_data_request(struct appctx *appctx, char **args)
/* condition on stored data value */
appctx->ctx.table.data_type[i] = stktable_get_data_type(args[3+3*i] + 5);
if (appctx->ctx.table.data_type[i] < 0)
- return cli_err(appctx, "Unknown data type\n");
+ return cli_dynerr(appctx, memprintf(, "Filter entry #%i: Unknown data type\n", i + 1));
if (!((struct stktable *)appctx->ctx.table.target)->data_ofs[appctx->ctx.table.data_type[i]])
- return cli_err(appctx, "Data type not stored in this table\n");
+ return cli_dynerr(appctx, memprintf(, "Filter entry #%i: Data type not stored in this table\n", i + 1));
appctx->ctx.table.data_op[i] = get_std_op(args[4+3*i]);
if (appctx->ctx.table.data_op < 0)
- return cli_err(appctx, "Require and operator among \"eq\", \"ne\", \"le\", \"ge\", \"lt\", \"gt\"\n");
+ return cli_dynerr(appctx, memprintf(, "Filter entry #%i: Require and operator among \"eq\", \"ne\", \"le\", \"ge\", \"lt\", \"gt\"\n", i + 1));
if (!*args[5+3*i] || strl2llrc(args[5+3*i], strlen(args[5+3*i]), >ctx.table.value[i]) != 0)
- return cli_err(appctx, "Require a valid integer value to compare against\n");
+ return cli_dynerr(appctx, memprintf(, "Filter entry #%i: Require a valid integer value to compare against\n", i + 1));
+ }
+
+ if (*args[3+3*i]) {
+ return cli_dynerr(appctx, memprintf(, "Detected extra data in filter, %ith word of input, after '%s'\n", 3+3*i + 1, args[2+3*i]));
}
/* OK we're done, all the fields are set */
--
2.25.0
Re: Haproxy loadbalancing out going mail to Antispam servers
Hi Guys Just to add. Im using Debian package version. I.e. HA-Proxy version 1.7.5-2 Regards Brent On 2020/01/22 17:18, Brent Clark wrote: Good day Guys We have a project where we are trying to load balance to our outbound Spamexperts Antispam relays / servers. We hit a snag where our clients servers are getting 'Too many concurrent SMTP connections from this IP address'. As a result the mail queue is building up on the servers. After reverting our change, the problem went away. Our setup is: (CLIENT SERVERS INDC) ---> 587 (HAPROXY) ---> (ANTISPAM) ---> (INTERNET) While I am performance tuning and repoking under the hood etc, could I ask if someone could please peer review my config / setup. https://pastebin.com/raw/3D8frtzw If someone from the community can help, it would be appreciated. Many thanks Regards Brent Clark
Haproxy loadbalancing out going mail to Antispam servers
Good day Guys We have a project where we are trying to load balance to our outbound Spamexperts Antispam relays / servers. We hit a snag where our clients servers are getting 'Too many concurrent SMTP connections from this IP address'. As a result the mail queue is building up on the servers. After reverting our change, the problem went away. Our setup is: (CLIENT SERVERS INDC) ---> 587 (HAPROXY) ---> (ANTISPAM) ---> (INTERNET) While I am performance tuning and repoking under the hood etc, could I ask if someone could please peer review my config / setup. https://pastebin.com/raw/3D8frtzw If someone from the community can help, it would be appreciated. Many thanks Regards Brent Clark
Re: [PATCH] BUG/MINOR: ssl: ssl_sock_load_pem_into_ckch is not consistent
On Mon, Jan 20, 2020 at 05:13:13PM +0100, Emmanuel Hocdet wrote: > > Hi, > > Proposal to fix the issue. > The purpose at the beginning was to be able to keep a .dh / .ocsp etc. But that probably does not make sense once you changed the private key, we should probably remove everything in a ckch once we load a new private key. Thanks, merged! -- William Lallemand
Re: [PATCH] MEDIUM: cli: Allow multiple filter entries for show table
Hi Adis, On Wed, Jan 22, 2020 at 10:38:55AM +0100, Adis Nezirovic wrote: > Hello, > > Here is a patch to extend 'show table' with multiple filters. We already > have this functionality in Lua (also adapted here to use the same definition > for number of filters) Oh that's really great, as I've been missing that feature quite a few times! > The current code is somewhat relaxed about the extra data (garbage) after > the > 'data. ' > part, and we've kept that behavior. For forward compatibility, it would be > nice if we can introduce either additional 'haproxy -vv' info line, or CLI > command (e.g. 'show cli filters'), which would return the supported number > of filter entries. Don't you think that the extension of the syntax is the right opportunity to make this parser a bit stricter ? Otherwise you risk to expose it a bit more, with possibly more users adding undetected mistakes, which will make the enforcement harder later. Anyway I merged it as-is because it shouldn't be a showstopper to its merge, but if you think you can quickly add a bit more control, that would be welcome! > Error handling could also be improved (we don't show which filter entry is > wrong, if we have more than one), but I wanted to first check with you guys > if we can add "snprintf" style error function to the cli, in addition to > existing cli_dynerr()/cli_dynmsg() Usually what we do is that we call memprintf() and pass it to cli_dyn*() which then frees it. So you shouldn't need to add a new one. Thanks! Willy
Re: [PATCH] temporarily mark openssl-1.0.2 builds as allowed failure
On Wed, Jan 22, 2020 at 03:27:24PM +0500, ??? wrote: > Hello, > > this is a follow up patch for the recent travis-ci improvement. applied, thanks! Willy
[ANNOUNCE] haproxy-2.2-dev1
Hi,
HAProxy 2.2-dev1 was released on 2020/01/22. It added 241 new commits
after version 2.2-dev0.
While many of us were apparently serving as a feedback loop between a
screen and a keyboard, time was still flying and 2 months elapsed since
2.1 was released. So I thought it was about time to issue a new
development preview to ease testing.
Let's keep the door open for discussions about upcoming changes for the
next 2 months (till end of March) and reserve the last two months to
finish and fix, aiming at a 2.2 release around end of May.
Now, regarding the changes that came in between 2.1 and 2.2-dev1, I'm
noticing approximately these (for those not listed, sorry if I forgot
your work, feel free to chime in) :
- a significant number of code cleanups and harmless refactoring
(connection, dns, HTTP/TCP actions)
- rework of errorfiles: new "http-errors" sections to ease their share
between sections, and the ability to specifiy the one to use on "deny"
rules.
- DNS traffic reduction for SRV records by reusing information already
available in extension parts of the response packet
- better reporting of internal processing errors. In the past, adding
a header in too large a buffer would be silently ignored. Now we can
finally fail and report an error by chosing between strict or relaxed
mode using the "strict-mode" action.
- some security hardening, such as preventing the creation of processes
at runtime by default, and preventing the process from switching UIDs
again, in order to limit the risks of abuses of bugs or uncontrollable
inherited Lua libraries. This is now enabled by default, and this will
break external checks, which will require to add the new global option
"insecure-fork-wanted", and even "insecure-setuid-wanted" if calling a
setuid binary (both options should ring a bell for the users who really
want to depend on them).
- Lua's GC is not systematically called anymore when dealing with outgoing
connections. This almost doubles the Lua's performance in such use cases.
- some performance improvements at the connection layer resulting in
less syscalls on average, especially for epoll.
- the "debug" converter is now always available and it logs to internal
event sinks (ring buffer, stdout, stderr).
- a number of new sample fetch functions expose HTX internals to help
live debugging.
- addition of status codes "404" and "410" for http-request deny
- new "replace-path" HTTP action to help replace old "reqrep" rules. It
works like replace-uri but only acts on the path. This makes a
difference in H2 or with absolute requests in HTTP/1.
- new "attr" field added to the "cookie" directive, in order to set cookie
attributes when adding persistence cookies.
- new CLI command "show ssl cert" to report detailed information on loaded
certificates, such as validity dates, issuer, alt names etc.
- LDAPv3 alternate output format for ssl_{c,f}_{i,s}_dn sample fetches.
- de-duplication of ca-file and crl-file, which should also save startup
speed and memory when there are many of them.
- 10 new regtests (thanks!)
- 84 bugs fixed
These are already quite a nice number of improvements. I suspect that next
versions might degrade a little bit as usual depending how things go, which
is also another reason to keep an expectedly clean reference version to work
on. If possible I'd like to release more often for the next ones to keep
changes visible and easy to test.
Ah, last thing, I messed up with the release, there are two RELEASE commits,
don't worry you're not drunk. But I noticed it far too late to fix it with
a forced push, so my punishment will be to look stupid now (I sensed it was
already the case anyway so that's OK).
Please find the usual URLs below :
Site index : http://www.haproxy.org/
Discourse: http://discourse.haproxy.org/
Slack channel: https://slack.haproxy.org/
Issue tracker: https://github.com/haproxy/haproxy/issues
Sources : http://www.haproxy.org/download/2.2/src/
Git repository : http://git.haproxy.org/git/haproxy.git/
Git Web browsing : http://git.haproxy.org/?p=haproxy.git
Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/
Willy
---
Complete changelog :
Baptiste Assmann (2):
MEDIUM: dns: use Additional records from SRV responses
BUG/MINOR: http_act: don't check capture id in backend
Ben51Degrees (1):
BUG/MINOR: 51d: Fix bug when HTX is enabled
Christopher Faulet (84):
BUG/MINOR: h1: Don't test the host header during response parsing
BUG/MINOR: http-htx: Don't make http_find_header() fail if the value is
empty
BUG/MINOR: fcgi-app: Make the directive pass-header case insensitive
BUG/MINOR: stats: Fix HTML output for the frontends heading
BUG/MEDIUM: mux-h1: Never reuse H1
Re: [PATCH] MEDIUM: dns: support for Additional section
gloups, I did fix all those points before sending the final version and I forgot to clean up the comments. Will send a patch to clean them up. Baptiste
[PATCH] temporarily mark openssl-1.0.2 builds as allowed failure
Hello,
this is a follow up patch for the recent travis-ci improvement.
Cheers,
Ilya Shipitcin
From f13c8cd8c28d376c914cee24ac9b7e09a7203473 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin
Date: Wed, 22 Jan 2020 15:23:51 +0500
Subject: [PATCH] BUILD: CI: temporarily mark openssl-1.0.2 as allowed failure
Until #429 is resolved, let us ignore openssl-1.0.2 failures
---
.travis.yml | 6 ++
1 file changed, 6 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index 9046e6dcd..a0d9502b4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -99,6 +99,12 @@ matrix:
before_script:
- git clone http://git.1wt.eu/git/libslz.git/
- cd libslz && make && make PREFIX=${HOME}/opt install && cd ..
+ allow_failures:
+ - os: linux
+arch: ppc64le
+if: type == cron
+compiler: gcc
+env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
install:
- git clone https://github.com/VTest/VTest.git ../vtest
--
2.24.1
[PATCH] MEDIUM: cli: Allow multiple filter entries for show table
Hello,
Here is a patch to extend 'show table' with multiple filters. We already
have this functionality in Lua (also adapted here to use the same
definition for number of filters)
The current code is somewhat relaxed about the extra data (garbage)
after the
'data. '
part, and we've kept that behavior. For forward compatibility, it would
be nice if we can introduce either additional 'haproxy -vv' info line,
or CLI command (e.g. 'show cli filters'), which would return the
supported number of filter entries.
Error handling could also be improved (we don't show which filter entry
is wrong, if we have more than one), but I wanted to first check with
you guys if we can add "snprintf" style error function to the cli, in
addition to existing cli_dynerr()/cli_dynmsg()
Best regards,
--
Adis Nezirovic
Software Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | https://www.haproxy.com
>From fdfcf4e124db1cffcce301116198bab9deec6583 Mon Sep 17 00:00:00 2001
From: Adis Nezirovic
Date: Thu, 16 Jan 2020 15:19:29 +0100
Subject: [PATCH] MEDIUM: cli: Allow multiple filter entries for "show table"
For complex stick tables with many entries/columns, it can be beneficial
to filter using multiple criteria. The maximum number of filter entries
can be controlled by defining STKTABLE_FILTER_LEN during build time.
This patch can be backported to older releases.
---
doc/management.txt| 4 +-
include/common/defaults.h | 5 ++
include/types/applet.h| 6 +--
src/hlua_fcn.c| 7 ++-
src/stick_table.c | 111 +-
5 files changed, 76 insertions(+), 57 deletions(-)
diff --git a/doc/management.txt b/doc/management.txt
index 521a67112..24969be88 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -2569,7 +2569,7 @@ show table
>>> # table: front_pub, type: ip, size:204800, used:171454
>>> # table: back_rdp, type: ip, size:204800, used:0
-show table [ data. ] | [ key ]
+show table [ data. [data. ...]] | [ key ]
Dump contents of stick-table . In this mode, a first line of generic
information about the table is reported as with "show table", then all
entries are dumped. Since this can be quite heavy, it is possible to specify
@@ -2588,6 +2588,8 @@ show table [ data. ] | [ key ]
- lt : match entries whose data is less than this value
- gt : match entries whose data is greater than this value
+ In this form, you can use multiple data filter entries, up to a maximum
+ defined during build time (4 by default).
When the key form is used the entry is shown. The key must be of the
same type as the table, which currently is limited to IPv4, IPv6, integer,
diff --git a/include/common/defaults.h b/include/common/defaults.h
index fbacca481..e86c9bce1 100644
--- a/include/common/defaults.h
+++ b/include/common/defaults.h
@@ -116,6 +116,11 @@
#define STKTABLE_EXTRA_DATA_TYPES 0
#endif
+// max # of stick-table filter entries that can be used during dump
+#ifndef STKTABLE_FILTER_LEN
+#define STKTABLE_FILTER_LEN 4
+#endif
+
// max # of loops we can perform around a read() which succeeds.
// It's very frequent that the system returns a few TCP segments at a time.
#ifndef MAX_READ_POLL_LOOPS
diff --git a/include/types/applet.h b/include/types/applet.h
index dfb489c9f..76a598d21 100644
--- a/include/types/applet.h
+++ b/include/types/applet.h
@@ -150,9 +150,9 @@ struct appctx {
void *target; /* table we want to dump, or NULL for all */
struct stktable *t; /* table being currently dumped (first if NULL) */
struct stksess *entry; /* last entry we were trying to dump (or first if NULL) */
- long long value; /* value to compare against */
- signed char data_type; /* type of data to compare, or -1 if none */
- signed char data_op; /* operator (STD_OP_*) when data_type set */
+ long long value[STKTABLE_FILTER_LEN]; /* value to compare against */
+ signed char data_type[STKTABLE_FILTER_LEN]; /* type of data to compare, or -1 if none */
+ signed char data_op[STKTABLE_FILTER_LEN];/* operator (STD_OP_*) when data_type set */
char action;/* action on the table : one of STK_CLI_ACT_* */
} table;
struct {
diff --git a/src/hlua_fcn.c b/src/hlua_fcn.c
index 7ef708ccb..f8024aa84 100644
--- a/src/hlua_fcn.c
+++ b/src/hlua_fcn.c
@@ -39,7 +39,6 @@ static int class_listener_ref;
static int class_regex_ref;
static int class_stktable_ref;
-#define MAX_STK_FILTER_LEN 4
#define STATS_LEN (MAX((int)ST_F_TOTAL_FIELDS, (int)INF_TOTAL_FIELDS))
static THREAD_LOCAL struct field stats[STATS_LEN];
@@ -682,7 +681,7 @@ int hlua_stktable_dump(lua_State *L)
int op;
int dt;
long long val;
- struct stk_filter filter[MAX_STK_FILTER_LEN];
+ struct stk_filter filter[STKTABLE_FILTER_LEN];
int filter_count = 0;
int i;
int skip_entry;
@@ -700,8 +699,8 @@ int
Re: [PATCH] introduce ARM64 travis-ci builds
On Wed, Jan 22, 2020 at 01:40:43PM +0500, ??? wrote: > ??, 22 ???. 2020 ?. ? 10:49, Willy Tarreau : > > > On Sun, Jan 19, 2020 at 12:18:00PM +0500, ??? wrote: > > > hello, > > > > > > sometimes arm64 builds fails, I think it is good chance to introduce > > > regular builds > > > and fix them. > > > > > > also, few small improvements. > > > > > occasionely, I introduced "socat" for linux builds. god knows, I only > wanted socat on osx. > > now we have daily failing openssl-1.0.2: > > https://travis-ci.com/haproxy/haproxy/jobs/278244081 > > should I temporarily mark it as "allowed failure" on travis ? (it will be > fixed in issue #429) Yes I noticed it and tried to figure what was happening until I figured it was the known issue. I'd prefer to mark it as "allowed failure" if possible so that we keep watching travis's output. Isn't it already the case, given that the icon still says "build|passing" ? Also given that it's a regtest that fails, maybe we should disable them for this target temporarily ? The right thing to do should be to tag this regtest as requiring 1.1.0 but we don't have this level of flexibility yet :-/ Willy
Re: [PATCH] introduce ARM64 travis-ci builds
ср, 22 янв. 2020 г. в 10:49, Willy Tarreau : > On Sun, Jan 19, 2020 at 12:18:00PM +0500, ??? wrote: > > hello, > > > > sometimes arm64 builds fails, I think it is good chance to introduce > > regular builds > > and fix them. > > > > also, few small improvements. > occasionely, I introduced "socat" for linux builds. god knows, I only wanted socat on osx. now we have daily failing openssl-1.0.2: https://travis-ci.com/haproxy/haproxy/jobs/278244081 should I temporarily mark it as "allowed failure" on travis ? (it will be fixed in issue #429) > > Merged, thanks Ilya. Next time, please be stricter and split your > additions and your small improvements as it can help track down issues > or do partial reverts. > > Willy >
