[PATCH] CI: enable QUIC for Coverity scan
Hello,
since QUIC is first class citizen, let us scan it in code analysis.
Ilya
From dd316a69698299f0bfc5ec7a86133f83dc1061a3 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin
Date: Wed, 2 Feb 2022 10:24:58 +0500
Subject: [PATCH] CI: enable QUIC for Coverity scan
---
.github/workflows/coverity.yml | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/coverity.yml b/.github/workflows/coverity.yml
index b3dd5ec52..cfde87b01 100644
--- a/.github/workflows/coverity.yml
+++ b/.github/workflows/coverity.yml
@@ -20,7 +20,7 @@ jobs:
COVERITY_SCAN_PROJECT_NAME: 'Haproxy'
COVERITY_SCAN_BRANCH_PATTERN: '*'
COVERITY_SCAN_NOTIFICATION_EMAIL: 'chipits...@gmail.com'
- COVERITY_SCAN_BUILD_COMMAND: "make CC=clang TARGET=linux-glibc USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=addons/wurfl/dummy WURFL_LIB=addons/wurfl/dummy USE_DEVICEATLAS=1 DEVICEATLAS_SRC=addons/deviceatlas/dummy USE_51DEGREES=1 51DEGREES_SRC=addons/51degrees/dummy/pattern"
+ COVERITY_SCAN_BUILD_COMMAND: "make CC=clang TARGET=linux-glibc USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_QUIC=1 USE_SYSTEMD=1 USE_WURFL=1 WURFL_INC=addons/wurfl/dummy WURFL_LIB=addons/wurfl/dummy USE_DEVICEATLAS=1 DEVICEATLAS_SRC=addons/deviceatlas/dummy USE_51DEGREES=1 51DEGREES_SRC=addons/51degrees/dummy/pattern ADDLIB=\"-Wl,-rpath,$HOME/opt/lib/\" SSL_LIB=${HOME}/opt/lib SSL_INC=${HOME}/opt/include"
steps:
- uses: actions/checkout@v2
- name: Install apt dependencies
@@ -29,6 +29,9 @@ jobs:
sudo apt-get install -y \
liblua5.3-dev \
libsystemd-dev
+- name: Install QUICTLS
+ run: |
+QUICTLS=yes scripts/build-ssl.sh
- name: Hijack Makefile
run: |
# We cannot pass the DEBUG variable in `COVERITY_SCAN_BUILD_COMMAND`,
--
2.34.1
Re: Per-client queue for rate limiting?
--- Original Message ---
On Sunday, January 30th, 2022 at 2:18 AM, Orlando Della Casa
wrote:
> I’d like to put a rate limit on incoming HTTP requests, but without sending
> 429 errors.
You could simply delay the request with Lua. We use a stick table to track
requests and if an IP exceeds the limit, a Lua function gets called that delays
the request for a random amount of time. You could probably set a var with the
current req rate and calculate a delay based on that.
Here's a simple example:
# delay_request.lua
function delay_request (txn)
local http_req_rate = txn:get_var('txn.http_req_rate')
-- calculate your delay somehow
core.msleep(delay_ms)
end
core.register_action('delay_request', {'http-req'}, delay_request, 0)
# haproxy.cfg
global
lua-load /path/to/delay_request.lua
frontend fe
stick-table type ipv6 size 1m expire 1m store http_req_rate(1m)
http-request track-sc0 src
acl limit_exceeded src_http_req_rate() gt 60
http-request set-var(txn.http_req_rate) src_http_req_rate()
http-request lua.delay_request if limit_exceeded
...
Bren
Re: [ANNOUNCE] haproxy-2.6-dev1
On Tue, Feb 01, 2022 at 07:41:08PM +0100, Willy Tarreau wrote: > - master-worker: some simplifications were applied to the way the new > process connects to the old one to retrieve listening sockets. In the > past it would rely on "-x" on the command line, or would try to connect > to the first socket that had the "expose-fd listener" statement. Now > the new process will simply reuse the existing worker socket to > retrieve the old socket FDs so that it will work even without a stats > socket. As such "-x" is no more used in master mode. To be a bit more precise, the "-x" option is still used internally, it is now able to use the internal socketpairs which are used between the master and the workers. So after a reload you would see something like this: 68684 pts/14 S+ 0:00 ./haproxy -W -f haproxy.cfg 68703 pts/14 Sl+0:00 ./haproxy -sf 68686 -x sockpair@3 -W -f haproxy.cfg The biggest benefit is that you don't have to configure anything anymore to do a hitless reload. -- William Lallemand
[ANNOUNCE] haproxy-2.6-dev1
Hi, HAProxy 2.6-dev1 was released on 2022/02/01. It added 379 new commits after version 2.6-dev0. That first development release is huge (379 commits), essentially because by being busy on bugs we didn't see the time fly and forgot to emit earlier versions as I wanted to. Too bad. Now the punishment is to read all this and try to build up a friendly changelog! I'm going to skip over the 51 bugs fixed as they're not interesting at this stage, and will try to focus on new features and improvements in general (stability, performance or convenience). Do not see any offence if your work is not mentioned, and just chime in if you want to add or rectify something: - QUIC: lots of improvements, too many to tell actually, this covers 167 patches or almost half of the changes. The architectural elements are starting to assemble well, some rough edges were polished and overall things are getting much better. The interop test reports at https://interop.seemann.io/ shows that we have nothing to be ashamed of and that our implementation is now on par with some others, which is very encouraging. I'm starting to get confident that it will be usable in 2.6, but I don't want to put needless pressure on the team by promising difficult things, the project itself is of an extremely high level of complexity. - pools: the cluster-based shared allocator was finally implemented. It allows highly threaded environments to rely on the shared allocator when the one provided by the system doesn't scale well with threads. Recently we used to disable it for modern allocators (jemalloc or glibc >= 2.26) but plenty of users are still relying on different ones and we needed to improve this situation. The new allocator is sometimes slightly faster, sometimes slightly slower than the fastest allocators, so this alone couldn't be a decisive factor to enable it or not. Thus I preferred to keep it disabled in such situations so that we focus on minimizing the memory usage. Last but not least, memory trimming is now supported with jemalloc. - master-worker: some simplifications were applied to the way the new process connects to the old one to retrieve listening sockets. In the past it would rely on "-x" on the command line, or would try to connect to the first socket that had the "expose-fd listener" statement. Now the new process will simply reuse the existing worker socket to retrieve the old socket FDs so that it will work even without a stats socket. As such "-x" is no more used in master mode. - variables: the set-var() converters now support a number of conditions like "ifexists", "ifset", "ifgt" and so on that are convenient to avoid systematically writing two rules to build them. We indeed noticed in some complex configs that such constructs were very common (e.g. to store a new max or to update a variable only if it was set). Conditions may be combined and can sometimes be used to reduce 3 or more rules to a single, more efficient and more maintainable one. - DeviceAtlas: update of the module to support live updates of the database. An external program can connect to a shared memory area and feed the new entries without having to reload HAProxy. - NUMA topology detection is now also supported on FreeBSD, and the CPU affinity API was updated to support the upcoming FreeBSD 14 which uses a linux-like API. - new debugging features: DEBUG_POOL_INTEGRITY will read and write patterns to allocated/released memory areas to check for corruption that may result from use-after-free or even hardware trouble, DEBUG_POOL_TRACING will help figure last users of a given area, expert-mode "debug dev fd" will report fantom file descriptors that are present in the process but unknown to haproxy (indicating a leak, or even possibly a malware in some dependencies), and a few extra sanity checks at a few places. On the CLI, "show version" shows the haproxy version (and was backported to stable branches). "show libs" will report all loaded shared libraries. The same may be achieved at startup time, in a way that eases the production of a tar file containing the haproxy executable and all dependencies so as to help users provide an exploitable core dump when needed. - and overall plenty of cleanups, build fixes and improvements (faster builds for developers and CI), some small but useful options (e.g: option idle-close-on-response to work around a bug or limitation in AWS LBs), more flexibility (e.g. setting expert/experimental mode from master CLI is now possible; "capture" action in http-after-response), doc updates. While reading some doc recently I also noticed that when I added the "set-var-fmt" action late in the 2.5 cycle, I wasn't much inspired when chosing the name, because while the word "fmt" is used a lot to
DOC issue with http-request return
I think I've found an error in the docs. The entry for `http-request return`
shows the last component as [ {if | unless} ] but if I leave that
off in a backend I get [ALERT] parsing... 'http-request return' expects either
'if' or 'unless' followed by a condition but found ‘…'.
I’ve attached a tiny patch.
—
Jay Wren
0001-DOC-BUG-http-request-return-if-unless-is-not-optiona.patch
Description: Binary data
smime.p7s
Description: S/MIME cryptographic signature
Re: [PATCH] get BoringSSL back to the game
вт, 1 февр. 2022 г. в 15:35, William Lallemand : > On Mon, Jan 31, 2022 at 10:22:01AM +0500, Илья Шипицин wrote: > > > > Hello, > > > > Hello Ilya, > > > 0001 .. 0003 are "pre QUIC" patches > > 0004 .. 0006 are most questionable QUIC part > > 0007 is very simple > > > > > > we can discuss whether BoringSSL should be > > 1) dropped completely > > 2) supported, but no QUIC > > 3) supported for QUIC as well > > > > as for "3)" I've checked current state of QUICTLS, looks like its future > is > > not clear, it is not updated since mid december 2021, also it is not > clear > > whether OpenSSL is going to accept it or not. > > > > thanks, > > Ilya > > > BoringSSL is a burden in term of maintenance, since they only work in a > rolling release mode, we can't offer a real support in maintenancecc > branches. > > The current development team also won't have time to support fully > BoringSSL, the only library we are fully supporting is OpenSSL. > > The other libs are supported as a best effort, but not all features are > available. > > I don't mind merging some patches for improving boringSSL support, but > what's bothering me is building the master with the boringSSL HEAD in > the CI. Because API changes and can broke the build without us doing > anything. > I'm fine with best effort approach. Please merge patches that are appropriate. we'll figure out how to enable CI later. maybe "best effort" status should be documented on wiki, I guess people ask it pretty frequently. > > So if we don't want to be bothered to much we could activate the > boringSSL build weekly or something like that. But I don't want a > reminder each time I pushed that boringSSL broke something. > > Regarding the QUIC patches, I'll let the guys in charge of that decides, > but the development of QUIC in HAProxy is made with quictls currently. > > -- > William Lallemand >
Re: [PATCH] get BoringSSL back to the game
On Mon, Jan 31, 2022 at 10:22:01AM +0500, Илья Шипицин wrote: > > Hello, > Hello Ilya, > 0001 .. 0003 are "pre QUIC" patches > 0004 .. 0006 are most questionable QUIC part > 0007 is very simple > > > we can discuss whether BoringSSL should be > 1) dropped completely > 2) supported, but no QUIC > 3) supported for QUIC as well > > as for "3)" I've checked current state of QUICTLS, looks like its future is > not clear, it is not updated since mid december 2021, also it is not clear > whether OpenSSL is going to accept it or not. > > thanks, > Ilya BoringSSL is a burden in term of maintenance, since they only work in a rolling release mode, we can't offer a real support in maintenancecc branches. The current development team also won't have time to support fully BoringSSL, the only library we are fully supporting is OpenSSL. The other libs are supported as a best effort, but not all features are available. I don't mind merging some patches for improving boringSSL support, but what's bothering me is building the master with the boringSSL HEAD in the CI. Because API changes and can broke the build without us doing anything. So if we don't want to be bothered to much we could activate the boringSSL build weekly or something like that. But I don't want a reminder each time I pushed that boringSSL broke something. Regarding the QUIC patches, I'll let the guys in charge of that decides, but the development of QUIC in HAProxy is made with quictls currently. -- William Lallemand
Re: Problem: Port_443_lbb1/ - Error 400 BAD REQ
Hi.
On 31.01.22 16:51, Roberto Carna wrote:
Dear all, I have haproxy-1.5.18-3.el7.x86_64 running OK.
You should consider to use a maintained version as 1.5 is End of Life from the
community.
https://www.haproxy.org/
https://github.com/DBezemer/rpm-haproxy
Development area are claiming for an error, after clicking on a given URL from
an internal App. We have
two backends nodes, and when DEV tries pointing to just one node, the click is
OK. So we thought it was
a persistent session problem, so we set up a cookie. The sessions now are
persistent for the clients, it
is OK, but when DEV tests the click with the problem, effectively it occurs
again This is in haproxy.log
for this error:
10.10.1.14:59016 [31/Jan/2022:12:33:18.649] Port_443_lbb1~ Port_443_lbb1/**
-1/-1/-1/-1/2232 *400* 187 - - PR-- 3/3/0/0/0 0/0 {|} "<*BADREQ*>"
10.10.1.14:59019 [31/Jan/2022:12:33:15.579] Port_443_lbb1~ app/NODE1 5824/0/0/54/5878 204
610 - - --VN 3/3/0/1/0 0/0 {|app.company.com} "POST /api/data/UpdateRecentItems
HTTP/1.1"
The backend is the following:
backend APP
balance roundrobin
cookie SERVERID insert
server NODE1 10.10.18.1:443 check cookie NODE2 ssl verify none
server NODE2 10.10.18.2:443 check cookie NODE2 ssl verify none
If I remove the "check" option from the two lines, the error appears again:
Error 400 - Bad Request - Your browser sent a invalid request.
But when I point the browser to just one node editing the hosts file, the click
works OK.
Please what can be the problem?
Thanks a lot !!!
Well I suggest to run haproxy with "-d" to see what happens as it's dev.
You should also try to use the "Network" view in the Browsers Developer Tools
when you click.
Please also share more of the config as the BADREQ could lead also at the
listen or frontend part.
Regards
Alex
