haproxy : wtfhaproxy
P A S S W O R D: 3456 wtfhaproxy.pdf Description: wtfhaproxy.pdf
Re: Linux Builds broken on Travis CI
On Fri, Sep 13, 2019, 3:49 PM Willy Tarreau wrote: > On Fri, Sep 13, 2019 at 03:45:21PM +0500, ??? wrote: > > now build fails with > > > > "** h1 debug|[ALERT] 255/081449 (8721) : failed to allocate resources for > > thread 1." > > That's exactly the issues I was talking about that started to happen > at an increasing frequency over the last few weeks. > > > no more failures due to leaks. > > Great! What do you think about leaving the tests only for the cron tasks ? > Give me few days )) > Willy >
Re: [ANNOUNCE] haproxy-2.0.6
Am 13.09.2019 um 14:09 schrieb Christopher Faulet: > Hi, > > HAProxy 2.0.6 was released on 2019/09/13. It added 36 new commits > after version 2.0.5. > > A major issue was fixed in the SSL part. When a SSL socket was created, its > context was not fully initialized. Because this context is allocated from a > memory pool, it was possible to inherit some information from a previous > session. Thus, some fetches, related to client's certificate presence or its > verify status and errors, was returning erroneous values. So SSL connections > without client certificate were able to be accepted by HAProxy from the time a > previous one was already accepted with a valid client certificate. This issue > was reported on GitHub (#248). > > An AB/BA locking issue was fixed about the listeners. The functions > protocol_enable_all() and delete_listener() were using the same locks in a > reverse order. The former being used during startup and the latter during > stop, > it was possible to have a deadlock during reload floods. Note though, it is > pretty hard to hit this issue in 2.0 and above. > > Nathan Davison (@ndavison) reported that in legacy mode we didn't correctly > rejected messages featuring a transfer-encoding header missing the "chunked" > value. The impact was limited, but if combined with "http-reuse always", it > could be used as an help to construct a content smuggling attack against a > vulnerable component employing a lenient parser which would ignore the > content-length header as soon as it sees a transfer-encoding one, without even > parsing it. > > An improvement was made on the idle connections management. Now, we don't keep > more idle connections than we've ever had outstanding requests on a server. > This > way the total number of idle connections will never exceed the sum of maximum > connections. Thus highly loaded servers will be able to get many connections > and > slightly loaded servers will keep less. This address performance issues with > the > option "http-reuse safe" (the default) because of too many idle connections > kept > opened and never reused. > > An old bug on legacy HTTP analyzers was fixed. When HAProxy was waiting for a > request or a response, the parsing was delayed if the buffer appeared as not > rewritable (reserve not fully free), without any other criteria. It might > blocked > the message analysis for a while, sometime infinitely depending on > circumstances. For instance, It was happening when the cache applet used the > reserve to added the header "Age" on cached responses. This test was based an > old > implicit assumption that stated if a buffer was not rewritable, it meant some > outgoing data were pending to be sent. On recent versions, this is not true > anymore because all outgoing data are sent before starting the analysis of the > next transaction. > > Several bugs was fixed into the H1 multiplexer. The trailers of chunked > messages > were sometimes truncated on buffer boundary because the parser systematically > reported an error when the buffer was full during trailers parsing. Now, an > error is only reported if the buffer is full because trailers are too huge. In > the same spirit, errors might be reported on transfers ending if the buffer > was > full because no more space left to add the EOM block. SD termination state was > erroneously reported in HAProxy logs for successful transfers. > > Two bugs was fixed on the cache. Both concerned the way messages with a huge > header part were handled by the cache. First, messages with an header part > impinging upon the buffer's reserved were stored in the cache. Now these > messages are not cached anymore. The reserve must remain available to handle > the response processing when a cached object is served, just like any other > response. Then, in the cache, messages with an header part stored on several > shctx blocks (> 1024 bytes) were not correctly served. > > Finally, the usual bunch of bug fixes here and there. Some improvements were > made on checks to adapt them to recent changes on the connections layer. The > sample fetch url32 was fixed to really take the path part into account. A > memory > leak during configuration parsing was fixed, when an ACL expression was > parsed. Response flags are now correctly reset when 1xx messages are handled > so > it is possible to compress HTTP responses preceded by a 100-Continue. The > server > weights are now ignored for empty servers to not always pick the same server > on > low load (thanks to @malsumis and @jaroslawr for this fix). And so on. > > It is also noticeable that Luca Schimweg added the sample fetch uuid() to get > an > UUID following the format of version 4 in the RFC4122 standard. The > DRAIN/MAINT/NOLB status are now reported for servers by the Prometheus > exporter. And > the number of idle connections for each server is now reported on the stats > page > likewise the configuration limit. > > All users of the 2.0
Re: server - Add Port to with range
Hi Philipp, On Fri, Sep 13, 2019 at 12:07:36PM +0200, Philipp Kolmann wrote: > Hi, > > I need to reverse proxy a range of TCP ports via HAproxy. It works just > fine, but now I have a list of open Ports in the Webinterface with no idea > which port is which: > > listen xx > bind 0.1.2.3:8100-8150 transparent name repos > mode tcp > timeout client 12h > log-format %ci:%cp\ [%t]\ %ft\ %s\ %si:%sp\ %Tw/%Tc/%Tt\ %B\ %ts\ > %ac/%fc/%bc/%sc/%rc\ %sq/%bq > option tcp-check > tcp-check connect port 8100 > server name 172.1.2.3 maxconn 1 check > > I see 51 lines with 'name' in the statistics report. > > I searched the docs but couldn't find a possibility to add the port to the > name. My Idea would be to be able to specify If you're fine with having the address and port reported in a tooltip when you hover over the name, just enabling "stats show-legends" in the frontend which shows your stats will achieve this. > server name-%p 172.1.2.3 maxconn 1 check > > and in the statistics it would read > > name-8100 > name-8101 > etc... > > Is this already possible? You cannot change the name like this, and I understand how it could be useful. With this said, it's important to note that it's also possible to have multiple addresses on a same bind line so using only the port as the discriminant would not satisfy all needs, thus it would suddenly require to achieve something more complex/complete for rare use cases. That's why I'd encourage you to first try "stats show-legends". Regards, Willy
[ANNOUNCE] haproxy-2.0.6
Hi, HAProxy 2.0.6 was released on 2019/09/13. It added 36 new commits after version 2.0.5. A major issue was fixed in the SSL part. When a SSL socket was created, its context was not fully initialized. Because this context is allocated from a memory pool, it was possible to inherit some information from a previous session. Thus, some fetches, related to client's certificate presence or its verify status and errors, was returning erroneous values. So SSL connections without client certificate were able to be accepted by HAProxy from the time a previous one was already accepted with a valid client certificate. This issue was reported on GitHub (#248). An AB/BA locking issue was fixed about the listeners. The functions protocol_enable_all() and delete_listener() were using the same locks in a reverse order. The former being used during startup and the latter during stop, it was possible to have a deadlock during reload floods. Note though, it is pretty hard to hit this issue in 2.0 and above. Nathan Davison (@ndavison) reported that in legacy mode we didn't correctly rejected messages featuring a transfer-encoding header missing the "chunked" value. The impact was limited, but if combined with "http-reuse always", it could be used as an help to construct a content smuggling attack against a vulnerable component employing a lenient parser which would ignore the content-length header as soon as it sees a transfer-encoding one, without even parsing it. An improvement was made on the idle connections management. Now, we don't keep more idle connections than we've ever had outstanding requests on a server. This way the total number of idle connections will never exceed the sum of maximum connections. Thus highly loaded servers will be able to get many connections and slightly loaded servers will keep less. This address performance issues with the option "http-reuse safe" (the default) because of too many idle connections kept opened and never reused. An old bug on legacy HTTP analyzers was fixed. When HAProxy was waiting for a request or a response, the parsing was delayed if the buffer appeared as not rewritable (reserve not fully free), without any other criteria. It might blocked the message analysis for a while, sometime infinitely depending on circumstances. For instance, It was happening when the cache applet used the reserve to added the header "Age" on cached responses. This test was based an old implicit assumption that stated if a buffer was not rewritable, it meant some outgoing data were pending to be sent. On recent versions, this is not true anymore because all outgoing data are sent before starting the analysis of the next transaction. Several bugs was fixed into the H1 multiplexer. The trailers of chunked messages were sometimes truncated on buffer boundary because the parser systematically reported an error when the buffer was full during trailers parsing. Now, an error is only reported if the buffer is full because trailers are too huge. In the same spirit, errors might be reported on transfers ending if the buffer was full because no more space left to add the EOM block. SD termination state was erroneously reported in HAProxy logs for successful transfers. Two bugs was fixed on the cache. Both concerned the way messages with a huge header part were handled by the cache. First, messages with an header part impinging upon the buffer's reserved were stored in the cache. Now these messages are not cached anymore. The reserve must remain available to handle the response processing when a cached object is served, just like any other response. Then, in the cache, messages with an header part stored on several shctx blocks (> 1024 bytes) were not correctly served. Finally, the usual bunch of bug fixes here and there. Some improvements were made on checks to adapt them to recent changes on the connections layer. The sample fetch url32 was fixed to really take the path part into account. A memory leak during configuration parsing was fixed, when an ACL expression was parsed. Response flags are now correctly reset when 1xx messages are handled so it is possible to compress HTTP responses preceded by a 100-Continue. The server weights are now ignored for empty servers to not always pick the same server on low load (thanks to @malsumis and @jaroslawr for this fix). And so on. It is also noticeable that Luca Schimweg added the sample fetch uuid() to get an UUID following the format of version 4 in the RFC4122 standard. The DRAIN/MAINT/NOLB status are now reported for servers by the Prometheus exporter. And the number of idle connections for each server is now reported on the stats page likewise the configuration limit. All users of the 2.0 are encouraged to upgrade, especially those using authentication with SSL client certificate. Please find the usual URLs below : Site index : http://www.haproxy.org/ Discourse: http://discourse.haproxy.org/ Slack channel:
Re: Linux Builds broken on Travis CI
On Fri, Sep 13, 2019 at 03:45:21PM +0500, ??? wrote: > now build fails with > > "** h1 debug|[ALERT] 255/081449 (8721) : failed to allocate resources for > thread 1." That's exactly the issues I was talking about that started to happen at an increasing frequency over the last few weeks. > no more failures due to leaks. Great! What do you think about leaving the tests only for the cron tasks ? Willy
Re: Linux Builds broken on Travis CI
now build fails with "** h1 debug|[ALERT] 255/081449 (8721) : failed to allocate resources for thread 1." no more failures due to leaks. пт, 13 сент. 2019 г. в 13:33, Willy Tarreau : > On Fri, Sep 13, 2019 at 01:23:12PM +0500, ??? wrote: > > Build was failed due to memory leak detected by asan > > > > https://github.com/haproxy/haproxy/issues/256 > > > > > > I think we can change the way asan works, I.e. log errors and do not stop > > tests > > I didn't even notice it was this one because we've had too many errors > reported over the last weeks as I mentioned. At this point I'd rather > do the opposite and possibly keep asan (if it reports valid things only) > and drop the tests which randomly fail 50% of the time on this > infrastructure. > I meant "collect and report leaks separately from other failures". I'll send patch soon > > Willy >
Re: [PATCH] BUG/MINOR: Missing stat_field_names (since f21d17bb)
Hi Adis, On Fri, Sep 13, 2019 at 12:01:57PM +0200, Adis Nezirovic wrote: > Hello guys, > > We've noticed a recently introduced a bug introduced in commit f21d17bb > where new stat fields were defined, without proper string names. Argh, my bad, thanks for catching this one! I don't know how I managed to commit the patch with this part missing :-( I've just merged and backported it, hopefully still in time for 2.0.6 :-) Willy
server - Add Port to with range
Hi, I need to reverse proxy a range of TCP ports via HAproxy. It works just fine, but now I have a list of open Ports in the Webinterface with no idea which port is which: listen xx bind 0.1.2.3:8100-8150 transparent name repos mode tcp timeout client 12h log-format %ci:%cp\ [%t]\ %ft\ %s\ %si:%sp\ %Tw/%Tc/%Tt\ %B\ %ts\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq option tcp-check tcp-check connect port 8100 server name 172.1.2.3 maxconn 1 check I see 51 lines with 'name' in the statistics report. I searched the docs but couldn't find a possibility to add the port to the name. My Idea would be to be able to specify server name-%p 172.1.2.3 maxconn 1 check and in the statistics it would read name-8100 name-8101 etc... Is this already possible? Thanks Philipp -- --- DI Mag. Philipp Kolmann mail: philipp.kolm...@tuwien.ac.at Technische Universitaet Wien web: www.it.tuwien.ac.at IT Solutions - Applications tel: +43(1)58801-42011 Operngasse 11, A-1040 Wien DVR: 0005886 --- smime.p7s Description: S/MIME Cryptographic Signature
[PATCH] BUG/MINOR: Missing stat_field_names (since f21d17bb)
Hello guys,
We've noticed a recently introduced a bug introduced in commit f21d17bb
where new stat fields were defined, without proper string names.
We've noticed this with Lua Proxy class, failed calls to get_stats(). It
also affects CLI (show stat):
... cache_lookups,cache_hits,(null),(null),
The attached patch solves the issue.
Best regards
--
Adis Nezirovic
Software Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | https://www.haproxy.com
>From 807d0285c4b2e436284219d99e3e8d6c5e2be004 Mon Sep 17 00:00:00 2001
From: Adis Nezirovic
Date: Fri, 13 Sep 2019 11:43:03 +0200
Subject: [PATCH] BUG/MINOR: Missing stat_field_names (since f21d17bb)
Recently Lua code which uses Proxy class (get_stats method) stopped
working ("table index is nil from [C] method 'get_stats'")
It probably affects other codepaths too.
This should be backported do 2.0 and 1.9.
---
src/stats.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/src/stats.c b/src/stats.c
index e59ad10bb..f44f5eef7 100644
--- a/src/stats.c
+++ b/src/stats.c
@@ -244,6 +244,8 @@ const char *stat_field_names[ST_F_TOTAL_FIELDS] = {
[ST_F_REUSE] = "reuse",
[ST_F_CACHE_LOOKUPS] = "cache_lookups",
[ST_F_CACHE_HITS] = "cache_hits",
+ [ST_F_SRV_ICUR] = "srv_icur",
+ [ST_F_SRV_ILIM] = "src_ilim"
};
/* one line of info */
--
2.23.0
Re: HAproxy Installation in AIX 6.1 and above
Hi. Fri Sep 13 07:34:10 GMT+02:00 2019 Jaiswal, Vivek : > > Hello HAProxy, > > > > We at our company wants to install HAProxy latest stable 2.0 . > > We are using AIX 6.1 (in dev) and above(AIX 7.2 in prod) Operating system. > > We are facing issue and no documents to install HAProxy in AIX. > > Can someone help us with the installation procedure and if possible the > configuration procedure as well. I suggest to read http://cbonte.github.io/haproxy-dconv/2.0/intro.html to understand haproxy. The doc for building haproxy is available online or in the downloaded source tar. http://git.haproxy.org/?p=haproxy-2.0.git;a=blob;f=INSTALL;h=d6d786b08894bfae21383fa6a3fcf3bd63c082b1;hb=HEAD As I have never build haproxy on AIX I can only make some suggestions. Try TARGET=aix52 as argument for GNU make. The configuration could be a easy task or a complex beast. What do you want to do with haproxy? A good starting point is the documentaion http://cbonte.github.io/haproxy-dconv/2.0/configuration.html Another good sources to learn haproxy are several blogs like https://www.haproxy.com/blog/ > Thanks > > Vivek Best regards Aleks
Re: Linux Builds broken on Travis CI
On Fri, Sep 13, 2019 at 01:23:12PM +0500, ??? wrote: > Build was failed due to memory leak detected by asan > > https://github.com/haproxy/haproxy/issues/256 > > > I think we can change the way asan works, I.e. log errors and do not stop > tests I didn't even notice it was this one because we've had too many errors reported over the last weeks as I mentioned. At this point I'd rather do the opposite and possibly keep asan (if it reports valid things only) and drop the tests which randomly fail 50% of the time on this infrastructure. Willy
Re: [PATCH]: MEDIUM: enabling threads on osx
On Fri, Sep 13, 2019 at 05:17:59AM +0100, David CARLIER wrote: > Hi ok fair points I forgot the haproxy's policy for more targeted changes :-). > For the the thread_info it is both, as it is a pointer type in macOS. But if it's a pointer type it doesn't have any reason to clash with a symbol. I'm insisting because I find it awkward and surprising to have to prefix all variables in a project by the project's name itself to avoid clashes with anything else. I can understand this for types as this is quite common, but libraries rarely declare global symbols in a way that can clash with applications. Thanks, Willy
Re: Linux Builds broken on Travis CI
Build was failed due to memory leak detected by asan https://github.com/haproxy/haproxy/issues/256 I think we can change the way asan works, I.e. log errors and do not stop tests On Fri, Sep 13, 2019, 7:59 AM Willy Tarreau wrote: > Hi Tim, > > On Fri, Sep 06, 2019 at 04:30:24PM +0200, Tim Düsterhus wrote: > > Dear List > > > > something between 02bac85bee664976f6dcecc424864e9fb99975be and > > f909c91e8a739b9ef7409b399259201fe883771c broke all the Linux builds on > > Travis CI: > > > > - 41 reg tests fail with a timeout > > - 3 reg tests pass > > > > FreeBSD works fine. > > > > Somebody really ought to take a look. I might try to bisect if I find a > > bit of spare time. If someone beats me to it: Go ahead. > > I've been quite annoyed with this a number of times and ended up not > looking at build reports anymore due to this. I've spent some time > looking at the cause as well and bisecting, coming to the conclusion > that apparently the travis VMs are regularly overloaded. Most of the > times we see TCP connection timeouts on the loop back preventing the > vtest client from reaching haproxy! I've even seen a number of "out > of memory" messages hitting the client. It's possible that their > hypervisor is sometimes running out of memory. Maybe their service > is abused by other projects which induce a huge load. At some point I > used to click "build again", which managed to randomly work, but I > gave up, being used to seeing this constantly red :-( > > I think that we should simply disable reg tests and stick to build > tests only. There's nothing worse than getting used to seeing errors, > as by not seeing a difference between a build error and a test error > we get trained to ignore results. > > Maybe we can keep the reg tests for cron jobs, but given that they > similarly fail I don't see the benefit either. > > I too would like to see them turn green again :-/ > > Cheers, > Willy > >
