Re: [PATCH] BUG/MEDIUM: tools: fix build with static only toolchains

2020-07-23 Thread Willy Tarreau
On Fri, Jul 24, 2020 at 07:52:20AM +0300, Baruch Siach wrote:
> uClibc toolchains built with no dynamic library support don't provide
> the dlfcn.h header. That leads to build failure:
> 
>   CC  src/tools.o
> src/tools.c:15:10: fatal error: dlfcn.h: No such file or directory
>  #include 
>   ^
> Enable dladdr only when USE_DL is defined.

Oh thank you, I've encountered it as well a few days ago while working
on something else and forgot to note it on my todo list! However, I'm
looking at the original commit which introduced this and it's almost a
revert. We used to rely on USE_DL and turned it into __ELF__ because
some platforms don't need USE_DL.

I wanted to turn to sets of WANT/NEED/HAVE instead of the generic USE,
and I think this one will be one of the first benefiting from this. I'll
have a look at this.

Thanks!
Willy



[PATCH] BUG/MEDIUM: tools: fix build with static only toolchains

2020-07-23 Thread Baruch Siach
uClibc toolchains built with no dynamic library support don't provide
the dlfcn.h header. That leads to build failure:

  CC  src/tools.o
src/tools.c:15:10: fatal error: dlfcn.h: No such file or directory
 #include 
  ^
Enable dladdr only when USE_DL is defined.
---
Please Cc me on replies. I'm not subscribed.
---
 src/tools.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/src/tools.c b/src/tools.c
index 1c664852ad73..0bd80f846d05 100644
--- a/src/tools.c
+++ b/src/tools.c
@@ -10,7 +10,7 @@
  *
  */
 
-#ifdef __ELF__
+#if defined(__ELF__) && defined(USE_DL)
 #define _GNU_SOURCE
 #include 
 #include 
@@ -4410,7 +4410,7 @@ const char *get_exec_path()
return ret;
 }
 
-#ifdef __ELF__
+#if defined(__ELF__) && defined(USE_DL)
 /* calls dladdr() or dladdr1() on  and . If dladdr1 is available,
  * also returns the symbol size in , otherwise returns 0 there.
  */
@@ -,7 +,7 @@ static int dladdr_and_size(const void *addr, Dl_info 
*dli, size_t *size)
  * The file name (lib or executable) is limited to what lies between the last
  * '/' and the first following '.'. An optional prefix  is prepended 
before
  * the output if not null. The file is not dumped when it's the same as the one
- * that contains the "main" symbol, or when __ELF__ is not set.
+ * that contains the "main" symbol, or when __ELF__ && USE_DL are not set.
  *
  * The symbol's base address is returned, or NULL when unresolved, in order to
  * allow the caller to match it against known ones.
@@ -4472,7 +4472,7 @@ const void *resolve_sym_name(struct buffer *buf, const 
char *pfx, void *addr)
 #endif
};
 
-#ifdef __ELF__
+#if defined(__ELF__) && defined(USE_DL)
Dl_info dli, dli_main;
size_t size;
const char *fname, *p;
@@ -4489,7 +4489,7 @@ const void *resolve_sym_name(struct buffer *buf, const 
char *pfx, void *addr)
}
}
 
-#ifdef __ELF__
+#if defined(__ELF__) && defined(USE_DL)
/* Now let's try to be smarter */
if (!dladdr_and_size(addr, , ))
goto unknown;
@@ -4529,7 +4529,7 @@ const void *resolve_sym_name(struct buffer *buf, const 
char *pfx, void *addr)
chunk_appendf(buf, "+%#lx", (long)(addr - dli.dli_fbase));
return NULL;
}
-#endif /* __ELF__ */
+#endif /* __ELF__ && USE_DL */
  unknown:
/* unresolved symbol from the main file, report relative offset to main 
*/
if ((void*)addr < (void*)main)
-- 
2.27.0




Question on multiple contributions to hckrnews.com...

2020-07-23 Thread QV Credit
Hi,

Hope you're staying safe and well! 
Did you manage to take a look at my previous email?
In a nutshell, I am seeking to contribute multiple informative articles to your 
site, hckrnews.com. If this sounds appealing to you, I’d love to have a 
conversation about how we could make it work.
On that note, a long term partnership is the ideal goal in mind and would love 
to hear back from you !
Thanks gratefully,
KathyDon't want emails from us anymore? Reply to this email with the word 
"UNSUBSCRIBE" in the subject line.


Re: Keepalive with backends

2020-07-23 Thread Willy Tarreau
Hello Zefferno,

On Thu, Jul 23, 2020 at 06:30:47PM +0300, Zefferno wrote:
> GET /demo HTTP/1.1
> Host: 
> User-Agent: curl/
  
(...)
> As can be seen in the last packet, HAproxy immediately closes the
> connection with [RST, ACK] after sending [ACK], and I'm expecting it to
> keep the connection alive for a few seconds.
> 
> Is it normal behaviour?

I suspect you're sending a single request with curl and closing the client
connection. In 1.8, the backend connection was associated with a client
connection. So if you send a single request then close the client side,
haproxy couldn't keep the connection open as it had nowhere to attach it.
This changed in 1.9 with the introduction of server pools, which became
used by default in 2.0. Note however that even in 2.0 connections could
only be shared among the same thread. It's only the recently released 2.2
that permits to share idle connections between threads. In this case you
could really connect multiple times with curl and see the server-side
connection being reused (provided you use "http-reuse always", as the
default is "safe", which consists in not taking risks for the first
request of a connection).

Regards,
Willy



Re: How to debug matching ACLs?

2020-07-23 Thread Willy Tarreau
Hello Ricardo,

On Thu, Jul 23, 2020 at 05:52:55PM +0200, Ricardo Fraile wrote:
> Hello,
> 
> On a complex configuration with multiples ACLs, is there a way to debug what
> of them are applied over a request?

Depending how you write them (and what version you are using), you may
be able to use the "debug" converter at the end of the expression to
capture the value and send it to a ring buffer that you can consult
elsewhere. You can also copy a value into a variable that you append
in the log.

> Is it possible to append the unique id of the ACLs to the line on the log?

Not really, the problem is that there isn't a single set of ACL, but a large
set which apply to many actions. I've seen configurations evaluating up to
6000 ACLs per request, it's definitely not even realistic to log anything
in such a mess. And there's no real "this use case is more common, let's
focus on it only" because there are many rulesets. In addition, anonymous
ACLs are fairly common, and sometimes more readable.

Cyril Bonté had worked on such a solution many years ago. By then we had
very few rulesets, and as we were adding new ones with many ACLs evaluated
for a single request, it quickly became difficult to exploit.

That's why I really encourage you to instead set a variable and log it.
There's even a "set-var" converter that duplicates the expression's
value into a variable. Thus for example, instead of doing :

   use_backend foo if { path_beg /foo }

You could do:

   use_backend foo if { path,set-var(txn.last_expr) -m beg /foo }

And log "var(txn.last_expr)" to always get the last evaluated ACL among
those you have instrumented.

This makes me think that it could be convenient to have a new converter
called "here" that, just like set-var() or debug(), passes the value and
stores the current config file's line number into a fixed variable so
that you could have it present in your logs.

You could then do something like:

   use_backend foo if { path,here -m beg /foo }

If you'd place them on all final rules, you could easily detect what
rule was the one taking the final decision (redirect, deny, return,
etc). I suspect that we'd need a few sets, though (tcp-request,
http-request, tcp-response, http-response, use_backend, use-server).
But maybe in this case storing a history of the last 3-10 "here"
would help.

Note however that this would only be useful to know that a rule was
evaluated last. It doesn't guarantee that it matched, but if placed
on a final rule it will be reliable enough.

Hoping this helps, feel free to fuel the discussion with more ideas!

Willy



How to debug matching ACLs?

2020-07-23 Thread Ricardo Fraile

Hello,

On a complex configuration with multiples ACLs, is there a way to debug 
what of them are applied over a request?


Is it possible to append the unique id of the ACLs to the line on the 
log?



Thanks,



Keepalive with backends

2020-07-23 Thread Zefferno
Hello everyone,

I've an issue with HAProxy keeping persistent connections to a backend.
Running HAProxy 1.8.25 (latest build).

My testing config file looks like this:

global
chroot  /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 200
userhaproxy
group   haproxy
daemon
stats socket /var/lib/haproxy/stats

defaults
mode http
log global
optiondontlognull
optionlog-separate-errors

retries 3
option  redispatch
timeout client 30s
timeout connect 4s
timeout http-request 10s
timeout http-keep-alive 2s
timeout queue 5s
timeout tunnel 2m
timeout client-fin 1s
timeout server-fin 1s

frontend frontend
maxconn 200
mode http
bind *:443 
bind *:80

http-request set-header X-Connection Keep-Alive
http-request set-header Connection Keep-Alive
use_backend my_backend

backend my_backend
option http-keep-alive
default-server inter 1s
retries 3
timeout http-request 15s
timeout http-keep-alive 15s
server  :

This is what I see in the pcap:
1 2020-07-23 14:59:22.616924   TCP 74 64
0.0 33506 →  [SYN] Seq=0 Win=2048 Len=0 MSS=8961 SACK_PERM=1
TSval=2791663819 TSecr=0 WS=512
2 2020-07-23 14:59:22.617142   TCP 74 128
0.000218000  → 33506 [SYN, ACK] Seq=0 Ack=1 Win=8192 Len=0 MSS=1460
WS=256 SACK_PERM=1 TSval=285952824 TSecr=2791663819
3 2020-07-23 14:59:22.617159   TCP 66 64
0.17000 33506 →  [ACK] Seq=1 Ack=1 Win=2048 Len=0
TSval=2791663819 TSecr=285952824
4 2020-07-23 14:59:22.617186   HTTP 211 64
0.27000 GET /demo HTTP/1.1
--- HTTP level ---
GET /demo HTTP/1.1
Host: 
User-Agent: curl/
Accept: */*
X-Connection: Keep-Alive
Connection: Keep-Alive
---
5 2020-07-23 14:59:22.617436   TCP 66 128
0.00025  → 33506 [ACK] Seq=1 Ack=146 Win=2108160 Len=0
TSval=285952824 TSecr=2791663819
6 2020-07-23 14:59:22.624752   HTTP 581 128
0.007316000 HTTP/1.1 200 OK  (text/javascript)
--- HTTP level ---
HTTP/1.1 200 OK
Date: Thu, 23 Jul 2020 14:59:22 GMT
Content-Type: text/javascript; charset=utf-8
Server: 
Content-Length: 198
Cache-Control: private
---
7 2020-07-23 14:59:22.624756   TCP 66 64
0.04000 33506 →  [ACK] Seq=146 Ack=516 Win=3584 Len=0
TSval=2791663827 TSecr=285952831
8 2020-07-23 14:59:22.777082   TCP 66 64
0.152326000 33506 →  [RST, ACK] Seq=146 Ack=516 Win=3584 Len=0
TSval=2791663979 TSecr=285952831

As can be seen in the last packet, HAproxy immediately closes the
connection with [RST, ACK] after sending [ACK], and I'm expecting it to
keep the connection alive for a few seconds.

Is it normal behaviour?

Why HA doesn't keep the connection alive?

Thanks,
Zeffy


Re: Load testing HAProxy 2.2 on x86_64 and aarch64 VMs

2020-07-23 Thread Илья Шипицин
чт, 23 июл. 2020 г. в 19:28, Martin Grigorov :

> Hi Илья,
>
> I didn't have much success with Google Perf Tools.
> I've tried both with adding '-lprofiler' to LDFLAGS when building HAProxy
> and with LD_PRELOAD.
> In both cases on ARM64 it fails with: terminated by signal SIGSEGV
> (Address boundary error)
>
> On x86_64 there is no such issue but the produced result file is always
> empty.
>


I can try on your hardware if you  give me an access,


>
> $ ldd haproxy
> linux-vdso.so.1 (0x7ffcfb5cd000)
> libprofiler.so.0 => /usr/local/lib/libprofiler.so.0
> (0x7f2115507000)<<
> libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1
> (0x7f21154cc000)
> libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7f21154b)
> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f21154aa000)
> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x7f211549f000)
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x7f211547c000)
> libssl.so.1.1 => /home/ubuntu/opt/lib/libssl.so.1.1
> (0x7f21151e6000)
> libcrypto.so.1.1 => /home/ubuntu/opt/lib/libcrypto.so.1.1
> (0x7f2114cf4000)
> liblua5.3.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.3.so.0
> (0x7f2114cb9000)
> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7f2114b6a000)
> libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0
> (0x7f2114abd000)
> libpcreposix.so.3 => /usr/lib/x86_64-linux-gnu/libpcreposix.so.3
> (0x7f2114ab8000)
> libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3
> (0x7f2114a43000)
> libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
> (0x7f2114a28000)
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f2114836000)
> libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8
> (0x7f2114819000)
> libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
> (0x7f2114638000)
> /lib64/ld-linux-x86-64.so.2 (0x7f211552e000)
> liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5
> (0x7f211460f000)
> liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1
> (0x7f21145ee000)
> libgcrypt.so.20 => /usr/lib/x86_64-linux-gnu/libgcrypt.so.20
> (0x7f21144d)
> libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0
> (0x7f21144ad000)
>
> CPUPROFILE=/tmp/haproxy-load-x64.prof haproxy -p pid.txt -f haproxy.cfg
>
> The above command creates /tmp/haproxy-load-x64.prof but it is never
> populated with any data. I've tried stopping HAProxy with signals INT, KILL
> and TERM.
> I even tried with env var CPUPROFILESIGNAL=12 to start/stop the profiler
> manually but again no success.
>
> I could share with you reports from Linux 'perf' command. Just let me know
> which events you'd be interested in!
>

perf is somewhat strange, I recall really weird reports from it. maybe
something was wrong with my hands.

yeah, let us have a look at perf from both arm64 and amd64



>
> Regards,
> Martin
>
> On Sat, Jul 18, 2020 at 11:40 AM Илья Шипицин 
> wrote:
>
>> Hello, Martin!
>>
>> Can you please compare load profiles using google perftools ?
>>
>> I never tried to use gperf on ARM64, also, my trial at Linaro is over, I
>> do not have an access to any ARM64 anymore.
>> in short, gperf can be found https://github.com/gperftools/gperftools
>>
>> please follow "CPU profiling part".
>>
>> it can collect cachegrind output, I attached example kcachegrind report
>> (you can sort by "self" time).
>> it would be interesting to compare amd64 <--> arm64
>>
>> [image: Screenshot from 2020-07-18 13-35-27.png]
>>
>>
>> пт, 10 июл. 2020 г. в 19:00, Martin Grigorov :
>>
>>> Hello HAProxy community,
>>>
>>> I wanted to compare how the newly released HAProxy 2.2 (Congrats!)
>>> behaves under heavy load so I've ran some tests on my x86_64 and aarch64
>>> VMs:
>>>
>>>
>>> https://medium.com/@martin.grigorov/compare-haproxy-performance-on-x86-64-and-arm64-cpu-architectures-bfd55d1d5566
>>>
>>> Without much surprise the x86_64 VM gave better results!
>>>
>>> It is *not* a real use case scenario: the backends serve on GET / and
>>> return "Hello World", without any file/network operations.
>>>
>>> What is interesting though is that I can get 120-160K reqs/sec when
>>> hitting directly one of the backend servers, and only 20-40K reqs/sec when
>>> using HAProxy as a load balancer in front of them.
>>>
>>> I'd be happy to re-run the tests with any kind of improvements you may
>>> have!
>>>
>>> Regards,
>>> Martin
>>>
>>


Re: Load testing HAProxy 2.2 on x86_64 and aarch64 VMs

2020-07-23 Thread Martin Grigorov
Hi Илья,

I didn't have much success with Google Perf Tools.
I've tried both with adding '-lprofiler' to LDFLAGS when building HAProxy
and with LD_PRELOAD.
In both cases on ARM64 it fails with: terminated by signal SIGSEGV (Address
boundary error)

On x86_64 there is no such issue but the produced result file is always
empty.

$ ldd haproxy
linux-vdso.so.1 (0x7ffcfb5cd000)
libprofiler.so.0 => /usr/local/lib/libprofiler.so.0
(0x7f2115507000)<<
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1
(0x7f21154cc000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7f21154b)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f21154aa000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x7f211549f000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x7f211547c000)
libssl.so.1.1 => /home/ubuntu/opt/lib/libssl.so.1.1
(0x7f21151e6000)
libcrypto.so.1.1 => /home/ubuntu/opt/lib/libcrypto.so.1.1
(0x7f2114cf4000)
liblua5.3.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.3.so.0
(0x7f2114cb9000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7f2114b6a000)
libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0
(0x7f2114abd000)
libpcreposix.so.3 => /usr/lib/x86_64-linux-gnu/libpcreposix.so.3
(0x7f2114ab8000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3
(0x7f2114a43000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x7f2114a28000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f2114836000)
libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8
(0x7f2114819000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x7f2114638000)
/lib64/ld-linux-x86-64.so.2 (0x7f211552e000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5
(0x7f211460f000)
liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1
(0x7f21145ee000)
libgcrypt.so.20 => /usr/lib/x86_64-linux-gnu/libgcrypt.so.20
(0x7f21144d)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0
(0x7f21144ad000)

CPUPROFILE=/tmp/haproxy-load-x64.prof haproxy -p pid.txt -f haproxy.cfg

The above command creates /tmp/haproxy-load-x64.prof but it is never
populated with any data. I've tried stopping HAProxy with signals INT, KILL
and TERM.
I even tried with env var CPUPROFILESIGNAL=12 to start/stop the profiler
manually but again no success.

I could share with you reports from Linux 'perf' command. Just let me know
which events you'd be interested in!

Regards,
Martin

On Sat, Jul 18, 2020 at 11:40 AM Илья Шипицин  wrote:

> Hello, Martin!
>
> Can you please compare load profiles using google perftools ?
>
> I never tried to use gperf on ARM64, also, my trial at Linaro is over, I
> do not have an access to any ARM64 anymore.
> in short, gperf can be found https://github.com/gperftools/gperftools
>
> please follow "CPU profiling part".
>
> it can collect cachegrind output, I attached example kcachegrind report
> (you can sort by "self" time).
> it would be interesting to compare amd64 <--> arm64
>
> [image: Screenshot from 2020-07-18 13-35-27.png]
>
>
> пт, 10 июл. 2020 г. в 19:00, Martin Grigorov :
>
>> Hello HAProxy community,
>>
>> I wanted to compare how the newly released HAProxy 2.2 (Congrats!)
>> behaves under heavy load so I've ran some tests on my x86_64 and aarch64
>> VMs:
>>
>>
>> https://medium.com/@martin.grigorov/compare-haproxy-performance-on-x86-64-and-arm64-cpu-architectures-bfd55d1d5566
>>
>> Without much surprise the x86_64 VM gave better results!
>>
>> It is *not* a real use case scenario: the backends serve on GET / and
>> return "Hello World", without any file/network operations.
>>
>> What is interesting though is that I can get 120-160K reqs/sec when
>> hitting directly one of the backend servers, and only 20-40K reqs/sec when
>> using HAProxy as a load balancer in front of them.
>>
>> I'd be happy to re-run the tests with any kind of improvements you may
>> have!
>>
>> Regards,
>> Martin
>>
>


Re: [ANNOUNCE] haproxy-2.2.1

2020-07-23 Thread Willy Tarreau
Hi Tim,

On Thu, Jul 23, 2020 at 02:23:08PM +0200, Tim Düsterhus wrote:
> Willy,
> 
> Am 23.07.20 um 09:46 schrieb Willy Tarreau:
> > these ones. I think we'll soon emit another 2.1 given that quite some fixes
> > have been accumulating since 2.1.7.
> 
> Any ETA for that?

Sadly, not. "Soon" is the best I can suggest.

Over the last few months, 100% of my work time has been spent chasing
bugs and emitting releases.  So if we continue on this trend, we'll
just feed the bug generator to make sure we have a lifetime job, but
a boring one. I've been strongly wanting to spend some time to address
some bug-generating architecture limitations for about two years now,
and while I'm totally convinced that one month spent on design saves
3 months of bugs the next year, we're still running after bugs of the
past instead of investing on the future :-(

> You also might want to do the final 1.9 as suggested previously:
> https://www.mail-archive.com/haproxy@formilux.org/msg37580.html. It
> would allow to remove the 1.9 label from the issue tracker and clean out
> quite a few already-fixed issues from the list.

Yep I noticed as well. I'll to do it as well, for the exact same reason
(clean up the backlog).

Cheers,
Willy



Re: http-reuse and Proxy protocol

2020-07-23 Thread Willy Tarreau
Hi Arnall,

On Tue, Jul 21, 2020 at 01:27:31PM +0200, Arnall wrote:
> Hello everyone,
> 
> I remember that in the past it was strongly discouraged to use http-reuse in
> combination with send-proxy, because of the client IP which is provided by
> the proxy protocol.
> 
> I have this configuration :
> 
> HA-Proxy version 2.0.14-1~bpo9+1 2020/04/16 - https://haproxy.org/
> 
> defaults
>     http-reuse always
> 
> backend abuse
>     timeout server 60s
>     balance roundrobin
>     hash-balance-factor 0
>     server s_abuse u...@abuse.sock send-proxy-v2 maxconn 4
> 
> listen l_abuse
>     bind u...@abuse.sock accept-proxy
>     http-request set-var(req.delay) int(500)
>     http-request lua.add_delay
>     server  192.168.000.aaa:80 maxconn 1
>     server  192.168.000.bbb:80  maxconn 1
>     server z 192.168.000.ccc:80  maxconn 1
> 
> Is it OK ? Because i have no warning when verifying the configuration, or
> should i add a "http-reuse never" in "backend abuse" ?

It is now properly dealt with, by marking the connection private, which
means it will not be shared at all. So what you'll see simply is that
there is no reuse for connections employing send-proxy. So your config
is safe, but you will just not benefit from the reuse.

Anyway it's generally not a good idea to use proxy protocol over HTTP
from an HTTP-aware agent. Better use Forward/X-Forwarded-for that passes
the info per request and that nowadays everyone can consume.

Regards,
Willy



Re: [ANNOUNCE] haproxy-2.2.1

2020-07-23 Thread Tim Düsterhus
Willy,

Am 23.07.20 um 09:46 schrieb Willy Tarreau:
> these ones. I think we'll soon emit another 2.1 given that quite some fixes
> have been accumulating since 2.1.7.

Any ETA for that?

You also might want to do the final 1.9 as suggested previously:
https://www.mail-archive.com/haproxy@formilux.org/msg37580.html. It
would allow to remove the 1.9 label from the issue tracker and clean out
quite a few already-fixed issues from the list.

Best regards
Tim Düsterhus



[ANNOUNCE] haproxy-2.2.1

2020-07-23 Thread Willy Tarreau
Hi,

HAProxy 2.2.1 was released on 2020/07/23. It added 29 new commits
after version 2.2.0.

This version addresses a small number of issues that were raised after the
release, a few of which are important enough to warrant an upgrade for all
2.2 users:
  - two crashes that can happen in the scheduler when using threads and
resolvers, reported by Tim in issue 758. This is a regression that I
introduced late in the development cycle while trying to address
scheduling issues. It only affects multi-threaded tasks (seems to be
resolvers only) whose timeout can be updated based on new activity.

  - the do-resolve action was not thread-safe (issue #236) and would
instantly segfault under load. In addition it was also able to leak
memory even when not using threads (issue #222). These ones were
already reported a year ago and affect 2.0 and possibly even as far
as 1.7.

  - actions, sample-fetches, converters taking a single optional argument
would not accept to be called with empty parenthesis anymore, as Luke
reported in issue #763. This was a 2.2 regression coming as a side
effect of the rewrite of the arg parser to support commas and quotes.
The problem is that it doesn't cause an error, it sees an empty
argument of the same type as the optional one, so an integer would be
zero, a string would be empty, etc. It can possibly cause non-obvious
issues depending on the use cases.

  - a bug in the way the channel code deals with shutdowns, causing endless
loops in Lua when peeking data from a channel (issue #744). The bug
itself is not a regression, but it wouldn't trigger on older versions
for scheduling reasons.

  - an end of stream signaling in HTX that causes MSG_MORE to be emitted
even on the final response from an FCGI server since 2.2 (issue #756).
The fix is not the prettiest we've done but it will be enough for the
time it takes to improve this area, and possibly backport something
better in the future.

  - idle-connections: FastCGI and H2 connections marked "private" were
mistakenly added to the shared list of available ones when they ought
not to.

  - a bug on HTTP/1 protocol upgrades in tunnel mode that broke WebSocket,
and also affects 2.1 post-2.1.7 (issue #737).

  - target servers in ring buffers, and peers declared using the "server"
keyword were not resolved if using an FQDN (issue #759). These would
simply cause a silent connection error.


A few other bugs were already present in older versions:
  - a small memory leak in fcgi-app that affects 2.1 and newer (issue #750).

  - a missing memory barrier in thread-safe lists that manages to cause
crashes in the scheduler only on ARM Cortex A53 (and probably other
weakly ordered CPUs).

  - logs: mixing "sampled" log servers with non-sampled ones would cause
the latter to lose logs due to an unexpected "static" variable retaining
this state during the config parsing.

  - Ilya addressed an FD leak that could happen with state files containing
errors (issues #660 and #738). This was apparently introduced in 2.1.

The rest is less important. There remains one bug on Fcgi, by which haproxy
would mistakenly decode the query string while it shouldn't (issue #769, also
affects 2.1). This will be addressed shortly.

Overall, it's not that bad at all. There definitely are some scary ones,
but not that many. I hope we'll have a bit of rest to restart to work on
addressing certain issues at the root instead of just their visible effect,
but the current trend seems encouraging.

In the mean time, every user of 2.2.0 should really upgrade to 2.2.1 to fix
these ones. I think we'll soon emit another 2.1 given that quite some fixes
have been accumulating since 2.1.7.

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Slack channel: https://slack.haproxy.org/
   Issue tracker: https://github.com/haproxy/haproxy/issues
   Sources  : http://www.haproxy.org/download/2.2/src/
   Git repository   : http://git.haproxy.org/git/haproxy-2.2.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-2.2.git
   Changelog: http://www.haproxy.org/download/2.2/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/

Willy
---
Complete changelog :
Christopher Faulet (12):
  BUG/MEDIUM: mux-h2: Don't add private connections in available connection 
list
  BUG/MEDIUM: mux-fcgi: Don't add private connections in available 
connection list
  BUG/MEDIUM: mux-h1: Continue to process request when switching in tunnel 
mode
  BUG/MINOR: mux-fcgi: Handle empty STDERR record
  BUG/MINOR: mux-fcgi: Set conn state to RECORD_P when skipping the record 
padding
  BUG/MINOR: mux-fcgi: Set flags on the right stream field for empty 
FCGI_STDOUT
  BUG/MEDIUM: channel: Be aware of SHUTW_NOW flag when output data 

Re: DOC: Update documentation / comments to be gender neutral

2020-07-23 Thread Tim Düsterhus
Jackie,

First: I'm a community contributor, so my review might not necessarily
reflect that of the HAProxy project. I'm also not a native English speaker.

Regarding your patch I have a few comments:

>From your PR message:

> Also, remove some trailing whitespaces from the files modified for funsies.

I disagree with this change. This makes the patch larger than it needs
to be and it distracts from the important parts of it. If anything it
should become a dedicated patch that is separately committed.

Regarding your commit messages:

Please have a look at the CONTRIBUTING file you modified. Commit
messages need to be prefixed with a "tag" ("DOC:" would probably be
appropriate) and also contain a body:

>As a rule of thumb, your patch MUST NEVER be made only of a subject line,
>it *must* contain a description. Even one or two lines, or indicating
>whether a backport is desired or not. It turns out that single-line commits
>are so rare in the Git world that they require special manual (hence
>painful) handling when they are backported, and at least for this reason
>it's important to keep this in mind.

(https://github.com/haproxy/haproxy/blob/f1ea47d8960730c79cc71fc634b3d7e5909d5683/CONTRIBUTING#L562-L567)

Then to the patch itself:

- It must not modify the license files doc/lgpl.txt and doc/gpl.txt,
because they are standard license texts:
https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
- In lua.txt it now reads "They just works carefully". I believe it
should be "They just work carefully".
- In proxy-protocol.txt it now reads "to hide it's activities". I
believe it should be "to hide its activities".

Best regards
Tim Düsterhus