Re: [ANNOUNCE] haproxy-1.6-dev6

[joris dedieu]

Hi,

2015-09-29 0:35 GMT+02:00 Willy Tarreau :
> Hi everyone,
>
> this is the end of a harrassing week! I wanted to issue dev6 last monday
> to have a calm week dedicated to bug fixes and documentation updates only
> and it ended up completely differently with numerous painful bugs rising
> at the same time while Thierry was testing his Lua update which uncovered
> a mess at the applet layer (well, shared between applets and Lua). After
> about 260 e-mails exchanged, thousands of tests and probably a lot of hair
> lost due to head scratching we ended up fixing all the remaining ones last
> night.
>
> So this version comes with a number of important and less important fixes,
> and still a few feature updates that despite the feature freeze were
> desirable to have before the release.
>
> Regarding the bugs first, all reported bugs and all the ones we found
> during the Lua vs applet debugging were fixed in this version, including
> the error on UDP sockets on FreeBSD, the issues causing Lua socket data
> to be truncated, other issues causing the CLI to sometimes ignore client
> disconnect and leak connections, and bugs affecting peers. The complete
> changelog below lists 134 patches among which 35 bug fixes. A few of these
> fixes will be backported to 1.5 as well.
>
> 22 patches concern doc updates, which is in line with our expectations for
> an approaching release. I have still not found the time to write the last
> missing doc piece allowing us to get rid of haproxy-{en,fr}.txt.
>
> Now regarding the last-minute changes that were merged :
>
>   - server-state conservation across reload that we've long been talking
> about was finally merged. Please check the backend directive
> load-server-state-from-file in the doc.
>
>   - cpu-map is now supported on FreeBSD.
>
>   - 51degress device identification changed their API to support last
> version (3.2). I didn't like this last-minute change but I understand
> that sometimes it is better to do that before the release than being
> forced to maintain an older API. The new implementation supports both
> a fetch method (to inspect all headers) and a converter (to inspect
> only a specific one). Please test this as the changes were important!
>
>   - DeviceAtlas also updated their module to support both a sample fetch
> function and a converter. Please test this as well, the changes were
> much smaller and I'm less worried though.
>
>   - Lua: change in the way actions are registered : instead of calling
> random functions from haproxy, only registered ones may be accessed,
> this is much safer to avoid namespace collisions over the long term
> and to avoid mistakes due to similar looking function names.
>
>   - Lua: do not limit socket addresses to IPv4/IPv6, support the same
> address classes as servers (including unix and abstract namespaces).
>
>   - Lua: add support for applet registration usable via the new
> "use-service" directive. This allows a script to process contents
> that are not limited to the size of a buffer anymore. It provides
> easy mapping for TCP and HTTP manipulation so that servers are easy
> to write. Thierry showed me that he could reimplement the haproxy
> stats page entirely in Lua using this, so that was definitely something
> to have before the release so that people don't feel limited anymore in
> what they can do in Lua.
>
>   - TCP actions: "silent-drop". Finally it got merged as the actions
> registration mechanism made it a no-brainer. It works like a deny except
> that it tries to prevent the TCP RST from reaching the client, so that's
> quite efficient against certain bots and scripts as their connections
> remain established on their side only. It works on Linux and could
> possibly work on other systems (not tested).

I can confirm that silent-drop is not working as expected on FreeBSD

listen drop
bind 80.247.233.40:2
tcp-request connection silent-drop

08:31:31.324885 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
[S], seq 1048805770, win 29200, options [mss 1460,sackOK,TS val
14874937 ecr 0,nop,wscale 7], length 0
08:31:31.324903 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
[S.], seq 510555620, ack 1048805771, win 65535, options [mss
1460,nop,wscale 6,sackOK,TS val 1100790208 ecr 14874937], length 0
08:31:31.367359 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
[.], ack 1, win 229, options [nop,nop,TS val 14874946 ecr 1100790208],
length 0
08:31:31.367425 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
[F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 1100790250 ecr
14874946], length 0
08:31:31.697612 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
[F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 1100790581 ecr
14874946], length 0
08:31:32.183981 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
[F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 

[PATCH]: BUG/MINOR: Wrong setsockopt for TTL fallback, "silent-drop" function

[David Carlier]

Hi,
This little patch avoids a build issue on systems different than Linux.

Hope it is useful.

Kindest regards.
From 24d0feb7981ebabf987ee881768eaf2aea685319 Mon Sep 17 00:00:00 2001
From: David CARLIER 
Date: Tue, 29 Sep 2015 14:48:28 +0100
Subject: [PATCH] BUG/MINOR: proto_tcp: Wrong setsockopt flag for other systems
 than Linux.

The SOL_IP flag is available only on Linux, whereas for Solaris and
*BSD flavors it is IPPROTO_IP causing build break in
the silent-drop function.
---
 src/proto_tcp.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/src/proto_tcp.c b/src/proto_tcp.c
index f698889..3642398 100644
--- a/src/proto_tcp.c
+++ b/src/proto_tcp.c
@@ -1456,7 +1456,11 @@ static enum act_return tcp_exec_action_silent_drop(struct act_rule *rule, struct
 	 * network and has no effect on local net.
 	 */
 #ifdef IP_TTL
+#ifdef SOL_IP
 	setsockopt(conn->t.sock.fd, SOL_IP, IP_TTL, , sizeof(one));
+#else
+	setsockopt(conn->t.sock.fd, IPPROTO_IP, IP_TTL, , sizeof(one));
+#endif
 #endif
  out:
 	/* kill the stream if any */
-- 
2.5.1

[PATCH] BUILD: IP_TTL: Fix compilation on almost FreeBSD and OpenBSD.

[Joris Dedieu]

IP_TTL socket option is defined on some systems that don't have SOL_IP.
Use IPPROTO_IP in this case.
---
 src/proto_tcp.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/src/proto_tcp.c b/src/proto_tcp.c
index f698889..3642398 100644
--- a/src/proto_tcp.c
+++ b/src/proto_tcp.c
@@ -1456,7 +1456,11 @@ static enum act_return 
tcp_exec_action_silent_drop(struct act_rule *rule, struct
 * network and has no effect on local net.
 */
 #ifdef IP_TTL
+#ifdef SOL_IP
setsockopt(conn->t.sock.fd, SOL_IP, IP_TTL, , sizeof(one));
+#else
+   setsockopt(conn->t.sock.fd, IPPROTO_IP, IP_TTL, , sizeof(one));
+#endif
 #endif
  out:
/* kill the stream if any */
-- 
2.3.6

Re: segfault writing to log from Lua

[Dragan Dosen]

Hi,

Thank you for reporting this bug. I'll fix it as soon as possible.


Best regards,
Dragan Dosen


On 29.09.2015 14:01, Thierry FOURNIER wrote:
> Michael, thank you for the bug repport. I reproduce it.
> 
> It is introduced with the support of the RFC5424 for the logs format
> sublitted by dragan.
> 
> Hi Dragan, after your patch about the RFC5424, the funcion __send_log()
> no longer support NULL for the "*p" param. Can you fix this ?
> 
> It seems that the 'log_htp' and the 'log_htp_rfc5424' members of the
> "struct proxy" contains preformatted strings. There makeby the function 
> "lf_host_tag_pid()".
> 
> Maybe, we should build a defaults strings (one for each rfc) at the
> start of haproxy, and uses these default strings if the proxy "*p" is
> NULL in the function "__send_log()" ?
> 
> Thierry
> 
> 
> 
> 
> On Tue, 29 Sep 2015 07:37:12 -0400
> Michael Ezzell  wrote:
> 
>> Although I am seeing this in Lua, it does not appear (to me) to a the fault
>> in the Lua code... recent changes in __send_log() in log.c appear to assume
>> p is never null (there's a test for this condition earlier, then p is
>> referenced later), and that is not valid behavior in this case, because
>> there is no proxy associated:
>>
>> (gdb) run
>> Starting program: /home/ubuntu/haproxy-1.6-dev6/haproxy -f ./haproxy.cfg
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> __send_log (p=p@entry=0x0, level=level@entry=1, message=,
>> size=10,
>> sd=sd@entry=0x711160  "- ",
>> sd_size=sd_size@entry=2) at src/log.c:1025
>> 1025if (unlikely(htp->len >= maxlen)) {
>> (gdb) print htp->len
>> Cannot access memory at address 0xda4
>> (gdb) print htp
>> $1 = (struct chunk *) 0xd98
>> (gdb) print >log_htp
>> $2 = (struct chunk *) 0xd98
>> (gdb) print p
>> $3 = (struct proxy *) 0x0
>> (gdb)
>>
>> This minimal test case can be replicated with the following two files:
>>
>> $ cat haproxy.cfg
>>
>> global
>> log 127.0.0.1 local0
>> lua-load crash.lua
>> user haproxy
>> group nogroup
>> daemon
>>
>> defaults
>> log global
>>
>> $ cat crash.lua
>>
>> core.Alert("hello.lua");
> 

Lua usage question - log format variables

[Michael Ezzell]

We have "fetches" accessible from Lua, but I have not found a mechanism
that allows me to access the value of log format variables.

Examples of useful variables:

%b   | backend_name
%f   | frontend_name
%rt  | request_counter

Is there a way to access values like these from Lua, since they aren't (as
far as I can tell) available as fetches?

I realize not all variables would be available in every context, and I
don't want to modify them, of course -- I only want to read them -- but I
would like to be able to read them and use them in content generated in Lua.

Are they accessible?

Re: [ANNOUNCE] haproxy-1.6-dev6

[joris dedieu]

Hi Willy

2015-09-29 13:59 GMT+02:00 Willy Tarreau :
> Hi Joris,
>
> On Tue, Sep 29, 2015 at 08:56:54AM +0200, joris dedieu wrote:
>> >   - TCP actions: "silent-drop". Finally it got merged as the actions
>> > registration mechanism made it a no-brainer. It works like a deny 
>> > except
>> > that it tries to prevent the TCP RST from reaching the client, so 
>> > that's
>> > quite efficient against certain bots and scripts as their connections
>> > remain established on their side only. It works on Linux and could
>> > possibly work on other systems (not tested).
>>
>> I can confirm that silent-drop is not working as expected on FreeBSD
>>
>> listen drop
>> bind 80.247.233.40:2
>> tcp-request connection silent-drop
>>
>> 08:31:31.324885 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
>> [S], seq 1048805770, win 29200, options [mss 1460,sackOK,TS val
>> 14874937 ecr 0,nop,wscale 7], length 0
>> 08:31:31.324903 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
>> [S.], seq 510555620, ack 1048805771, win 65535, options [mss
>> 1460,nop,wscale 6,sackOK,TS val 1100790208 ecr 14874937], length 0
>> 08:31:31.367359 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
>> [.], ack 1, win 229, options [nop,nop,TS val 14874946 ecr 1100790208],
>> length 0
>> 08:31:31.367425 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
>> [F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 1100790250 ecr
>> 14874946], length 0
> (...)
>> [F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 1100817450 ecr
>> 14874946], length 0
>> 08:32:22.886834 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
>> [P.], seq 1:7, ack 1, win 229, options [nop,nop,TS val 14887826 ecr
>> 1100790208], length 6
>> 08:32:22.886850 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
>> [R], seq 510555621, win 0, length 0
>
> Thanks for your feedback. The fact that the FIN is retranmsmitted like
> this tends to confirm that the TTL setting works. However I feel quite
> concerned about the fact that a FIN was emitted instead of a reset. I
> fear that SO_LINGER doesn't work as expected which is much more of a
> problem if that happens on the server side too when closing a server
> connection!
>
> Could you please run the same test under strace/truss/whatever you find
> equivalent on your platform ?
>
> I'd be interested in seeing each syscall status.

kevent(3,0x0,0,{},5,{1.0 }) = 0 (0x0)
kevent(3,0x0,0,{0x4,EVFILT_READ,0x0,0,0x1,0x0},5,{1.0 }) = 1 (0x1)
accept(4,{ AF_INET 80.247.233.242:48068 },0x7fffe804) = 5 (0x5)
fcntl(5,F_SETFL,O_NONBLOCK) = 0 (0x0)
recvfrom(5,0x801407000,16384,0x20080,0x0,0x0) ERR#35 'Resource
temporarily unavailable'
setsockopt(0x5,0x0,0x4,0x48668c,0x4,0x0) = 0 (0x0)
setsockopt(0x5,0x6,0x1,0x48668c,0x4,0x0) = 0 (0x0)
accept(4,0x7fffe808,0x7fffe804) ERR#35 'Resource
temporarily unavailable'
shutdown(5,SHUT_WR) = 0 (0x0)
close(5) = 0 (0x0)

Has you can see it doesn't looks great.

Joris

>
> Thanks,
> Willy
>

Re: segfault writing to log from Lua

[Thierry FOURNIER]

ps: a temporary workaround waiting for the fix, is to add the following
line at the start of the first loaded lua script.

   core.Alert = function(msg) print(msg) end

Thierry


On Tue, 29 Sep 2015 07:37:12 -0400
Michael Ezzell  wrote:

> Although I am seeing this in Lua, it does not appear (to me) to a the fault
> in the Lua code... recent changes in __send_log() in log.c appear to assume
> p is never null (there's a test for this condition earlier, then p is
> referenced later), and that is not valid behavior in this case, because
> there is no proxy associated:
> 
> (gdb) run
> Starting program: /home/ubuntu/haproxy-1.6-dev6/haproxy -f ./haproxy.cfg
> 
> Program received signal SIGSEGV, Segmentation fault.
> __send_log (p=p@entry=0x0, level=level@entry=1, message=,
> size=10,
> sd=sd@entry=0x711160  "- ",
> sd_size=sd_size@entry=2) at src/log.c:1025
> 1025if (unlikely(htp->len >= maxlen)) {
> (gdb) print htp->len
> Cannot access memory at address 0xda4
> (gdb) print htp
> $1 = (struct chunk *) 0xd98
> (gdb) print >log_htp
> $2 = (struct chunk *) 0xd98
> (gdb) print p
> $3 = (struct proxy *) 0x0
> (gdb)
> 
> This minimal test case can be replicated with the following two files:
> 
> $ cat haproxy.cfg
> 
> global
> log 127.0.0.1 local0
> lua-load crash.lua
> user haproxy
> group nogroup
> daemon
> 
> defaults
> log global
> 
> $ cat crash.lua
> 
> core.Alert("hello.lua");

Re: segfault writing to log from Lua

[Michael Ezzell]

This a clean build, on both systems, using a freshly-extracted tarball of
1.6-dev6 downloaded from
http://www.haproxy.org/download/1.6/src/devel/haproxy-1.6-dev6.tar.gz.

I'll recheck and send files to replicate.
On Sep 29, 2015 4:47 AM, "Thierry FOURNIER" 
wrote:

> On Mon, 28 Sep 2015 20:50:44 -0400
> Michael Ezzell  wrote:
>
> > I fired up HA-Proxy version 1.6-dev6-e7ae656 2015/09/28 for testing, and
> > was greeted with...
> >
> > Segmentation fault (core dumped)
> >
> > Since I've been primarily testing Lua, I started by commenting out my Lua
> > config lines.  Startup succeeds.  Re-enabling the scripts, I find this to
> > be the offending line:
> >
> > core.Alert("hello.lua");
> >
> > I can't seem to write to syslog in any Lua context without a segfault.
> The
> > failure occurs even if this is the only line, in the only Lua script
> loaded
> > in global config.
> >
> > Behavior is observed on both 64-bit and 32-bit builds on Ubuntu 14.04.
> >
> > HA-Proxy version 1.6-dev6-e7ae656
>
>
> Hello,
>
> Thank you for the bug repport. I can't reproduce it.
>
> Are you sure, that you did a "make clean" before compiling your
> HAProxy ? Some structs are changed, and doesn't running a "make clean"
> is a common way for segfaults :)
>
> If it is the case, can you send me your test files ?
>
> Thierry
>

Execution order of some directives

[Ruoshan Huang]

hi,
After reading the config manual, I have some questions about the 
directives’ executing order:

   1. `reqadd` VS. `http-request add-header`. I can see the warning in the log 
about the reqadd must be added before http-request, but it’s not documented(or 
I miss it).
   2. `http-request add-header` VS. `capture request header`. I want to add 
some headers in the log, but failed to capture the manipulated header.


Coming from NGINX, I’m used to the PHASES model in nginx. say the `set` is 
promised to execute before `echo`, things like that. But in HAProxy, I 
sometimes got confused in which directives will execute first and which 
follows. and I can’t spot the mistake until I’m given a warning in the log. How 
can I find more info to figure these out.

Thank you.

--
Good day!
ruoshan

Re: segfault writing to log from Lua

[Thierry FOURNIER]

On Mon, 28 Sep 2015 20:50:44 -0400
Michael Ezzell  wrote:

> I fired up HA-Proxy version 1.6-dev6-e7ae656 2015/09/28 for testing, and
> was greeted with...
> 
> Segmentation fault (core dumped)
> 
> Since I've been primarily testing Lua, I started by commenting out my Lua
> config lines.  Startup succeeds.  Re-enabling the scripts, I find this to
> be the offending line:
> 
> core.Alert("hello.lua");
> 
> I can't seem to write to syslog in any Lua context without a segfault.  The
> failure occurs even if this is the only line, in the only Lua script loaded
> in global config.
> 
> Behavior is observed on both 64-bit and 32-bit builds on Ubuntu 14.04.
> 
> HA-Proxy version 1.6-dev6-e7ae656


Hello,

Thank you for the bug repport. I can't reproduce it.

Are you sure, that you did a "make clean" before compiling your
HAProxy ? Some structs are changed, and doesn't running a "make clean"
is a common way for segfaults :)

If it is the case, can you send me your test files ?

Thierry

Re: [ANNOUNCE] haproxy-1.6-dev6

[Willy Tarreau]

Hi Joris,

On Tue, Sep 29, 2015 at 08:56:54AM +0200, joris dedieu wrote:
> >   - TCP actions: "silent-drop". Finally it got merged as the actions
> > registration mechanism made it a no-brainer. It works like a deny except
> > that it tries to prevent the TCP RST from reaching the client, so that's
> > quite efficient against certain bots and scripts as their connections
> > remain established on their side only. It works on Linux and could
> > possibly work on other systems (not tested).
> 
> I can confirm that silent-drop is not working as expected on FreeBSD
> 
> listen drop
> bind 80.247.233.40:2
> tcp-request connection silent-drop
> 
> 08:31:31.324885 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
> [S], seq 1048805770, win 29200, options [mss 1460,sackOK,TS val
> 14874937 ecr 0,nop,wscale 7], length 0
> 08:31:31.324903 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
> [S.], seq 510555620, ack 1048805771, win 65535, options [mss
> 1460,nop,wscale 6,sackOK,TS val 1100790208 ecr 14874937], length 0
> 08:31:31.367359 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
> [.], ack 1, win 229, options [nop,nop,TS val 14874946 ecr 1100790208],
> length 0
> 08:31:31.367425 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
> [F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 1100790250 ecr
> 14874946], length 0
(...)
> [F.], seq 1, ack 1, win 1040, options [nop,nop,TS val 1100817450 ecr
> 14874946], length 0
> 08:32:22.886834 IP 82.236.20.129.60620 > 80.247.233.40.2: Flags
> [P.], seq 1:7, ack 1, win 229, options [nop,nop,TS val 14887826 ecr
> 1100790208], length 6
> 08:32:22.886850 IP 80.247.233.40.2 > 82.236.20.129.60620: Flags
> [R], seq 510555621, win 0, length 0

Thanks for your feedback. The fact that the FIN is retranmsmitted like
this tends to confirm that the TTL setting works. However I feel quite
concerned about the fact that a FIN was emitted instead of a reset. I
fear that SO_LINGER doesn't work as expected which is much more of a
problem if that happens on the server side too when closing a server
connection!

Could you please run the same test under strace/truss/whatever you find
equivalent on your platform ?

I'd be interested in seeing each syscall status.

Thanks,
Willy

Re: segfault writing to log from Lua

[Thierry FOURNIER]

Michael, thank you for the bug repport. I reproduce it.

It is introduced with the support of the RFC5424 for the logs format
sublitted by dragan.

Hi Dragan, after your patch about the RFC5424, the funcion __send_log()
no longer support NULL for the "*p" param. Can you fix this ?

It seems that the 'log_htp' and the 'log_htp_rfc5424' members of the
"struct proxy" contains preformatted strings. There makeby the function 
"lf_host_tag_pid()".

Maybe, we should build a defaults strings (one for each rfc) at the
start of haproxy, and uses these default strings if the proxy "*p" is
NULL in the function "__send_log()" ?

Thierry




On Tue, 29 Sep 2015 07:37:12 -0400
Michael Ezzell  wrote:

> Although I am seeing this in Lua, it does not appear (to me) to a the fault
> in the Lua code... recent changes in __send_log() in log.c appear to assume
> p is never null (there's a test for this condition earlier, then p is
> referenced later), and that is not valid behavior in this case, because
> there is no proxy associated:
> 
> (gdb) run
> Starting program: /home/ubuntu/haproxy-1.6-dev6/haproxy -f ./haproxy.cfg
> 
> Program received signal SIGSEGV, Segmentation fault.
> __send_log (p=p@entry=0x0, level=level@entry=1, message=,
> size=10,
> sd=sd@entry=0x711160  "- ",
> sd_size=sd_size@entry=2) at src/log.c:1025
> 1025if (unlikely(htp->len >= maxlen)) {
> (gdb) print htp->len
> Cannot access memory at address 0xda4
> (gdb) print htp
> $1 = (struct chunk *) 0xd98
> (gdb) print >log_htp
> $2 = (struct chunk *) 0xd98
> (gdb) print p
> $3 = (struct proxy *) 0x0
> (gdb)
> 
> This minimal test case can be replicated with the following two files:
> 
> $ cat haproxy.cfg
> 
> global
> log 127.0.0.1 local0
> lua-load crash.lua
> user haproxy
> group nogroup
> daemon
> 
> defaults
> log global
> 
> $ cat crash.lua
> 
> core.Alert("hello.lua");

Re: segfault writing to log from Lua

[Michael Ezzell]

Although I am seeing this in Lua, it does not appear (to me) to a the fault
in the Lua code... recent changes in __send_log() in log.c appear to assume
p is never null (there's a test for this condition earlier, then p is
referenced later), and that is not valid behavior in this case, because
there is no proxy associated:

(gdb) run
Starting program: /home/ubuntu/haproxy-1.6-dev6/haproxy -f ./haproxy.cfg

Program received signal SIGSEGV, Segmentation fault.
__send_log (p=p@entry=0x0, level=level@entry=1, message=,
size=10,
sd=sd@entry=0x711160  "- ",
sd_size=sd_size@entry=2) at src/log.c:1025
1025if (unlikely(htp->len >= maxlen)) {
(gdb) print htp->len
Cannot access memory at address 0xda4
(gdb) print htp
$1 = (struct chunk *) 0xd98
(gdb) print >log_htp
$2 = (struct chunk *) 0xd98
(gdb) print p
$3 = (struct proxy *) 0x0
(gdb)

This minimal test case can be replicated with the following two files:

$ cat haproxy.cfg

global
log 127.0.0.1 local0
lua-load crash.lua
user haproxy
group nogroup
daemon

defaults
log global

$ cat crash.lua

core.Alert("hello.lua");

Re: [ANNOUNCE] haproxy-1.6-dev6

[Willy Tarreau]

On Tue, Sep 29, 2015 at 02:58:04PM +0200, joris dedieu wrote:
> kevent(3,0x0,0,{},5,{1.0 }) = 0 (0x0)
> kevent(3,0x0,0,{0x4,EVFILT_READ,0x0,0,0x1,0x0},5,{1.0 }) = 1 (0x1)
> accept(4,{ AF_INET 80.247.233.242:48068 },0x7fffe804) = 5 (0x5)
> fcntl(5,F_SETFL,O_NONBLOCK) = 0 (0x0)
> recvfrom(5,0x801407000,16384,0x20080,0x0,0x0) ERR#35 'Resource
> temporarily unavailable'
> setsockopt(0x5,0x0,0x4,0x48668c,0x4,0x0) = 0 (0x0)
> setsockopt(0x5,0x6,0x1,0x48668c,0x4,0x0) = 0 (0x0)
> accept(4,0x7fffe808,0x7fffe804) ERR#35 'Resource
> temporarily unavailable'
> shutdown(5,SHUT_WR) = 0 (0x0)
> close(5) = 0 (0x0)
> 
> Has you can see it doesn't looks great.

Indeed, the SO_LINGER is missing and I'm seeing the same here while the flag
is properly set on the stream interface. I have no idea why this does this,
but for sure this is a bug that I have to fix :-)

Thanks!
Willy

Address selection policy in dual-stack environments

[Dmitry Sivachenko]

Hello,

in case when machine has both A and  records, there is an address selection 
policy algorithm which determines which address to use first.
https://www.freebsd.org/cgi/man.cgi?query=ip6addrctl=8

I use it in "prefer ipv4" form, to use ipv4 first when available.

All programs like ssh work as expected.

In haproxy backends are resolved always to ipv6, even when there is an ipv4 
address.

Is it possible to make it to respect address selection policy?

Thanks.

Re: [ANNOUNCE] haproxy-1.6-dev6

[Willy Tarreau]

On Tue, Sep 29, 2015 at 02:58:04PM +0200, joris dedieu wrote:
> kevent(3,0x0,0,{},5,{1.0 }) = 0 (0x0)
> kevent(3,0x0,0,{0x4,EVFILT_READ,0x0,0,0x1,0x0},5,{1.0 }) = 1 (0x1)
> accept(4,{ AF_INET 80.247.233.242:48068 },0x7fffe804) = 5 (0x5)
> fcntl(5,F_SETFL,O_NONBLOCK) = 0 (0x0)
> recvfrom(5,0x801407000,16384,0x20080,0x0,0x0) ERR#35 'Resource
> temporarily unavailable'
> setsockopt(0x5,0x0,0x4,0x48668c,0x4,0x0) = 0 (0x0)
> setsockopt(0x5,0x6,0x1,0x48668c,0x4,0x0) = 0 (0x0)
> accept(4,0x7fffe808,0x7fffe804) ERR#35 'Resource
> temporarily unavailable'
> shutdown(5,SHUT_WR) = 0 (0x0)
> close(5) = 0 (0x0)
> 
> Has you can see it doesn't looks great.

OK I found the reason, in my case the RST I was seeing was caused by pending
data otherwise haproxy didn't send it by itself since we're facing the client.
I've fixed it so that lingering is *really* disabled this time. You can retry
with the attached patch if you want. The second one will get rid of the
useless recvfrom() call if your system doesn't have TCP_QUICKACK. The third
patch addresses a build issue reported off-list on another FreeBSD machine
(SOL_IP not defined).

Thanks,
Willy

>From f50ec0fdbc7f5c8eff3fa91c09ce19c5df3cf8d6 Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Tue, 29 Sep 2015 18:11:32 +0200
Subject: BUG/MINOR: tcp: make silent-drop always force a TCP reset

The silent-drop action is supposed to close with a TCP reset that is
either not sent or not too far. But since it's on the client-facing
side, the socket's lingering is enabled by default and the RST only
occurs if some pending unread data remain in the queue when closing.
This causes some clean shutdowns to occur with retransmits, which is
not good at all. Force linger_risk on the socket to flush all data
and destroy the socket.

No backport is needed, this was introduced in 1.6-dev6.
---
 src/proto_tcp.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/src/proto_tcp.c b/src/proto_tcp.c
index f698889..4c5005e 100644
--- a/src/proto_tcp.c
+++ b/src/proto_tcp.c
@@ -1445,6 +1445,11 @@ static enum act_return 
tcp_exec_action_silent_drop(struct act_rule *rule, struct
if (strm)
strm->si[0].flags |= SI_FL_NOLINGER;
 
+   /* We're on the client-facing side, we must force to disable lingering 
to
+* ensure we will use an RST exclusively and kill any pending data.
+*/
+   fdtab[conn->t.sock.fd].linger_risk = 1;
+
 #ifdef TCP_REPAIR
if (setsockopt(conn->t.sock.fd, SOL_TCP, TCP_REPAIR, , sizeof(one)) 
== 0) {
/* socket will be quiet now */
-- 
1.7.12.1

>From fc2a2d97d6855e9a00a22bc26623e4f2f2c1aeda Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Tue, 29 Sep 2015 18:15:01 +0200
Subject: CLEANUP: tcp: silent-drop: only drain the connection when quick-ack
 is disabled

The conn_sock_drain() call is only there to force the system to ACK
pending data in case of TCP_QUICKACK so that the client doesn't retransmit,
otherwise it leads to a real RST making the feature useless. There's no
point in draining the connection when quick ack cannot be disabled, so
let's move the call inside the ifdef part.
---
 src/proto_tcp.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/proto_tcp.c b/src/proto_tcp.c
index 4c5005e..0655b0d 100644
--- a/src/proto_tcp.c
+++ b/src/proto_tcp.c
@@ -1431,8 +1431,10 @@ static enum act_return 
tcp_exec_action_silent_drop(struct act_rule *rule, struct
if (!conn_ctrl_ready(conn))
goto out;
 
-   conn_sock_drain(conn);
 #ifdef TCP_QUICKACK
+   /* drain is needed only to send the quick ACK */
+   conn_sock_drain(conn);
+
/* re-enable quickack if it was disabled to ack all data and avoid
 * retransmits from the client that might trigger a real reset.
 */
-- 
1.7.12.1

>From ae459f3b9f9810d18aaa090264398dee2d4de23b Mon Sep 17 00:00:00 2001
From: Willy Tarreau 
Date: Tue, 29 Sep 2015 18:19:32 +0200
Subject: BUILD: tcp: use IPPROTO_IP when SOL_IP is not available

Dmitry Sivachenko reported a build failure on FreeBSD due to SOL_IP not
being defined. IPPROTO_IP must be used there instead.
---
 include/common/compat.h | 5 +
 1 file changed, 5 insertions(+)

diff --git a/include/common/compat.h b/include/common/compat.h
index 07dd01d..b2e16af 100644
--- a/include/common/compat.h
+++ b/include/common/compat.h
@@ -129,6 +129,11 @@
 #endif
 #endif
 
+/* FreeBSD doesn't define SOL_IP and prefers IPPROTO_IP */
+#ifndef SOL_IP
+#define SOL_IP IPPROTO_IP
+#endif
+
 /* If IPv6 is supported, define IN6_IS_ADDR_V4MAPPED() if missing. */
 #if defined(IPV6_TCLASS) && !defined(IN6_IS_ADDR_V4MAPPED)
 #define IN6_IS_ADDR_V4MAPPED(a) \
-- 
1.7.12.1

Re: Health check and flapping

[drakaz]

Le 2015-08-28 16:40, Baptiste a écrit :

Le 28 août 2015 15:45,  a écrit :
 >
 > Hello,
 >
 > We have tcp-check configured on some backends, which works fine,
except when service is flapping.
 >
 > If the backend server is in transitional state, for example
transitionally DOWN (going up), the counter is not reset to 0 if
tcp-check give a KO state between some OK state. The result is that if
the service is flapping, backend become up for a few seconds quite
often, even if all OK state are not consecutives.
 >
 > Example of sequence with rise 3:
 >
 > KO -> 0/3
 > KO -> 0/3
 > OK -> 1/3
 > KO -> 1/3 <- should back to 0/3
 > KO -> 1/3
 > KO -> 1/3
 > OK -> 2/3
 > KO -> 2/3
 > KO -> 2/3
 > OK -> 3/3 -> Server UP
 >
 > Is there a way to configure the counter to reset itself in case of
flapping ?
 >
 > Thanks.

Hi there,

Thanks for reporting this behavior.

I'll have a look and come back to you.

Baptiste


Hello,

Are you able to reproduce on your side ?

Thanks

Re: Lua usage question - log format variables

[thierry . fournier]

On Tue, 29 Sep 2015 09:57:46 -0400
Michael Ezzell  wrote:

> We have "fetches" accessible from Lua, but I have not found a mechanism
> that allows me to access the value of log format variables.
> 
> Examples of useful variables:
> 
> %b   | backend_name
> %f   | frontend_name
> %rt  | request_counter
> 
> Is there a way to access values like these from Lua, since they aren't (as
> far as I can tell) available as fetches?
> 
> I realize not all variables would be available in every context, and I
> don't want to modify them, of course -- I only want to read them -- but I
> would like to be able to read them and use them in content generated in Lua.
> 
> Are they accessible?

Hi,

Sadly these information are not avalaible. In addition the 1.6.0
release is very close, so Willy freezes the features.

You can get the id of the current fontend/backend with the following
fetches:

   TXN.f.be_id
   TXN.f.fe_id

But it not the name. You have some ugly methods: The most ugly is
adding a request header containing the frontend / backend name, the Lua
permit to fetch headers.

So the HAProxy is like this:

   http-request add-header x-haproxy-fe-name %f
   http-request add-header x-haproxy-be-name %b

In the lua:

   TXN.f.req_fhdr("x-haproxy-fe-name")
   TXN.f.req_fhdr("x-haproxy-be-name")

Another way is use the vars, but we considers that the fe/be name must
be enter with manual input, like this:

   tcp-request content set-var(txn.be) name-of-the-current-backend
   tcp-request content set-var(txn.fe) name-of-the-current-frontend

In the Lua:

   TXN.f.var("txn.be")
   TXN.f.var("txn.be")

For the request counter is avalaible with this fetch:

   TXN.f.http_req_cnt

Thierry

Re: Execution order of some directives

[Ruoshan Huang]

Oops, It’s "`reqadd` must come before the `http-request add-header`". 

> On Sep 29, 2015, at 5:38 PM, Ruoshan Huang  wrote:
> 
>   1. `reqadd` VS. `http-request add-header`. I can see the warning in the log 
> about the reqadd must be added before http-request, but it’s not 
> documented(or I miss it).

--
Good day!
ruoshan

Re: segfault writing to log from Lua

[Willy Tarreau]

On Tue, Sep 29, 2015 at 02:01:59PM +0200, Thierry FOURNIER wrote:
> Michael, thank you for the bug repport. I reproduce it.
> 
> It is introduced with the support of the RFC5424 for the logs format
> sublitted by dragan.
> 
> Hi Dragan, after your patch about the RFC5424, the funcion __send_log()
> no longer support NULL for the "*p" param. Can you fix this ?
> 
> It seems that the 'log_htp' and the 'log_htp_rfc5424' members of the
> "struct proxy" contains preformatted strings. There makeby the function 
> "lf_host_tag_pid()".
> 
> Maybe, we should build a defaults strings (one for each rfc) at the
> start of haproxy, and uses these default strings if the proxy "*p" is
> NULL in the function "__send_log()" ?

Yes indeed we should have one of each, statically defined in log.c with
the global log tag I guess. In fact this would even allow not to allocate
one tag per proxy since we'd be able to reuse the global one when a proxy
defines no log-tag.

Thanks Thierry for the diag.

Willy

Re: Address selection policy in dual-stack environments

[Willy Tarreau]

Hi Dmitry,

On Tue, Sep 29, 2015 at 08:08:51PM +0300, Dmitry Sivachenko wrote:
> Hello,
> 
> in case when machine has both A and  records, there is an address 
> selection policy algorithm which determines which address to use first.
> https://www.freebsd.org/cgi/man.cgi?query=ip6addrctl=8
> 
> I use it in "prefer ipv4" form, to use ipv4 first when available.
> 
> All programs like ssh work as expected.
> 
> In haproxy backends are resolved always to ipv6, even when there is an ipv4 
> address.
> 
> Is it possible to make it to respect address selection policy?

I *think* that getaddrinfo() provides this. You can try to build by
adding USE_GETADDRINFO=1 to your makefile. It's not enabled by default
because there are numerous bogus implementations on various systems.
If it works for you it could be the best solution as other programs
which work are likely using it. I don't know if it's safe to enable
it by default on FreeBSD.

Regards,
Willy

haproxy 1.5.4 with ssl-bridging

[Douglas Harmon]

Hello group. I'm new to haproxy. I have read the documentation but
still require some assistance. I'm trying to configure haproxy to:

1. accept https connection with client certs required.
2. pass the client cert to a backend https server based on https url path

First, can I accomplish this with haproxy? If so, could someone share
a sample haproxy 1.5 configuration? I have the item 1 above working in
tcp mode. But I believe I need to be in http mode to get item 2 to
work.

-Doug

Implementing HAProxy First Time: Conditional backend issue

[Susheel Jalali]

Dear HAProxy Developers,

We are implementing HAProxy for reverse proxy load balancing for the 
first time in our LAN.  We are seeking your guidance below to optimize 
our HAProxy configuration. Currently, we are unable to access our fully 
working products via HAProxy on two different servers.  We would 
appreciate any pointers to learn from so that we could solve this 
HAPRoxy conditional backend issue.


Issue:  "Cannot get /Product1"

We have working products (Product-1, 2, 3, …)  instances residing on 2 
servers hosted in internal LAN. HAProxy is in DMZ.  We are using the 
following haproxy.cfg template to configure access to our products 
behind HAProxy:


·Live Product-1 can be accessed directly: 
http://Server_IP:Port#_of_Product1 
  orhttp://Coscend.com:Port#_of_Product1 



·HAProxy stats on Web browser show all working products are UP.

·But we cannot access them via HAProxy by their names, Product1, 
Product2, ...


Access URL of Product 1 via HAProxy reverse proxy: 
http://HAProxy_Server_IP:Port#_of_HAProxy/Product1 
 orhttp://Coscend.com/Product1 



Test scenario created

·Global and Default sections producing proper logs.

·Product-1 is accessible via default backend of HAProxy.

·Bypassing Local DNS issues by using server IP Address and product Port 
#s in haproxy.cfg.


·Frontend is working and directing us to Product-1, as seen in the logs.

·Product-1 is not accessible via conditional back end.

-- --

frontend webapps-frontend

bind  *:80

acl host_coscendreq.hdr(Host) coscend.com:80

acl path_subdomain_p1 path_beg -i /Product1

use_backend subdomain_p1-backend if host_coscend path_subdomain_p1

default_backend webapps-backend

backend webapps-backend

[…]

option   http-server-close

backend subdomain_p1-backend

http-request set-header Host 12.12.12.112:6012

acl hdr_location res.hdr(Location) -m found

rspirep ^Location:\ (http?://12.12.12.112(:[0-9]+)?)?(/.*) Location:\ 
/Product1  if hdr_location


server Product1.VM0 12.12.12.112:6012 cookie pad-p check

Thank you.

Sincerely,

--

Regards,

Susheel Jalali


Coscend Communications Solutions

__

Web site:www.Coscend.com 

--

*Coscend’s**Software Service Factory*

"*Coscend Communications* is ... *pioneering a new approach*to ... 
software applications development, and systems integration."


*Light Reading Network, *December, 2007

"*Coscend*is at the*vanguard of a new evolution*in telco OSS/BSS systems 
integration."


*Caroline Chappell*
A leading authority in the communications services software industry

"There are *innovative*…*tools*from ... *Coscend *bubbling up, which 
will help accelerate the data consolidation process and reduce its cost."


*Dennis Mendyk, */Editor,/Building a *Telco Service Factory*

--


CONFIDENTIALITY NOTICE: See 'Confidentiality Notice Regarding E-mail 
Messages from Coscend Communications Solutions' posted at: 
http://www.Coscend.com/Terms_and_Conditions.html 

Re: Address selection policy in dual-stack environments

[Dmitry Sivachenko]

> On 29 сент. 2015 г., at 21:26, Willy Tarreau  wrote:
> 
> Hi Dmitry,
> 
> On Tue, Sep 29, 2015 at 08:08:51PM +0300, Dmitry Sivachenko wrote:
>> Hello,
>> 
>> in case when machine has both A and  records, there is an address 
>> selection policy algorithm which determines which address to use first.
>> https://www.freebsd.org/cgi/man.cgi?query=ip6addrctl=8
>> 
>> I use it in "prefer ipv4" form, to use ipv4 first when available.
>> 
>> All programs like ssh work as expected.
>> 
>> In haproxy backends are resolved always to ipv6, even when there is an ipv4 
>> address.
>> 
>> Is it possible to make it to respect address selection policy?
> 
> I *think* that getaddrinfo() provides this. You can try to build by
> adding USE_GETADDRINFO=1 to your makefile. It's not enabled by default
> because there are numerous bogus implementations on various systems.
> If it works for you it could be the best solution as other programs
> which work are likely using it. I don't know if it's safe to enable
> it by default on FreeBSD.
> 


I do have this enabled:

Build options :
  TARGET  = freebsd
  CPU = generic
  CC  = cc
  CFLAGS  = -O2 -pipe -O2 -fno-strict-aliasing -pipe -fstack-protector 
-DFREEBSD_PORTS
  OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_STATIC_PCRE=1 
USE_PCRE_JIT=1