Re: JWT payloads break b64dec convertor

2018-05-27 Thread cripy 
while probably not the most ideal solution... i found a quick method to do
this using the builtin converters within the configuration to append the
padding where necessary.

here is an example:

log-format %[var(txn.jwtpayload),b64dec]

http-request set-var(txn.jwtpayload) req.hdr('x-jwtpayload')
http-request set-var(txn.jwtpayload) str(),concat(,txn.jwtpayload,==) if !{
var(txn.jwtpayload),length,mod(4) -m int eq 0 }  {
var(txn.jwtpayload),length,mod(2) -m int eq 0 }
http-request set-var(txn.jwtpayload) str(),concat(,txn.jwtpayload,=) if !{
var(txn.jwtpayload),length,mod(4) -m int eq 0 }

[1.8.9] 100 CPU when using lxd resolver

2018-05-27 Thread Zinsser Zhang 
After I updated from 1.8.8 to 1.8.9, the child process consumes 100% of one
core from startup. Most of haproxy's functionality works fine (and of
course much slower than before) except that commands (e.g. set state
to MAINT) in the stat web page often result in a 502 bad gateway.

After some debugging, it turns out to be an issue with the resolver. I am
using LXD containers and dns resolution is done by LXD which only listen on
its subnet. If I change all .lxd domains in my configuration to IP
addresses, everything gets back to normal.

Wonder if anyone has a clue or it's a bug.

Additional information:
When I updated haproxy, I also updated my kernel from 4.4.0-124 to
4.4.0-127. The server has 2 cores and 2GB memory. No obvious memory leak
after 3 hours of 100% CPU.

[haproxy -v]
HA-Proxy version 1.8.9-1ppa1~xenial 2018/05/19
Copyright 2000-2018 Willy Tarreau 
(I'm using Vincent Bernat's ppa)

[uname -a]
Linux  4.4.0-127-generic #153-Ubuntu SMP Sat May 19 10:58:46
UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[lsb_release -a]
LSB Version:
core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch:security-9.20160110ubuntu0.2-amd64:security-9.20160110ubuntu0.2-noarch
Distributor ID: Ubuntu
Description: Ubuntu 16.04.4 LTS
Release: 16.04
Codename: xenial

===
[Original config file]
===
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd
listeners
stats timeout 30s
user haproxy
group haproxy
daemon

# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private

# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
#  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
# An alternative list with additional directives can be obtained from
#
https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
ssl-default-bind-ciphers
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3

resolvers lxd
nameserver lxd 10.12.34.1:53

userlist stats-auth
group admin users zinsser
user zinsser password 

defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000
timeout client  5
timeout server  5
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http

default-server init-addr none resolvers lxd

listen imap
mode tcp
bind *:143
server mail-main mail-main.lxd:143 send-proxy

listen smtp
mode tcp
bind *:25
server mail-main mail-main.lxd:25 send-proxy

listen submission
mode tcp
bind *:587
server mail-main mail-main.lxd:587 send-proxy

frontend https-in
mode http
bind *:443 ssl crt /etc/haproxy/certs/full-priv.pem alpn h2,http/1.1

# X-Forwarded-For
option forwardfor
# X-Forwarded-Proto
reqadd X-Forwarded-Proto:\ https

# Domains
acl cloud hdr_beg(host) -i cloud.
acl rspamd hdr_beg(host) -i rspamd.
# Url
acl openproject url_beg /openproject

# Stats
acl stats_auth http_auth(stats-auth)
acl stats_admin http_auth_group(stats-auth) admin
stats http-request auth unless stats_auth

stats enable
stats hide-version
stats uri /ha_stats
stats admin if stats_admin

# Backends
use_backend openproject if cloud openproject
use_backend cloud if cloud
use_backend rspamd if rspamd
default_backend wordpress

frontend http-in
mode http
bind *:80

redirect scheme https code 301

backend wordpress
mode http
server wordpress wordpress.lxd:80

backend cloud
mode http
server cloud cloud.lxd:80

backend openproject
mode http
server openproject openproject.lxd:80

backend rspamd
mode http
server rspamd mail-filter.lxd:11334

===
[Workaround config file]  Notice that the default-server line is not
commented out.
===
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd
listeners
stats timeout 30s
user haproxy
group haproxy
daemon

# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private

# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
#

Re: Use acl on spoe events

2018-05-27 Thread Joao Morais 

> Em 27 de mai de 2018, à(s) 12:02, Daniel Corbett  
> escreveu:
> 
> Hello Joao,
> 
> On 05/26/2018 05:54 PM, Joao Morais wrote:
>> 
>> There is no difference if I use acl like the example above, or use the `if 
>> {...}` syntax or remove the acl at all, my modsecurity agent always receive 
>> a new connection despite of the host I’m using.
>> 
>> Is there a way to use a spoe filter only if some l7 conditions match?
> 
> This appears to have been fixed in 1.9-dev with this commit:
> 
> https://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=333694d7715952a9610a3e6f00807eaf5edd209a;hp=336d3ef0e77192582c98b3c578927a529ceadd9b

Hi Daniel, I can confirm this fixes my 1.8.9 deployment, thanks!

Is there any plan to backport to 1.8?

> 
> Thanks,
> -- Daniel

~jm

Re: [PATCH] BUG/MEDIUM: stick-tables: Decrement ref_cnt in table_* converters

2018-05-27 Thread Daniel Corbett 

Hello Willy,


On 05/24/2018 04:16 PM, Willy Tarreau wrote:


Interesting one! However it's not correct, it doesn't decrement the
refcount on the return 0 path so the problem remains when a data
type is looked up in a table where it is not stored. For example :


Nice catch.  Thanks for pointing this out.


Given that all functions seem to be written the same way, I suggest
that we change the end and invert the !ptr condition to centralize
the release call. It would give this above :

 ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_CUR);
 if (ptr)
smp->data.u.sint = stktable_data_cast(ptr, conn_cur);
 stktable_release(t, ts);
 return !!ptr;

Could you please rework your patch to do this so that I can merge it ?



I have attached the latest patch with these changes.

Thanks,
-- Daniel


>From e47065ac6cc7478d21cfa00c5c45a0ae7cd412bf Mon Sep 17 00:00:00 2001
From: Daniel Corbett 
Date: Sun, 27 May 2018 09:47:12 -0400
Subject: [PATCH] BUG/MEDIUM: stick-tables: Decrement ref_cnt in table_*
 converters

When using table_* converters ref_cnt was incremented
and never decremented causing entries to not expire.

The root cause appears to be that stktable_lookup_key()
was called within all sample_conv_table_* functions which was
incrementing ref_cnt and not decrementing after completion.

Added stktable_release() to the end of each sample_conv_table_*
function and reworked the end logic to ensure that ref_cnt is
always decremented after use.

This should be backported to 1.8
---
 src/stick_table.c | 170 +++---
 1 file changed, 86 insertions(+), 84 deletions(-)

diff --git a/src/stick_table.c b/src/stick_table.c
index fcc6fe6..101a4e2 100644
--- a/src/stick_table.c
+++ b/src/stick_table.c
@@ -875,6 +875,7 @@ static int sample_conv_in_table(const struct arg *arg_p, struct sample *smp, voi
 	smp->data.type = SMP_T_BOOL;
 	smp->data.u.sint = !!ts;
 	smp->flags = SMP_F_VOL_TEST;
+	stktable_release(t, ts);
 	return 1;
 }
 
@@ -907,12 +908,12 @@ static int sample_conv_table_bytes_in_rate(const struct arg *arg_p, struct sampl
 		return 1;
 
 	ptr = stktable_data_ptr(t, ts, STKTABLE_DT_BYTES_IN_RATE);
-	if (!ptr)
-		return 0; /* parameter not stored */
+	if (ptr)
+		smp->data.u.sint = read_freq_ctr_period(_data_cast(ptr, bytes_in_rate),
+   t->data_arg[STKTABLE_DT_BYTES_IN_RATE].u);
 
-	smp->data.u.sint = read_freq_ctr_period(_data_cast(ptr, bytes_in_rate),
-	   t->data_arg[STKTABLE_DT_BYTES_IN_RATE].u);
-	return 1;
+	stktable_release(t, ts);
+	return !!ptr;
 }
 
 /* Casts sample  to the type of the table specified in arg(0), and looks
@@ -944,11 +945,11 @@ static int sample_conv_table_conn_cnt(const struct arg *arg_p, struct sample *sm
 		return 1;
 
 	ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_CNT);
-	if (!ptr)
-		return 0; /* parameter not stored */
+	if (ptr)
+		smp->data.u.sint = stktable_data_cast(ptr, conn_cnt);
 
-	smp->data.u.sint = stktable_data_cast(ptr, conn_cnt);
-	return 1;
+	stktable_release(t, ts);
+	return !!ptr;
 }
 
 /* Casts sample  to the type of the table specified in arg(0), and looks
@@ -980,11 +981,11 @@ static int sample_conv_table_conn_cur(const struct arg *arg_p, struct sample *sm
 		return 1;
 
 	ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_CUR);
-	if (!ptr)
-		return 0; /* parameter not stored */
+	if (ptr)
+		smp->data.u.sint = stktable_data_cast(ptr, conn_cur);
 
-	smp->data.u.sint = stktable_data_cast(ptr, conn_cur);
-	return 1;
+	stktable_release(t, ts);
+	return !!ptr;
 }
 
 /* Casts sample  to the type of the table specified in arg(0), and looks
@@ -1016,12 +1017,12 @@ static int sample_conv_table_conn_rate(const struct arg *arg_p, struct sample *s
 		return 1;
 
 	ptr = stktable_data_ptr(t, ts, STKTABLE_DT_CONN_RATE);
-	if (!ptr)
-		return 0; /* parameter not stored */
+	if (ptr)
+		smp->data.u.sint = read_freq_ctr_period(_data_cast(ptr, conn_rate),
+   t->data_arg[STKTABLE_DT_CONN_RATE].u);
 
-	smp->data.u.sint = read_freq_ctr_period(_data_cast(ptr, conn_rate),
-	   t->data_arg[STKTABLE_DT_CONN_RATE].u);
-	return 1;
+	stktable_release(t, ts);
+	return !!ptr;
 }
 
 /* Casts sample  to the type of the table specified in arg(0), and looks
@@ -1053,12 +1054,12 @@ static int sample_conv_table_bytes_out_rate(const struct arg *arg_p, struct samp
 		return 1;
 
 	ptr = stktable_data_ptr(t, ts, STKTABLE_DT_BYTES_OUT_RATE);
-	if (!ptr)
-		return 0; /* parameter not stored */
+	if (ptr)
+		smp->data.u.sint = read_freq_ctr_period(_data_cast(ptr, bytes_out_rate),
+   t->data_arg[STKTABLE_DT_BYTES_OUT_RATE].u);
 
-	smp->data.u.sint = read_freq_ctr_period(_data_cast(ptr, bytes_out_rate),
-	   t->data_arg[STKTABLE_DT_BYTES_OUT_RATE].u);
-	return 1;
+	stktable_release(t, ts);
+	return !!ptr;
 }

Re: Use acl on spoe events

2018-05-27 Thread Daniel Corbett 

Hello Joao,


On 05/26/2018 05:54 PM, Joao Morais wrote:


There is no difference if I use acl like the example above, or use the `if 
{...}` syntax or remove the acl at all, my modsecurity agent always receive a 
new connection despite of the host I’m using.

Is there a way to use a spoe filter only if some l7 conditions match?


This appears to have been fixed in 1.9-dev with this commit:

https://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=333694d7715952a9610a3e6f00807eaf5edd209a;hp=336d3ef0e77192582c98b3c578927a529ceadd9b


Thanks,
-- Daniel