Queue flushing if backend goes down

[Walecki, Jakub (Nokia - PL/Wroclaw)]

Hi,

Is there a way to preven backend form flushing queue if all machines go down?

Best regards,
Jakub Walecki

Re: Segfault on haproxy 1.7.10 with state file and slowstart

[Willy Tarreau]

Hi Raghu,

On Sun, Jan 14, 2018 at 05:59:16PM +0530, Raghu Udiyar wrote:
> Hi Willy
> 
> Yes, this patch works, no more segfaults.
> 
> However, unrelated to this issue, while testing I noticed that the address
> from the state file is not applied if the configuration does not use a
> hostname.
> 
> In the example I provided, the configuration originally has the IP address
> set to 169.254.0.9, which is then updated through the socket to say 
> 10.0.19.17.

I noticed it as well but I thought I was the one doing something wrong.
CCing Baptiste in case he remembers certain cases that have to be
addressed a specific way by the state file. I remember this stuff was
tested quite a bit so I suspect that it could depend on certain
conditions.

> Expectation was that upon haproxy restart the address in the state fill
> will override the address in the configuration, but this does not happen.
> 
> I see the relevant code in srv_init_addr :
> 
>   if (srv->hostname)
> return_code |= srv_iterate_initaddr(srv);
> 
> As per the server-template example in the dynamic scaling blog post, this
> should have been supported. Am I missing some configuration?

I can't really tell, let's wait for Baptiste's opinion on this.

Thanks,
Willy

Re: Warning: upgrading to openssl master+ enable_tls1_3 (coming v1.1.1) could break handshakes for all protocol versions .

[Lukas Tribus]

Hello,


On 13 January 2018 at 20:57, Pavlos Parissis  wrote:
> On 13/01/2018 04:22 μμ, Lukas Tribus wrote:
>> Hello,
>>
>>
>> On 13 January 2018 at 15:17, Pavlos Parissis  
>> wrote:
 Not exactly, the moment you force a cipher list that does not include a
 TLSv1.3 cipher in the server side (which has TLSv1.3 enabled) the TLS
 handshake will break regardless of what is in the Client hello.

>>>
>>> But, can we have TLSv3 enabled on server side and still accept TLSv2 
>>> sessions?
>>
>> Only if your cipher-list contains TLSv1.3 ciphers, otherwise nothing
>> will work (regardless of the TLS version).
>>
>
> and all those ciphers are supported by all recent browsers, right ?

That's not the point, you can always specify old ciphers as well. It's
just that you MUST specify at least 1 TLSv1.3 cipher (for any TLS
version to work).



>> OpenSSL really goes the extra mile to make everyone's life miserable.
>
> Is this the result of the implementation or of the TLSv1.3 design ?

TLSv1.3 is fine, the discussion in the IETF working-group has lots of
participants and the process works.

The OpenSSL implementation (and especially the API) is decided by a
small number of people, they have (rightfully so) their own opinions,
but I also don't see them receptive of different opinions.


That's why Google forked it and why other are switching to that fork:
https://blog.cloudflare.com/make-ssl-boring-again/


But abandoning OpenSSL for a fork like BoringSSL brings their own
problems, it's certainly not a change at the push of a button.


Lukas

[PATCH v3 8/8] MEDIUM: sample: Add IPv6 support to the ipmask converter

[Tim Duesterhus]

Add an optional second parameter to the ipmask converter that specifies
the number of bits to mask off IPv6 addresses.

If the second parameter is not given IPv6 addresses fail to mask (resulting
in an empty string), preserving backwards compatibility: Previously
a sample like `src,ipmask(24)` failed to give a result for IPv6 addresses.

This feature can be tested like this:

  defaults
log global
modehttp
option  httplog
option  dontlognull
timeout connect 5000
timeout client  5
timeout server  5

  frontend fe
bind :::8080 v4v6

# Masked IPv4 for IPv4, empty for IPv6 (with and without this commit)
http-response set-header Test %[src,ipmask(24)]
# Correctly masked IP addresses for both IPv4 and IPv6
http-response set-header Test2 %[src,ipmask(24,:::::)]
# Correctly masked IP addresses for both IPv4 and IPv6
http-response set-header Test3 %[src,ipmask(24,64)]

default_backend be

  backend be
server s example.com:80
---
 doc/configuration.txt | 11 +++
 src/sample.c  | 25 -
 2 files changed, 27 insertions(+), 9 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 827506788..eaaf0f71f 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -12870,11 +12870,14 @@ in_table()
   the presence of a certain key in a table tracking some elements (e.g. whether
   or not a source IP address or an Authorization header was already seen).
 
-ipmask()
-  Apply a mask to an IPv4 address, and use the result for lookups and storage.
+ipmask(, [])
+  Apply a mask to an IP address, and use the result for lookups and storage.
   This can be used to make all hosts within a certain mask to share the same
-  table entries and as such use the same server. The mask can be passed in
-  dotted form (e.g. 255.255.255.0) or in CIDR form (e.g. 24).
+  table entries and as such use the same server. The mask4 can be passed in
+  dotted form (e.g. 255.255.255.0) or in CIDR form (e.g. 24). The mask6 can
+  be passed in quadruplet form (e.g. :::) or in CIDR form (e.g. 64).
+  If no mask6 is given IPv6 addresses will fail to convert for backwards
+  compatibility reasons.
 
 json([])
   Escapes the input string and produces an ASCII output string ready to use as 
a
diff --git a/src/sample.c b/src/sample.c
index 04cec4fc8..0adc88179 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -1603,11 +1603,26 @@ static int sample_conv_str2upper(const struct arg 
*arg_p, struct sample *smp, vo
return 1;
 }
 
-/* takes the netmask in arg_p */
-static int sample_conv_ipmask(const struct arg *arg_p, struct sample *smp, 
void *private)
+/* takes the IPv4 mask in args[0] and an optional IPv6 mask in args[1] */
+static int sample_conv_ipmask(const struct arg *args, struct sample *smp, void 
*private)
 {
-   smp->data.u.ipv4.s_addr &= arg_p->data.ipv4.s_addr;
-   smp->data.type = SMP_T_IPV4;
+   /* Attempt to convert to IPv4 to apply the correct mask. */
+   c_ipv62ip(smp);
+
+   if (smp->data.type == SMP_T_IPV4) {
+   smp->data.u.ipv4.s_addr &= args[0].data.ipv4.s_addr;
+   smp->data.type = SMP_T_IPV4;
+   }
+   else if (smp->data.type == SMP_T_IPV6) {
+   /* IPv6 cannot be converted without an IPv6 mask. */
+   if (args[1].type != ARGT_IPV6)
+   return 0;
+
+   for (size_t i = 0; i < 16; i++)
+   smp->data.u.ipv6.s6_addr[i] &= 
args[1].data.ipv6.s6_addr[i];
+   smp->data.type = SMP_T_IPV6;
+   }
+
return 1;
 }
 
@@ -2797,7 +2812,7 @@ static struct sample_conv_kw_list sample_conv_kws = {ILH, 
{
{ "length", sample_conv_length,0,NULL, SMP_T_STR,  
SMP_T_SINT },
{ "hex",sample_conv_bin2hex,   0,NULL, SMP_T_BIN,  
SMP_T_STR  },
{ "hex2i",  sample_conv_hex2int,   0,NULL, SMP_T_STR,  
SMP_T_SINT },
-   { "ipmask", sample_conv_ipmask,ARG1(1,MSK4), NULL, SMP_T_IPV4, 
SMP_T_IPV4 },
+   { "ipmask", sample_conv_ipmask,ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, 
SMP_T_IPV4 },
{ "ltime",  sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, 
SMP_T_STR },
{ "utime",  sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, 
SMP_T_STR },
{ "crc32",  sample_conv_crc32, ARG1(0,SINT), NULL, SMP_T_BIN,  
SMP_T_SINT  },
-- 
2.15.1

Re: [PATCH 0/8] Add IPv6 support to the ipmask converter

[Tim Düsterhus]

Jarno,

Am 14.01.2018 um 10:01 schrieb Jarno Huuskonen:

> Quick question: how does your patch handle ipv4mapped ipv6
> address(:::1.2.3.4) ? Does it use v4 or v6 mask for these addresses ?
> (From code I think v4 (first tries to convert v6 to v4 with c_ipv62ip) ?)

Yes, you are correct. I took take care that these are handled as IPv4.

> Do you see any use for ip6mask sample that would always use ipv6 mask
> and only work with ipv6 addresses (ipv4mapped and pure ipv6) ?

That basically would boil down to completely masking away IPv4
addresses. You can do that using: `ipmask(0,64)` with my patch as well
(resulting in 0.0.0.0 for IPv4 addresses). Personally I don't see a real
use case for handling IPv6 only.

> Patch8 could add comment about arg_p parameters to sample_conv_ipmask
> maybe something like:
> +/* takes the ipv4 netmask in arg_p[0] and optional ipv6 netmask in
> arg_p[1] */

Yes, thank you. I also noticed that the parameter is called `args` if
multiple parameters are accepted. I fixed the comment, renamed the
parameter and will send the updated patch in a separate mail to the list.

> Have you tested that req.hdr_ip / stick tables work w/both masks ? I
> used something like:
> http-request track-sc0 req.hdr_ip(X,1),ipmask(24,64) table test_be
> http-request set-var(sess.myx) req.hdr_ip(X,1),ipmask(24,64)
> http-response set-header X-MY %[var(sess.myx)]
> 
> backend test_be
>   stick-table type ipv6 size 20 expire 180s store gpc0,conn_cnt

I must admit that I never used stick tables in haproxy before. The use
case that lead me to implement this patch is privacy reasons (mask off
the lower bits for the logs). I verified my patch using the haproxy.cfg
given in the commit message and don't see a reason why it should not
work in other places.
I'd be grateful if you could verify the correct workings of stick
tables, as you already know how they are supposed to work.

Thanks
Tim Düsterhus

Re: Segfault on haproxy 1.7.10 with state file and slowstart

[Raghu Udiyar]

Hi Willy

Yes, this patch works, no more segfaults.

However, unrelated to this issue, while testing I noticed that the address
from the state file is not applied if the configuration does not use a
hostname.

In the example I provided, the configuration originally has the IP address
set to 169.254.0.9, which is then updated through the socket to say 10.0.19.17.
Expectation was that upon haproxy restart the address in the state fill
will override the address in the configuration, but this does not happen.

I see the relevant code in srv_init_addr :

  if (srv->hostname)
return_code |= srv_iterate_initaddr(srv);

As per the server-template example in the dynamic scaling blog post, this
should have been supported. Am I missing some configuration?

Thanks!
-- Raghu


-- Raghu

On Fri, Jan 12, 2018 at 4:14 PM, Willy Tarreau  wrote:

> Hello Raghu,
>
> On Thu, Jan 11, 2018 at 02:20:34PM +0530, Raghu Udiyar wrote:
> > Hello,
> >
> > Haproxy 1.7.10 segfaults when the srv_admin_state is set to
> > SRV_ADMF_CMAINT (0x04)
> > for a backend server, and that backend has the `slowstart` option set.
> >
> > The following configuration reproduces it :
> (...)
>
> Thanks for all the details, they made it easy to reproduce.
>
> From what I'm seeing, it's a fundamental design issue in the state
> file handling in 1.7. It starts checks before they have been
> initialized, and try to wake up a NULL task. In 1.8 due to the more
> dynamic config, the initialization sequence has changed and checks
> are initialized before parsing the state file, but I don't feel at
> ease with doing in 1.7 since I don't know if some config elements
> may remain non-updated.
>
> So instead I've just protected against using the task wakeups during
> the state file parsing, and they will be initialized later with the
> appropriate parameters.
>
> Could you please check the attached patch on top of 1.7.10 ?
>
> Thanks,
> Willy
>

Re: [PATCH 0/8] Add IPv6 support to the ipmask converter

[Jarno Huuskonen]

Hi,

On Sat, Jan 13, Tim Duesterhus wrote:
> this is a patch series ultimately leading to IPv6 support for the
> ipmask converter. Please note the following:

+1

I did something similar (never 100% finished) 
(http://haproxy.formilux.narkive.com/PZHJ5xNH/rfc-ipv6mask-converter).

Quick question: how does your patch handle ipv4mapped ipv6
address(:::1.2.3.4) ? Does it use v4 or v6 mask for these addresses ?
(From code I think v4 (first tries to convert v6 to v4 with c_ipv62ip) ?)

Do you see any use for ip6mask sample that would always use ipv6 mask
and only work with ipv6 addresses (ipv4mapped and pure ipv6) ?

Patch8 could add comment about arg_p parameters to sample_conv_ipmask
maybe something like:
+/* takes the ipv4 netmask in arg_p[0] and optional ipv6 netmask in
arg_p[1] */

Have you tested that req.hdr_ip / stick tables work w/both masks ? I
used something like:
http-request track-sc0 req.hdr_ip(X,1),ipmask(24,64) table test_be
http-request set-var(sess.myx) req.hdr_ip(X,1),ipmask(24,64)
http-response set-header X-MY %[var(sess.myx)]

backend test_be
  stick-table type ipv6 size 20 expire 180s store gpc0,conn_cnt

-Jarno

> - The first patch contains non-US-ASCII characters, as the sole purpose
>   of it is to remove the non-US-ASCII characters from the source file.
>   Please take extra care when applying the patch to make sure the result
>   is good.
> - The next two patches fix two small bugs in the sample converters and
>   are of general utility.
> - The last five patches implement the progression to IPv6 support for the
>   ipmask converter and only make changes to the mask handling.
> 
> Think of this as 3 logical patch series when reviewing.
> 
> Tim Duesterhus (8):
>   BUG/MINOR: sample: Fix encoding of sample.c
>   DOC: sample: Fix outdated comment about sample casts functions
>   BUG/MINOR: sample: Fix output type of c_ipv62ip
>   DOC: Fix typo in ARGT_MSK6 comment
>   CLEANUP: standard: Use len2mask4 in str2mask
>   MINOR: standard: Add str2mask6 function
>   MINOR: config: Add support for ARGT_MSK6
>   MEDIUM: sample: Add IPv6 support to the ipmask converter
> 
>  doc/configuration.txt | 11 +++
>  include/common/standard.h |  6 ++
>  include/types/arg.h   |  2 +-
>  src/arg.c | 11 +--
>  src/hlua.c|  7 +++
>  src/sample.c  | 31 ++-
>  src/standard.c| 28 
>  7 files changed, 76 insertions(+), 20 deletions(-)
> 
> -- 
> 2.15.1
> 
> 

-- 
Jarno Huuskonen