Re: Inquiry of Vertical Balancer

2021-10-12 Thread Илья Шипицин
hello,

please tell us more about the relation between Vertical Balancer and the
mailing list you used.

пн, 11 окт. 2021 г. в 18:25, Mr. Reeder :

> Greetings,
> Hope you are fine. Well I am in the market to purchase a Vertical Balancer
> and through my search I came across your address. Kindly let me know the
> models you have or a link to the ones you have in stock. Also want to know
> whether you consider credit card an option for payment? Will await your
> responses.
>
> Best Regards,
>
> Bill Reeder
> Reeder Equipment Co.
> Glendale, CA 91203, USA
>


Re: [ANNOUNCE] haproxy-2.5-dev9

2021-10-12 Thread Илья Шипицин
can remaining coverity findings be reviewed before 2.5 ?

https://github.com/haproxy/haproxy/issues/1163
https://github.com/haproxy/haproxy/issues/1405

пт, 8 окт. 2021 г. в 22:23, Willy Tarreau :

> Hi,
>
> HAProxy 2.5-dev9 was released on 2021/10/08. It added 162 new commits
> after version 2.5-dev8.
>
> This brings the last round of possibly breaking changes. From this point
> we should be careful not to change significant stuff and only to finish
> what was begun, fix bugs, and perform some cleanups and doc updates,
> especially since there has been a growing number of issues lately, some
> of which might have accumulated due to developers being busy finishing
> their changes and also because we're seeing an increase of feature
> requests that take time to review and/or qualify. Thus my hope for next
> versions is to see this number of issues go down, and likely a lot of
> the small pending stuff completed.
>
> This version looks large but it's mostly due to some recent pain with
> includes (recurring issue) that managed to put a halt to the progress on
> thread-groups. However as usual, many files are touched to move stuff
> around but if it builds it's doesn't bring anything, otherwise it breaks
> and we discover that some fixes are missing :-)  The nice part in this is
> that the routine build times dropped by ~38%, showing that code hygiene
> ultimately pays off. If we keep that stuff away, and the usual bugs fixed
> in every version, we're left mostly with:
>
>   - initial support for a thread group in front of the "thread" keyword
> on "bind" lines, and for the "thread-groups" directive in the config.
> For now it has no visible effect (group limited to 1 by default) but
> it will help maintain compatible configs with future versions, that
> will ease migrations back and forth.
>
>   - HTTP/1 updates to comply with latest updates to the spec:
> Transfer-Encoding should not appear with HTTP/1.0 and can be abused
> depending on how other intermediaries parse it; now a request or
> response featuring a Transfer-Encoding header will automatically be
> the last one on the connection. Similarly, since Content-Length is
> forbidden to send together with Transfer-Encoding, seeing them both
> implies talking with a non-conforming agent. The connection will also
> be closed after the transfer in this case. The "TE" header is sanitized
> to make sure not to advertise unsupported encodings to the server. And
> unsupported encodings in requests or responses will be rejected to
> prevent cache pollution or corrupted transfers.
>
>   - A number of improvements and fixes were brought to the http client
> (both Lua an native), mostly on resource freeing.
>
>   - a new batch of QUIC fixes was merged, which mainly focuses on resource
> freeing.
>
>   - "show pools" on the CLI will indicate what part of the "used" value
> represents free memory in thread-local caches; some users were confused
> into thinking they were facing a leak, and it's not normal that we only
> report confusing information there.
>
>   - the "ssl_bc_hsk_err" sample fetch introduced in 2.5-dev6 was renamed to
> "ssl_bc_err" because it will report more than just handshake errors in
> TLS 1.3. Now SSL errors should be more accurate, especially when they
> involve a peer rejecting a certificate.
>
>   - 3 regtests were added and 3 other ones fixed and re-enabled.
>
>   - the "conn_cur" stick-table data is not learned anymore from other
> peers.
> This was a flaw since this element became replicable, which has caused
> a number of questions (and even fixes). It represents a gauge that
> corresponds to the number of currently active connections tracking a
> key on the local peer, or on the one that pushed it last. Writing a
> value from another peer here only results in the entry reaching zero
> before the end, or worse, not being able to reach zero because the
> value
> is higher than the local number of connections on the key. This happens
> quite often during reloads or in active-backup setups so let's put an
> end to this mistake. It's still emitted though, in case users developed
> monitoring systems based on the protocol, they will continue to work.
> The patch is trivial to backport, if some users are annoyed enough by
> the current behavior, we could discuss about backporting it (but not
> too far, say 2.4 max).
>
>   - usual small batch of doc updates
>
> For the pending stuff, I have a local list of small trivial things to be
> done that are independent on the release and that can get merged as they
> are done. There's the set-src/set-dst stuff to be fixed (discussed in issue
> #1303), enabling support for set-var() in "tcp-request connection", and
> checking with Björn if we can get MPTCP finished in time (I think so but
> as usual there's still some work to be done for both of us). This one
> should 

Re: [PATCH v2] BUILD: SSL: function "ERR_func_error_string" is deprecated in OpenSSL-3.0.0

2021-10-07 Thread Илья Шипицин
чт, 7 окт. 2021 г. в 12:49, Willy Tarreau :

> On Thu, Oct 07, 2021 at 11:30:54AM +0500,  ??? wrote:
> > > Just thinking about something, given that the new API was already
> adopted
> > > by BoringSSL and will probably be at some point in time by LibreSSL,
> would
> > > it not be better to have a single macro "HA_SSL_USE_API_V3" or
> something
> > > like this that we set based on the various libs' versions, and rely on
> this
> > > one for all other defines ? I think it could significantly simplify the
> > > porting to other libs and avoid a real mess with version numbers
> > > everywhere.
> > >
> >
> > even BoringSSL states "it adopted upstream changes", it is different in
> > details, for example ERR_func_error_string
> > is not deprecated in BoringSSL.
> >
> > Well, there might be a common divisor of course. I'll keep an eye on it
> :)
> >
> > as for this particular patch, it is openssl specific (at least now)
>
> OK. I'll let the SSL maintainers deal with this.
>

I set up Fedora Rawhide builds.
Fedora now uses openssl-3.0.0, all builds are bloody murder:

fedora_clang (#1657157053) · Jobs · Ilya Shipitsin / haproxy-ci-playground
· GitLab




>
> Thanks!
> Willy
>


Re: [PATCH v2] BUILD: SSL: function "ERR_func_error_string" is deprecated in OpenSSL-3.0.0

2021-10-07 Thread Илья Шипицин
чт, 7 окт. 2021 г. в 10:58, Willy Tarreau :

> Hi Ilya,
>
> On Wed, Oct 06, 2021 at 11:26:13PM +0500, Ilya Shipitsin wrote:
> > +/* ERR_func_error_string is deprecated in OpenSSL-3.0.0 */
> > +#if (OPENSSL_VERSION_NUMBER >= 0x3000L)
> > +#define HA_ERR_func_error_string(ret) "OPENSSL_internal"
> > +#else
> > +#define HA_ERR_func_error_string(ret) ERR_func_error_string(ret)
> > +#endif
>
> Just thinking about something, given that the new API was already adopted
> by BoringSSL and will probably be at some point in time by LibreSSL, would
> it not be better to have a single macro "HA_SSL_USE_API_V3" or something
> like this that we set based on the various libs' versions, and rely on this
> one for all other defines ? I think it could significantly simplify the
> porting to other libs and avoid a real mess with version numbers
> everywhere.
>

even BoringSSL states "it adopted upstream changes", it is different in
details, for example ERR_func_error_string
is not deprecated in BoringSSL.

Well, there might be a common divisor of course. I'll keep an eye on it :)

as for this particular patch, it is openssl specific (at least now)



>
> Just my two cents,
> Willy
>


Re: executable properties (checksec, BinSkim)

2021-10-06 Thread Илья Шипицин
No interest :) ?

On Sat, Sep 18, 2021, 3:05 PM Илья Шипицин  wrote:

> Hello,
>
> I checked how looks binary shipped in several popular distributions
> (ppa:vbernat/haproxy-2.4, docker haproxytech/haproxy-ubuntu, docker
> haproxy).
>
> are we aware of those security features ? shall we move them to Makefile ?
> or is it up to distribution ?
>
>
> ppa:vbernat/haproxy-2.4
>
> [root@fedora haproxy-bionic]# ~ilia/checksec.sh/checksec --file=haproxy
> RELRO   STACK CANARY  NXPIE RPATH
>  RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
> Full RELRO  Canary found  NX enabledPIE enabled No RPATH
> No RUNPATH   No Symbols  Yes 12 26 haproxy
>
> BinSkim:
> Analyzing 'haproxy'...
> Analysis completed successfully.
>
>
> docker haproxytech/haproxy-ubuntu
>
> [fedora haproxy-docker]# ~ilia/checksec.sh/checksec --file=haproxy-tech
> RELRO   STACK CANARY  NXPIE RPATH
>  RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
> Full RELRO  Canary found  NX enabledPIE enabled No RPATH
> No RUNPATH   5664) Symbols  Yes 12 26 haproxy-tech
>
> BinSkim
> Analyzing 'haproxy-tech'...
> /home/ilia/haproxy-docker/haproxy-tech: error BA3004: 'haproxy-tech' is
> using debugging dwarf version '4'. The dwarf version 5 contains more
> information and should be used. To enable the debugging version 5 use
> '-gdwarf-5'.
> Analysis completed successfully.
>
> docker haproxy
>
> [ilia@fedora checksec.sh]$ ./checksec
> --file=/home/ilia/haproxy-docker/haproxy
> RELRO   STACK CANARY  NXPIE RPATH
>  RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
> Partial RELRO   No canary found   NX enabledPIE enabled No RPATH
> No RUNPATH   5926) Symbols  Yes 0 20 /home/ilia/haproxy-docker/haproxy
>
> BinSkim
>
> /home/ilia/haproxy-docker/haproxy: error BA3003: The stack protector was
> not found in 'haproxy'. This may be because '--stack-protector-strong' was
> not used, or because it was explicitly disabled by '-fno-stack-protectors'.
> Modules did not meet the criteria: slz.c, ev_poll.c, ev_epoll.c, cpuset.c,
> ssl_sample.c, ssl_sock.c, ssl_crtlist.c, ssl_ckch.c, ssl_utils.c,
> cfgparse-ssl.c, hlua.c, hlua_fcn.c, service-prometheus.c, namespace.c,
> mux_h2.c, mux_fcgi.c, http_ana.c, mux_h1.c, stream.c, tcpcheck.c, stats.c,
> flt_spoe.c, server.c, tools.c, sample.c, log.c, backend.c, stick_table.c,
> cfgparse.c, peers.c, cli.c, pattern.c, resolvers.c, proxy.c, http_htx.c,
> check.c, cache.c, cfgparse-listen.c, haproxy.c, http_act.c,
> stream_interface.c, http_fetch.c, listener.c, dns.c, connection.c,
> tcp_rules.c, debug.c, sink.c, payload.c, mux_pt.c, filters.c, fcgi-app.c,
> server_state.c, vars.c, map.c, cfgparse-global.c, task.c, flt_http_comp.c,
> session.c, sock.c, cfgcond.c, flt_trace.c, acl.c, trace.c, http_rules.c,
> queue.c, mjson.c, h2.c, h1.c, mworker.c, lb_chash.c, ring.c, activity.c,
> tcp_sample.c, proto_tcp.c, htx.c, h1_htx.c, extcheck.c, channel.c,
> proto_sockpair.c, fd.c, compression.c, mqtt.c, tcp_act.c, raw_sock.c,
> frontend.c, http_conv.c, xprt_handshake.c, pool.c, applet.c, mailers.c,
> lb_fwrr.c, lb_fwlc.c, lb_fas.c, proto_uxst.c, http.c, action.c, protocol.c,
> thread.c, sock_unix.c, proto_udp.c, lb_map.c, sock_inet.c, lru.c,
> cfgparse-tcp.c, cfgdiag.c, proto_uxdg.c, ev_select.c, cfgparse-unix.c,
> uri_normalizer.c, ebmbtree.c, sha1.c, time.c, signal.c, mworker-prog.c,
> hpack-dec.c, fix.c, arg.c, eb64tree.c, chunk.c, shctx.c, regex.c, fcgi.c,
> eb32tree.c, eb32sctree.c, dynbuf.c, uri_auth.c, hpack-tbl.c, ebimtree.c,
> auth.c, ebsttree.c, ebistree.c, base64.c, wdt.c, pipe.c, http_acl.c,
> hpack-enc.c, dict.c, dgram.c, init.c, hpack-huff.c, freq_ctr.c, ebtree.c,
> hash.c, version.c, errors.c, http_client.c
> /home/ilia/haproxy-docer/haproxy: error BA3004: 'haproxy' is using
> debugging dwarf version '4'. The dwarf version 5 contains more information
> and should be used. To enable the debugging version 5 use '-gdwarf-5'.
> /home/ilia/haproxy-docer/haproxy: error BA3005: The Stack Clash Protection
> is missing from this binary, so the stack from 'haproxy' can clash/colide
> with another memory region. Ensure you are compiling with the compiler
> flags '-fstack-clash-protection' to address this.
> Modules did not meet the criteria: slz.c, ev_poll.c, ev_epoll.c, cpuset.c,
> ssl_sample.c, ssl_sock.c, ssl_crtlist.c, ssl_ckch.c, ssl_utils.c,
> cfgparse-ssl.c, hlua.c, hlua_fcn.c, service-prometheus.c, namespace.c,
> mux_h2.c, mux_fcgi.c, http_ana.c, mux_h1.c, stream.c, tcpcheck.c, stats.c,
> flt_spoe.c, server.c, tools.c, sample.c, log.c, backend.c, stick_table.c,
> cfgparse.c, peers.c, cli.c, pattern.c, resolvers.c,

Re: [PATCH] guard "ERR_func_error_string" for OpenSSL-3.0.0 no deprecated mode

2021-09-24 Thread Илья Шипицин
пт, 24 сент. 2021 г. в 20:23, Willy Tarreau :

> On Fri, Sep 24, 2021 at 08:09:29PM +0500,  ??? wrote:
> > ??, 24 . 2021 ?. ? 19:49, Willy Tarreau :
> >
> > > On Fri, Sep 24, 2021 at 07:14:40PM +0500,  ??? wrote:
> > > > > I'd really prefer that we address all this API stuff through the
> > > > > openssl-compat stuff, so that over time we can more easily drop
> > > > > unneeded stuff. Above that could be done this way:
> > > > >
> > > > >   #if (OPENSSL_VERSION_NUMBER >= 0x3000L)
> > > > >   #  define ERR_func_error_string(ret) "OPENSSL_internal"
> > > > >   #endif
> > > > >
> > > >
> > > >
> > > > This introduces dangerous dependency on ERR_func_error_string being
> > > > substituted by preprocessor before it is passed to the compiler (or
> not)
> > >
> > > If it were defined you wouldn't have to work around it. And if you're
> > > worried that it may still be defined in some cases (which I perfectly
> > > understand), then you can just prepend a #undef before the #define.
> > >
> >
> > I believe it is a function (at least for earlier openssl)
>
> G... so we'll have to provide a wrapper outselves :-/
>


I thought that it would be overcomplicating, but I've got your idea.
I will send v2 this weekend or early next week


>
> Willy
>


Re: [PATCH] guard "ERR_func_error_string" for OpenSSL-3.0.0 no deprecated mode

2021-09-24 Thread Илья Шипицин
пт, 24 сент. 2021 г. в 19:49, Willy Tarreau :

> On Fri, Sep 24, 2021 at 07:14:40PM +0500,  ??? wrote:
> > > I'd really prefer that we address all this API stuff through the
> > > openssl-compat stuff, so that over time we can more easily drop
> > > unneeded stuff. Above that could be done this way:
> > >
> > >   #if (OPENSSL_VERSION_NUMBER >= 0x3000L)
> > >   #  define ERR_func_error_string(ret) "OPENSSL_internal"
> > >   #endif
> > >
> >
> >
> > This introduces dangerous dependency on ERR_func_error_string being
> > substituted by preprocessor before it is passed to the compiler (or not)
>
> If it were defined you wouldn't have to work around it. And if you're
> worried that it may still be defined in some cases (which I perfectly
> understand), then you can just prepend a #undef before the #define.
>

I believe it is a function (at least for earlier openssl)


>
> Cheers,
> Willy
>


Re: [PATCH] guard "ERR_func_error_string" for OpenSSL-3.0.0 no deprecated mode

2021-09-24 Thread Илья Шипицин
пт, 24 сент. 2021 г. в 18:44, Willy Tarreau :

> Hi Ilya,
>
> On Mon, Sep 20, 2021 at 10:37:04PM +0500,  ??? wrote:
> > Subject: [PATCH] BUILD: SSL: function "ERR_func_error_string" is
> deprecated in
> >  OpenSSL-3.0.0
> >
> > let us prepare for using OpenSSL-3.0.0 in no deprecation mode
> > ---
> >  src/ssl_sock.c | 7 ++-
> >  1 file changed, 6 insertions(+), 1 deletion(-)
> >
> > diff --git a/src/ssl_sock.c b/src/ssl_sock.c
> > index a87d70b89..79b8b53ca 100644
> > --- a/src/ssl_sock.c
> > +++ b/src/ssl_sock.c
> > @@ -606,7 +606,12 @@ static forceinline void ssl_sock_dump_errors(struct
> connection *conn)
> >   return;
> >   fprintf(stderr, "fd[%#x] OpenSSL error[0x%lx] %s:
> %s\n",
> >   conn->handle.fd, ret,
> > - ERR_func_error_string(ret),
> ERR_reason_error_string(ret));
> > +#if (OPENSSL_VERSION_NUMBER >= 0x3000L)
> > + "OPENSSL_internal",
> > +#else
> > + ERR_func_error_string(ret),
> > +#endif
> > + ERR_reason_error_string(ret));
>
> I'd really prefer that we address all this API stuff through the
> openssl-compat stuff, so that over time we can more easily drop
> unneeded stuff. Above that could be done this way:
>
>   #if (OPENSSL_VERSION_NUMBER >= 0x3000L)
>   #  define ERR_func_error_string(ret) "OPENSSL_internal"
>   #endif
>


This introduces dangerous dependency on ERR_func_error_string being
substituted by preprocessor before it is passed to the compiler (or not)


>
> This will also help us deal with the various forks that will sooner or
> later start to adopt the new API.
>
> thanks!
> Willy
>


Re: HA-Proxy inquiry

2021-09-22 Thread Илья Шипицин
hello,

there are several tutorials to start with, for example HAProxy version
2.4.0 - Starter Guide (cbonte.github.io)


ср, 22 сент. 2021 г. в 10:16, Lhendup Norbu :

> Dear Sir/Madan,
>
>
>
> I am Lhendup Norbu working in Bank of Bhutan under Data Center Division.
> We want to do POC with the HA proxy load balancer in our environment.
>
> Please guide us on the way forward in HA-Proxy Load Balancer.
>
>
>
>
>
> *Warm Regards*
>
>
>
> Lhendup Norbu
>
> IT Officer, Data Center Division
>
> IT Department
>
> *Bank of Bhutan Limited *
> Data Center, Thimphu : Kingdom of Bhutan
>
> *+975 77281157, IP -0060*
>
> *http://www.bob.bt *
>
>
>
>
> The information in this mail is strictly confidential and is intended
> solely for the addressee(s). Access to this mail by anyone else is
> unauthorized. Copying or further distribution beyond the original
> recipient(s) may be unlawful. Please note that any unauthorized
> addressee(s) needs a specific written consent for further circulation of
> the information(s). Any opinion expressed in this mail is that of sender
> and does not necessarily reflect that of the Bank of Bhutan Limited
>


[PATCH] guard "ERR_func_error_string" for OpenSSL-3.0.0 no deprecated mode

2021-09-20 Thread Илья Шипицин
Hello,

Fedora Rawhide now is shipped with OpenSSL-3.0.0 :)
let us fix deprecations one by one.


thanks,
Ilya
From 1d437e57e43e9c2f38977c373404e167f8230a08 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Mon, 20 Sep 2021 22:27:35 +0500
Subject: [PATCH] BUILD: SSL: function "ERR_func_error_string" is deprecated in
 OpenSSL-3.0.0

let us prepare for using OpenSSL-3.0.0 in no deprecation mode
---
 src/ssl_sock.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index a87d70b89..79b8b53ca 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -606,7 +606,12 @@ static forceinline void ssl_sock_dump_errors(struct 
connection *conn)
return;
fprintf(stderr, "fd[%#x] OpenSSL error[0x%lx] %s: %s\n",
conn->handle.fd, ret,
-   ERR_func_error_string(ret), 
ERR_reason_error_string(ret));
+#if (OPENSSL_VERSION_NUMBER >= 0x3000L)
+   "OPENSSL_internal",
+#else
+   ERR_func_error_string(ret),
+#endif
+   ERR_reason_error_string(ret));
}
}
 }
-- 
2.29.2.windows.2



executable properties (checksec, BinSkim)

2021-09-18 Thread Илья Шипицин
Hello,

I checked how looks binary shipped in several popular distributions
(ppa:vbernat/haproxy-2.4, docker haproxytech/haproxy-ubuntu, docker
haproxy).

are we aware of those security features ? shall we move them to Makefile ?
or is it up to distribution ?


ppa:vbernat/haproxy-2.4

[root@fedora haproxy-bionic]# ~ilia/checksec.sh/checksec --file=haproxy
RELRO   STACK CANARY  NXPIE RPATH
 RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
Full RELRO  Canary found  NX enabledPIE enabled No RPATH
No RUNPATH   No Symbols  Yes 12 26 haproxy

BinSkim:
Analyzing 'haproxy'...
Analysis completed successfully.


docker haproxytech/haproxy-ubuntu

[fedora haproxy-docker]# ~ilia/checksec.sh/checksec --file=haproxy-tech
RELRO   STACK CANARY  NXPIE RPATH
 RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
Full RELRO  Canary found  NX enabledPIE enabled No RPATH
No RUNPATH   5664) Symbols  Yes 12 26 haproxy-tech

BinSkim
Analyzing 'haproxy-tech'...
/home/ilia/haproxy-docker/haproxy-tech: error BA3004: 'haproxy-tech' is
using debugging dwarf version '4'. The dwarf version 5 contains more
information and should be used. To enable the debugging version 5 use
'-gdwarf-5'.
Analysis completed successfully.

docker haproxy

[ilia@fedora checksec.sh]$ ./checksec
--file=/home/ilia/haproxy-docker/haproxy
RELRO   STACK CANARY  NXPIE RPATH
 RUNPATH Symbols FORTIFY Fortified Fortifiable FILE
Partial RELRO   No canary found   NX enabledPIE enabled No RPATH
No RUNPATH   5926) Symbols  Yes 0 20 /home/ilia/haproxy-docker/haproxy

BinSkim

/home/ilia/haproxy-docker/haproxy: error BA3003: The stack protector was
not found in 'haproxy'. This may be because '--stack-protector-strong' was
not used, or because it was explicitly disabled by '-fno-stack-protectors'.
Modules did not meet the criteria: slz.c, ev_poll.c, ev_epoll.c, cpuset.c,
ssl_sample.c, ssl_sock.c, ssl_crtlist.c, ssl_ckch.c, ssl_utils.c,
cfgparse-ssl.c, hlua.c, hlua_fcn.c, service-prometheus.c, namespace.c,
mux_h2.c, mux_fcgi.c, http_ana.c, mux_h1.c, stream.c, tcpcheck.c, stats.c,
flt_spoe.c, server.c, tools.c, sample.c, log.c, backend.c, stick_table.c,
cfgparse.c, peers.c, cli.c, pattern.c, resolvers.c, proxy.c, http_htx.c,
check.c, cache.c, cfgparse-listen.c, haproxy.c, http_act.c,
stream_interface.c, http_fetch.c, listener.c, dns.c, connection.c,
tcp_rules.c, debug.c, sink.c, payload.c, mux_pt.c, filters.c, fcgi-app.c,
server_state.c, vars.c, map.c, cfgparse-global.c, task.c, flt_http_comp.c,
session.c, sock.c, cfgcond.c, flt_trace.c, acl.c, trace.c, http_rules.c,
queue.c, mjson.c, h2.c, h1.c, mworker.c, lb_chash.c, ring.c, activity.c,
tcp_sample.c, proto_tcp.c, htx.c, h1_htx.c, extcheck.c, channel.c,
proto_sockpair.c, fd.c, compression.c, mqtt.c, tcp_act.c, raw_sock.c,
frontend.c, http_conv.c, xprt_handshake.c, pool.c, applet.c, mailers.c,
lb_fwrr.c, lb_fwlc.c, lb_fas.c, proto_uxst.c, http.c, action.c, protocol.c,
thread.c, sock_unix.c, proto_udp.c, lb_map.c, sock_inet.c, lru.c,
cfgparse-tcp.c, cfgdiag.c, proto_uxdg.c, ev_select.c, cfgparse-unix.c,
uri_normalizer.c, ebmbtree.c, sha1.c, time.c, signal.c, mworker-prog.c,
hpack-dec.c, fix.c, arg.c, eb64tree.c, chunk.c, shctx.c, regex.c, fcgi.c,
eb32tree.c, eb32sctree.c, dynbuf.c, uri_auth.c, hpack-tbl.c, ebimtree.c,
auth.c, ebsttree.c, ebistree.c, base64.c, wdt.c, pipe.c, http_acl.c,
hpack-enc.c, dict.c, dgram.c, init.c, hpack-huff.c, freq_ctr.c, ebtree.c,
hash.c, version.c, errors.c, http_client.c
/home/ilia/haproxy-docer/haproxy: error BA3004: 'haproxy' is using
debugging dwarf version '4'. The dwarf version 5 contains more information
and should be used. To enable the debugging version 5 use '-gdwarf-5'.
/home/ilia/haproxy-docer/haproxy: error BA3005: The Stack Clash Protection
is missing from this binary, so the stack from 'haproxy' can clash/colide
with another memory region. Ensure you are compiling with the compiler
flags '-fstack-clash-protection' to address this.
Modules did not meet the criteria: slz.c, ev_poll.c, ev_epoll.c, cpuset.c,
ssl_sample.c, ssl_sock.c, ssl_crtlist.c, ssl_ckch.c, ssl_utils.c,
cfgparse-ssl.c, hlua.c, hlua_fcn.c, service-prometheus.c, namespace.c,
mux_h2.c, mux_fcgi.c, http_ana.c, mux_h1.c, stream.c, tcpcheck.c, stats.c,
flt_spoe.c, server.c, tools.c, sample.c, log.c, backend.c, stick_table.c,
cfgparse.c, peers.c, cli.c, pattern.c, resolvers.c, proxy.c, http_htx.c,
check.c, cache.c, cfgparse-listen.c, haproxy.c, http_act.c,
stream_interface.c, http_fetch.c, listener.c, dns.c, connection.c,
tcp_rules.c, debug.c, sink.c, payload.c, mux_pt.c, filters.c, fcgi-app.c,
server_state.c, vars.c, map.c, cfgparse-global.c, task.c, flt_http_comp.c,
session.c, sock.c, cfgcond.c, flt_trace.c, acl.c, trace.c, http_rules.c,
queue.c, mjson.c, h2.c, h1.c, mworker.c, lb_chash.c, ring.c, activity.c,
tcp_sample.c, proto_tcp.c, htx.c, h1_htx.c, extcheck.c, channel.c,

Re: BoringSSL commit dddb60e breaks compilation of HAProxy

2021-09-08 Thread Илья Шипицин
ср, 8 сент. 2021 г. в 13:54, Willy Tarreau :

> On Wed, Sep 08, 2021 at 12:05:23PM +0500,  ??? wrote:
> > Hello, Bob
> >
> > I tracked an issue  https://github.com/haproxy/haproxy/issues/1386
> >
> >
> > let's track activity there
>
> Quite frankly, I'm seriously wondering how long we'll want to keep
> supporting that constantly breaking library. Does it still provide
>

by "let us track activity" I do not mean that we are going to maintain
BoringSSL :)

people will come from time to time with BoringSSL support request. Existing
github issue is good to redirect them to.



> any value that I'm not aware of ? It's not even possible to have
> the same version at the beginning and at the end of a maintenance
> branch, it's designed to constantly change and documented as such.
>
> I'd be strongly in favor for dropping it once it becomes too painful
> to deal with, and I sense that we're starting to get closer to that
> point now.
>
> Willy
>


Re: BoringSSL commit dddb60e breaks compilation of HAProxy

2021-09-08 Thread Илья Шипицин
Hello, Bob

I tracked an issue  https://github.com/haproxy/haproxy/issues/1386


let's track activity there

вт, 7 сент. 2021 г. в 22:58, Zakharychev, Bob :

> BoringSSL commit dddb60e, "Make most of crypto/x509 opaque.", breaks
> compilation of HAProxy with the following errors (log from compiling
> HAProxy 2.4.4 with BoringSSL master branch commit a03c34c, but I suppose
> all other versions are equally affected):
>
>
> …
>
>   CC  src/ssl_sample.o
>
> In file included from include/haproxy/listener-t.h:37,
>
>  from include/haproxy/server-t.h:36,
>
>  from include/haproxy/lb_map-t.h:26,
>
>  from include/haproxy/backend-t.h:30,
>
>  from include/haproxy/proxy-t.h:35,
>
>  from include/haproxy/applet-t.h:31,
>
>  from include/haproxy/obj_type.h:26,
>
>  from src/ssl_sample.c:27:
>
> include/haproxy/openssl-compat.h: In function ‘X509_OBJECT_get0_X509_CRL’:
>
> include/haproxy/openssl-compat.h:182:23: error: dereferencing pointer to
> incomplete type ‘X509_OBJECT’ {aka ‘const struct x509_object_st’}
>
>  if (a == NULL || a->type != X509_LU_CRL) {
>
>^~
>
> src/ssl_sample.c: In function ‘smp_fetch_ssl_x_key_alg’:
>
> include/haproxy/openssl-compat.h:122:37: error: dereferencing pointer to
> incomplete type ‘X509’ {aka ‘struct x509_st’}
>
> #define X509_get_X509_PUBKEY(x) ((x)->cert_info->key)
>
>  ^~
>
> src/ssl_sample.c:716:55: note: in expansion of macro ‘X509_get_X509_PUBKEY’
>
>   X509_PUBKEY_get0_param(, NULL, NULL, NULL,
> X509_get_X509_PUBKEY(crt));
>
>^~~~
>
> make: *** [Makefile:945: src/ssl_sample.o] Error 1
>
>
>
> Indeed, BoringSSL commit dddb60e “unexported” these structs “aligning with
> OpenSSL” and directs to “Use the accessor APIs instead”. I couldn't figure
> out an easy fix to this - simply removing the two macros conditional on
> OPENSSL_IS_BORINGSSL in affected places doesn't fully help because while
> X509_get_X509_PUBKEY() accessor is now defined, X509_OBJECT_get0_X509_CRL()
> is still missing in BoringSSL. Therefore I defer the fix to HAProxy SSL
> experts - maybe it's actually BoringSSL that needs to be fixed by adding
> the missing accessor, or maybe the single loop in ssl_set_cert_crl_file()
> over all X509 store objects needs to be broken into two loops: one over
> certs returned by X509_STORE_get1_certs() and another over CRLs returned by
> X509_STORE_get1_crls().
>
> Thanks in advance for taking a stab at this,
>   Bob
>
>
>
>
>


Re: BoringSSL commit dddb60e breaks compilation of HAProxy

2021-09-07 Thread Илья Шипицин
yep :)

CI: Github Actions: temporarily disable BoringSSL builds ·
haproxy/haproxy@30ee296


I had a look, I found the same as you (no easy fix).

let us open github issue for tracking this.


вт, 7 сент. 2021 г. в 22:58, Zakharychev, Bob :

> BoringSSL commit dddb60e, "Make most of crypto/x509 opaque.", breaks
> compilation of HAProxy with the following errors (log from compiling
> HAProxy 2.4.4 with BoringSSL master branch commit a03c34c, but I suppose
> all other versions are equally affected):
>
>
> …
>
>   CC  src/ssl_sample.o
>
> In file included from include/haproxy/listener-t.h:37,
>
>  from include/haproxy/server-t.h:36,
>
>  from include/haproxy/lb_map-t.h:26,
>
>  from include/haproxy/backend-t.h:30,
>
>  from include/haproxy/proxy-t.h:35,
>
>  from include/haproxy/applet-t.h:31,
>
>  from include/haproxy/obj_type.h:26,
>
>  from src/ssl_sample.c:27:
>
> include/haproxy/openssl-compat.h: In function ‘X509_OBJECT_get0_X509_CRL’:
>
> include/haproxy/openssl-compat.h:182:23: error: dereferencing pointer to
> incomplete type ‘X509_OBJECT’ {aka ‘const struct x509_object_st’}
>
>  if (a == NULL || a->type != X509_LU_CRL) {
>
>^~
>
> src/ssl_sample.c: In function ‘smp_fetch_ssl_x_key_alg’:
>
> include/haproxy/openssl-compat.h:122:37: error: dereferencing pointer to
> incomplete type ‘X509’ {aka ‘struct x509_st’}
>
> #define X509_get_X509_PUBKEY(x) ((x)->cert_info->key)
>
>  ^~
>
> src/ssl_sample.c:716:55: note: in expansion of macro ‘X509_get_X509_PUBKEY’
>
>   X509_PUBKEY_get0_param(, NULL, NULL, NULL,
> X509_get_X509_PUBKEY(crt));
>
>^~~~
>
> make: *** [Makefile:945: src/ssl_sample.o] Error 1
>
>
>
> Indeed, BoringSSL commit dddb60e “unexported” these structs “aligning with
> OpenSSL” and directs to “Use the accessor APIs instead”. I couldn't figure
> out an easy fix to this - simply removing the two macros conditional on
> OPENSSL_IS_BORINGSSL in affected places doesn't fully help because while
> X509_get_X509_PUBKEY() accessor is now defined, X509_OBJECT_get0_X509_CRL()
> is still missing in BoringSSL. Therefore I defer the fix to HAProxy SSL
> experts - maybe it's actually BoringSSL that needs to be fixed by adding
> the missing accessor, or maybe the single loop in ssl_set_cert_crl_file()
> over all X509 store objects needs to be broken into two loops: one over
> certs returned by X509_STORE_get1_certs() and another over CRLs returned by
> X509_STORE_get1_crls().
>
> Thanks in advance for taking a stab at this,
>   Bob
>
>
>
>
>


[PATCH] spell fixes

2021-08-22 Thread Илья Шипицин
hello,

yet another spell fixes.

Ilya
From af89f34503eed1f36b0e2262bb2cef286336fbc5 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 22 Aug 2021 22:18:07 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

This is 26th iteration of typo fixes
---
 doc/lua-api/index.rst| 30 +++---
 include/haproxy/hlua-t.h |  2 +-
 src/hlua.c   | 22 +++---
 3 files changed, 27 insertions(+), 27 deletions(-)

diff --git a/doc/lua-api/index.rst b/doc/lua-api/index.rst
index 0f25760cd..d671b3222 100644
--- a/doc/lua-api/index.rst
+++ b/doc/lua-api/index.rst
@@ -1386,7 +1386,7 @@ Channel class
   retrieve a maximum of data and, if called by an action, it yields if
   necessary. It also waits for more data if the requested length exceeds the
   available amount of incoming data. Do not providing an offset is the same 
that
-  setting it to 0. A positive offset is relative to the begining of incoming
+  setting it to 0. A positive offset is relative to the beginning of incoming
   data of the channel buffer while negative offset is relative to their end.
 
   If there is no incoming data and the channel can't receive more data, a 'nil'
@@ -1426,7 +1426,7 @@ Channel class
   success or -1 if data cannot be copied.
 
   By default, if no offset is provided, the string is copied in front of
-  incoming data. A positive offset is relative to the begining of incoming data
+  incoming data. A positive offset is relative to the beginning of incoming 
data
   of the channel buffer while negative offset is relative to their end.
 
   :param class_channel channel: The manipulated Channel.
@@ -1462,7 +1462,7 @@ Channel class
   retrieve a maximum of data and, if called by an action, yields if
   necessary. It also waits for more data if the requested length exceeds the
   available amount of incoming data. Do not providing an offset is the same 
that
-  setting it to 0. A positive offset is relative to the begining of incoming
+  setting it to 0. A positive offset is relative to the beginning of incoming
   data of the channel buffer while negative offset is relative to their end.
 
   If there is no incoming data and the channel can't receive more data, a 'nil'
@@ -1512,7 +1512,7 @@ Channel class
 
   By default, if no length is provided, all incoming data, starting at the 
given
   offset, are removed. Do not providing an offset is the same that setting it
-  to 0. A positive offset is relative to the begining of incoming data of the
+  to 0. A positive offset is relative to the beginning of incoming data of the
   channel buffer while negative offset is relative to their end.
 
   :param class_channel channel: The manipulated Channel.
@@ -1526,7 +1526,7 @@ Channel class
 .. js:function:: Channel.send(channel, string)
 
   This function requires immediate send of the string **string**. It means the
-  string is copied at the begining of incoming data of the channel buffer and
+  string is copied at the beginning of incoming data of the channel buffer and
   immediately forwarded. Unless if the connection is close, and if called by an
   action, this function yields to copied and forward all the string.
 
@@ -1542,7 +1542,7 @@ Channel class
 
   By default, if no length is provided, all incoming data, starting at the 
given
   offset, are replaced. Do not providing an offset is the same that setting it
-  to 0. A positive offset is relative to the begining of incoming data of the
+  to 0. A positive offset is relative to the beginning of incoming data of the
   channel buffer while negative offset is relative to their end.
 
   :param class_channel channel: The manipulated Channel.
@@ -2082,7 +2082,7 @@ TXN class
   processing after some data have been returned to the client (eg: a redirect).
   To do so, a reply may be provided. This object is optional and may contain a
   status code, a reason, a header list and a body. All these fields are
-  optionnals. When not provided, the default values are used. By default, with
+  optional. When not provided, the default values are used. By default, with
   an empty reply object, an empty HTTP 200 response is returned to the
   client. If no reply object is provided, the transaction is terminated without
   any reply.
@@ -3169,7 +3169,7 @@ Filter class
   **context**: filter
 
   Enable the data filtering on the channel **chn** for the current filter. It
-  may be called at any time from any callback functions preceeding the data
+  may be called at any time from any callback functions proceeding the data
   analysis.
 
   :param class_Channel chn: A :ref:`channel_class`.
@@ -3208,7 +3208,7 @@ attributes:
 
 Such filter class must also define all required callback functions in the
 following list. Note that :js:func:`Filter.new()` must be defined otherwise the
-filter is ignored. Others are optionnal.
+filter is ignored. Others are optional.
 
 * .. js:function:: FILTER.new()
 
@@ -3355,7 +3355,7 @@ 

[PATCH] prepare scripts/build-ssl.sh for OpenSSL-3.0.0beta2

2021-08-21 Thread Илья Шипицин
hello,

starting with 3.0.0beta2 we need to specify libdir.

thanks,
Ilya
From 6d000345e5f738d8e35f3818843ea8ab92d54f70 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 21 Aug 2021 16:01:25 +0500
Subject: [PATCH] BUILD: adopt script/build-ssl.sh for OpenSSL-3.0.0beta2

starting with
https://github.com/openssl/openssl/commit/74b7f339aa58af57c0e71b7efca66e6f2db5ae2e,
libs are installed to "lib64", to get back required behaviour, let us
set libdir explicitly
---
 scripts/build-ssl.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/build-ssl.sh b/scripts/build-ssl.sh
index 1a1d04a67..e1d89a0eb 100755
--- a/scripts/build-ssl.sh
+++ b/scripts/build-ssl.sh
@@ -22,7 +22,7 @@ download_openssl () {
 build_openssl_linux () {
 (
 cd "openssl-${OPENSSL_VERSION}/"
-./config shared --prefix="${HOME}/opt" --openssldir="${HOME}/opt" 
-DPURIFY
+./config shared --prefix="${HOME}/opt" --openssldir="${HOME}/opt" 
--libdir=lib -DPURIFY
 if [ -z "${OPENSSL_VERSION##1.*}" ]; then
 make all
 else
@@ -36,7 +36,7 @@ build_openssl_osx () {
 (
 cd "openssl-${OPENSSL_VERSION}/"
 ./Configure darwin64-x86_64-cc shared \
---prefix="${HOME}/opt" --openssldir="${HOME}/opt" -DPURIFY
+--prefix="${HOME}/opt" --openssldir="${HOME}/opt" --libdir=lib 
-DPURIFY
 make depend build_sw install_sw
 )
 }
-- 
2.29.2.windows.2



Re: double // after domain causes ERR_HTTP2_PROTOCOL_ERROR after upgrade to 2.4.3

2021-08-20 Thread Илья Шипицин
double slashes behaviour is changed in BUG/MEDIUM: h2: match absolute-path
not path-absolute for :path · haproxy/haproxy@46b7dff (github.com)


however, Tim submitted several "normalization" patches recently.
as far as I recall, merging slashes is one of http normalizations.

Tim, can you help please ?

пт, 20 авг. 2021 г. в 16:02, Jarno Huuskonen :

> Hi,
>
> On 8/20/21 1:46 PM, Olaf Buitelaar wrote:
> > After we upgraded to haproxy version 2.4.3 from 2.4.2 urls with a double
> > slash after the domain stopped working. We're running the standard
> > docker image
> > For example;
> > https://www.example.com// 
> > https://www.example.com//some/path/  >
> > https://www.haproxy.org// 
> >
> > the browser gives a ERR_HTTP2_PROTOCOL_ERROR
> > while on 2.4.2 this worked fine. probably this has something todo with
> > the mitigations of
> > https://www.mail-archive.com/haproxy@formilux.org/msg41041.html
> > 
> >
> > our bind line looks like;
> > bind *:8443 allow-0rtt ssl crt /usr/local/etc/haproxy/xxx.bundle.pem crt
> > /usr/local/etc/haproxy/yyy.bundle.pem alpn h2,http/1.1
> > Generally our backend servers are http1.
>
> Same thing happens to me with 2.4.3 and 2.2.16.
>
> Seems to happen only for https://www.example.com// but not for
> https://www.example.com/somepath//something
>
> -Jarno
>
> --
> Jarno Huuskonen
>
>


Re: [PATCH] CI: Remove obsolete USE_SLZ=1 CI job

2021-08-15 Thread Илья Шипицин
ack from me.

сб, 14 авг. 2021 г. в 17:40, Tim Duesterhus :

> Using SLZ is a default, thus this build is equivalent to the "no features"
> build.
> ---
>  .github/matrix.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/.github/matrix.py b/.github/matrix.py
> index cfef53c9e..fdd76958c 100755
> --- a/.github/matrix.py
> +++ b/.github/matrix.py
> @@ -94,7 +94,7 @@ for CC in ["gcc", "clang"]:
>  }
>  )
>
> -for compression in ["USE_SLZ=1", "USE_ZLIB=1"]:
> +for compression in ["USE_ZLIB=1"]:
>  matrix.append(
>  {
>  "name": "{}, {}, gz={}".format(
> --
> 2.32.0
>
>
>


[PATCH] CI: relax OpenSSL version comparision

2021-08-15 Thread Илья Шипицин
we do not need strict comparison here, 3.0.0 is enough.

thanks,
Ilya
From 779e30baa8385d4484441fdb7a1069933d30be4a Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 15 Aug 2021 12:55:08 +0500
Subject: [PATCH] CI: github actions: relax OpenSSL-3.0.0 version comparision

we better to check for 3.0.0 presense, than exact version
---
 .github/matrix.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/matrix.py b/.github/matrix.py
index cd86cdd76..cca5b2e30 100755
--- a/.github/matrix.py
+++ b/.github/matrix.py
@@ -118,7 +118,7 @@ for CC in ["gcc", "clang"]:
 flags = ["USE_OPENSSL=1"]
 if ssl == "BORINGSSL=yes":
 flags.append("USE_QUIC=1")
-if ssl == "OPENSSL_VERSION=3.0.0-alpha17":
+if "OPENSSL_VERSION=3.0.0" in ssl:
 flags.append('DEBUG_CFLAGS="-g -Wno-deprecated-declarations"')
 if ssl != "stock":
 flags.append("SSL_LIB=${HOME}/opt/lib")
-- 
2.29.2.windows.2



Re: [PATCH] assorted spelling fixes

2021-08-13 Thread Илья Шипицин
Gentle ping

On Sat, Aug 7, 2021, 2:45 PM Илья Шипицин  wrote:

> Hello,
>
> yet another spelling fixes.
>
> Ilya
>


is haproxy affected by new "Request Smuggling" attack ?

2021-08-09 Thread Илья Шипицин
Hello,

HTTP/2: The Sequel is Always Worse | PortSwigger Research


just in case (I'm sure it should have been addressed already).

Ilya


Re: [PATCH] CI: travis-ci: disable arm64 builds

2021-08-09 Thread Илья Шипицин
I'm using arm64 in Oracle Cloud Ampere A1 Compute | Oracle



also, I've found promising approach (using ARM on Github Actions)  Bump
Bootstrap version from 5.0.2 to 5.1.0 · phpmyadmin/phpmyadmin@c90affe
(github.com) 

пн, 9 авг. 2021 г. в 14:29, Willy Tarreau :

> Hi Martin,
>
> On Mon, Aug 09, 2021 at 11:04:34AM +0300, Martin Grigorov wrote:
> > TravisCI just announced some improvements related to 'arch: arm64' (using
> > Equnix Metal machines) -
> https://blog.travis-ci.com/2021-08-06-oss-equinix.
>
> Thanks for the info!
>
> > But I also had some similar problems with them recently and replaced the
> > config with 'arch: arm64-graviton2; group: edge; virt: vm;', i.e. AWS
> > Graviton2 machines. In my experience they behave more stable!
>
> Yeah, these machines are really fantastic. The real problem anyway is
> likely that nowadays everyone is interested in testing on arm and very
> few have one available, let alone even a cross-compiler, so I suspect
> that these days a lot of people enable arm builds in such CI environments
> because it's the only way they have to make sure their code builds there
> at all.
>
> Cheers,
> Willy
>


[PATCH] assorted spelling fixes

2021-08-07 Thread Илья Шипицин
Hello,

yet another spelling fixes.

Ilya
From 973815c22edbb65f71d22d5022e6e02b12d8cbd4 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 7 Aug 2021 14:41:56 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

This is 25th iteration of typo fixes
---
 CONTRIBUTING| 2 +-
 INSTALL | 2 +-
 doc/SPOE.txt| 2 +-
 doc/configuration.txt   | 6 +++---
 doc/internals/fd-migration.txt  | 4 ++--
 include/import/ebmbtree.h   | 2 +-
 include/import/ebtree.h | 2 +-
 reg-tests/lua/bad_http_clt_req_duration.vtc | 2 +-
 src/cfgcond.c   | 4 ++--
 src/ebmbtree.c  | 2 +-
 src/fcgi-app.c  | 2 +-
 src/fix.c   | 2 +-
 src/lb_fwrr.c   | 2 +-
 src/mux_fcgi.c  | 2 +-
 src/peers.c | 8 
 src/xprt_quic.c | 2 +-
 tests/exp/filltab25.c   | 4 ++--
 17 files changed, 25 insertions(+), 25 deletions(-)

diff --git a/CONTRIBUTING b/CONTRIBUTING
index db89ef7bc..60a78baee 100644
--- a/CONTRIBUTING
+++ b/CONTRIBUTING
@@ -523,7 +523,7 @@ do not think about them anymore after a few patches.
should be able to tell whether or not a patch is likely to address an issue
they are facing. Indicating what the code will do after the fix doesn't help
if it does not say what problem is encountered without the patch. Note that
-   in some cases the bug is purely theorical and observed by reading the code.
+   in some cases the bug is purely theoretical and observed by reading the 
code.
In this case it's perfectly fine to provide an estimate about possible
effects. Also, in HAProxy, like many projects which take a great care of
maintaining stable branches, patches are reviewed later so that some of them
diff --git a/INSTALL b/INSTALL
index 447b9a086..cc5b894c0 100644
--- a/INSTALL
+++ b/INSTALL
@@ -178,7 +178,7 @@ compilation errors. Also, some early versions of the GNU 
libc used to include a
 regex engine which could be slow or even crash on certain patterns.
 
 If you plan on importing a particularly heavy configuration involving a lot of
-regex, you may benefit from using some alternative regex implementations sur as
+regex, you may benefit from using some alternative regex implementations such 
as
 PCRE. HAProxy natively supports PCRE and PCRE2, both in standard and JIT
 flavors (Just In Time). The following options are available depending on the
 library version provided on your system :
diff --git a/doc/SPOE.txt b/doc/SPOE.txt
index ba7aca016..4792963f4 100644
--- a/doc/SPOE.txt
+++ b/doc/SPOE.txt
@@ -450,7 +450,7 @@ timeout idle 
 timeout processing 
   Set the maximum time to wait for a stream to process an event, i.e to acquire
   a stream to talk with an agent, to encode all messages, to send the NOTIFY
-  frame, to receive the corrsponding acknowledgement and to process all
+  frame, to receive the corresponding acknowledgement and to process all
   actions. It is applied on the stream that handle the client and the server
   sessions.
 
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 9ec017334..b928ac44c 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -813,7 +813,7 @@ expression made of any combination of:
   - a non-nul integer (e.g. '1'), always returns "true".
   - a predicate optionally followed by argument(s) in parenthesis.
   - a condition placed between a pair of parenthesis '(' and ')'
-  - a question mark ('!') preceeding any of the non-empty elements above, and
+  - a question mark ('!') preceding any of the non-empty elements above, and
 which will negate its status.
   - expressions combined with a logical AND ('&&'), which will be evaluated
 from left to right until one returns false
@@ -17797,11 +17797,11 @@ fc_conn_err : integer
   Returns the ID of the error that might have occurred on the current
   connection. Any strictly positive value of this fetch indicates that the
   connection did not succeed and would result in an error log being output (as
-  decribed in section 8.2.5). See the "fc_conn_err_str" fetch for a full list 
of
+  described in section 8.2.5). See the "fc_conn_err_str" fetch for a full list 
of
   error codes and their corresponding error message.
 
 fc_conn_err_str : string
-  Returns an error message decribing what problem happened on the current
+  Returns an error message describing what problem happened on the current
   connection, resulting in a connection failure. This string corresponds to the
   "message" part of the error log format (see section 8.2.5). See below for a
   full list of error codes and their corresponding error messages :
diff --git a/doc/internals/fd-migration.txt 

[PATCH] CI: travis-ci: disable arm64 builds

2021-08-03 Thread Илья Шипицин
Hello,

it looks like "something on travis-ci side".

CC  src/raw_sock.o
gcc: fatal error: Killed signal terminated program cc1
compilation terminated.


let us disable arm64 for a while.

Ilya
From a06f34da9d0eb50a95776aafa097341ea1459430 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 3 Aug 2021 14:54:09 +0500
Subject: [PATCH] CI: travis-ci: temporarily disable arm64 builds

few recent builds failed with "gcc: fatal error: Killed signal
terminated program cc1", let us disable arm64 builds until investigation
---
 .travis.yml | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 3083e302c..7f5110e32 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -22,10 +22,10 @@ matrix:
 arch: ppc64le
 compiler: gcc
 if: type == cron
-  - os: linux
-arch: arm64
-compiler: gcc
-if: type == cron
+#  - os: linux
+#arch: arm64
+#compiler: gcc
+#if: type == cron
   - os: linux
 arch: arm64-graviton2
 group: edge
-- 
2.29.2.windows.2



Re: Supported certificate formats?

2021-08-03 Thread Илья Шипицин
I had a look at your email domain :)

I had some experience with SAP --> SSL communication, it turned out that
SAP client disabled SNI by default (thus, if you haveSNI based ssl binding,
you just hit wrong site).

so ... full config is really welcome.

вт, 3 авг. 2021 г. в 13:41, Froehlich, Dominik :

> Hi,
>
>
>
> My question may have been a little misleading:
>
> To be clear: The HAproxy config is PEM only for both the server
> certificate and the CA-file for client certificates.
>
>
>
> The issue is that the client uses a p7b binary certificate and chain to
> connect to HAproxy. HAproxy then responds with a “unknown CA” error, even
> though the root of the client certificate is part of the CA-file.
>
>
>
> That got me to think HAproxy maybe does not support clients using non-PEM
> client certificates. But I could not find any source as to what is actually
> supported.
>
>
>
> Best regards,
>
> D
>
>
>
> *From: *Илья Шипицин 
> *Date: *Monday, 2. August 2021 at 20:14
> *To: *"Froehlich, Dominik" 
> *Subject: *Re: Supported certificate formats?
>
>
>
> if you are familiar with Wireshark, I suggest to capture Client Hello <-->
> Server Hello.
>
> certificates are displayed there, so you can see whether haproxy sends its
> certificate (and chain) or not.
>
>
>
>
>
> my money would be on "if haproxy does not complain on config, so it loaded
> it properly, including certificates"
>
>
>
> пн, 2 авг. 2021 г. в 17:28, Froehlich, Dominik  >:
>
> Hi,
>
>
>
> We have an issue with a client certificate in DER (binary) encoded PKCS7
> format (.p7b).
>
> The file contains the full certificate chain and the CA-file at HAproxy
> matches the root CA of the chain, so it should work.
>
>
>
> However, the client connecting receives an “unknown CA” alert and HAproxy
> says “SSL client certificate not trusted”
>
>
>
> My strong suspicion is that HAproxy only supports PEM (text) encoded CRT
> format when connecting but I haven’t found a definitive source
>
> in the documentation. There are only examples using PEM so assume this is
> the only supported format.
>
>
>
> Can someone confirm / deny this or point me to a list of supported formats
> for certificates?
>
>
>
> Thanks a lot,
>
> Dominik
>
>


Re: HashiCorp

2021-07-20 Thread Илья Шипицин
вт, 20 июл. 2021 г. в 17:28, Willy Tarreau :

> Hello Joe,
>
> On Tue, Jul 20, 2021 at 11:04:38AM +, Joe Siganto wrote:
> > Hi Illya,
> >
> > Please could you have our Emails removed from the subscription list? I
> will
> > have all emails with your domains from our campaigns, and as checked our
> > first email ever sent to you was on 14th of July sent by our external
> agency:
> > mailto:donna.n...@acquireb2bleads.com
> >
> > Kindly let me know if we agree on this.
>
> I don't think Ilya did anything on his own, but I rechecked and can
> confirm you are not and were never subscribed to this list, so whatever
> you receive is unrelated to this. Now please stop posting your messages
> on the public list, it's a development list, not a marketing list.
>

I've been learning how spammers interact. Kind of social experiment.
I did not send any hijacking or other malicious email.

by "did on purpose" I meant blocking your emails.



>
> Willy
>


Re: FYI: kubernetes api deprecation in 1.22

2021-07-16 Thread Илья Шипицин
after googling I came to the same conclusion as Dinko Korunic :)
I missed that one

пт, 16 июл. 2021 г. в 13:43, Aleksandar Lazic :

> On 16.07.21 10:27, Илья Шипицин wrote:
> > I wonder if Kubernetes has sort of ingress compliance test. Or is it up
> to ingress itself
>
> Yes, there is such a thing but I never used it.
> https://github.com/kubernetes-sigs/ingress-controller-conformance
>
> > On Fri, Jul 16, 2021, 1:21 PM Aleksandar Lazic  <mailto:al-hapr...@none.at>> wrote:
> >
> > Hi.
> >
> > FYI that the 1.22 have some changes which also impacts Ingress and
> Endpoints.
> >
> >
> https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22
> >
> > Regards
> > Alex
> >
>
>


Re: FYI: kubernetes api deprecation in 1.22

2021-07-16 Thread Илья Шипицин
I wonder if Kubernetes has sort of ingress compliance test. Or is it up to
ingress itself

On Fri, Jul 16, 2021, 1:21 PM Aleksandar Lazic  wrote:

> Hi.
>
> FYI that the 1.22 have some changes which also impacts Ingress and
> Endpoints.
>
> https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22
>
> Regards
> Alex
>
>


Re: Replying to spam [was: Some Spam Mail]

2021-07-15 Thread Илья Шипицин
I really wonder what they will suggest.

I'm not a spam source, since we do not have "opt in" policy, anybody can
send mail. so they do.
please address the issue properly, either change list policy or be calm
with my experiments.

чт, 15 июл. 2021 г. в 12:51, Tim Düsterhus :

> Ilya,
>
> On 7/14/21 5:20 PM, Илья Шипицин wrote:
> > Yes, go ahead
>
> Would you please stop replying to spam, especially with both the sender
> and the list in Cc? It just causes more spam down the road.
>
> Best regards
> Tim Düsterhus
>


Re: HashiCorp

2021-07-14 Thread Илья Шипицин
Yes, go ahead

On Wed, Jul 14, 2021, 6:55 PM Donna Nash 
wrote:

> Good Day,
>
>
>
> I would like to know you are interested HashiCorp Users across a range of
> industries and geographic regions.
>
>
>
> If yes we can move forward.
>
>
>
> Who we are
>
>
>
> We are a global database providing company .
>
>
>
> Hope we get positive reply from your team.
>
>
>
> Thank
>
> Donna Nash
>
> Access tech data
>
>
>
>
>


Re: [PATCH] JA3 TLS Fingerprinting

2021-07-12 Thread Илья Шипицин
JA3 is good approach, but it lacks few ideas.

we fingerprinted clients by "ssl ciphers" (all ciphers sent by client in
Client Helo) + "all client curves" (also sent by client).

however you approach is flexible enough to be extended.

пн, 12 июл. 2021 г. в 20:03, Marcin Deranek :

> Hi,
>
> Over a past few weeks I have been working on implementing JA3 compatible
> TLS Fingerprinting[1] in the HAProxy. You can find the outcome in
> attachments. Feel free to review/comment them.
> Here are some choices I made which you should be aware of:
> - I decided to go with a "modular" approach where you can build JA3
> compatible fingerprint with available fetchers/converters rather than a
> single JA3 fetcher. This makes approach more "reusable" in some other
> scenarios.
> - Each Client Hello related fetcher has option to include/exclude GREASE
> (RFC8701) values from the output. This is mainly for backward compatibility
> and ability to get "pure" data. I suspect in most cases people do not want
> GREASE values to be present in the output (which is not the case right now
> for cipherlist fetchers).
> - exclude_grease function allocates trash on demand depending on GREASE
> (RFC8701) values position in the list. We can get away without creating
> trash buffer if GREASE values are present at the very beginning and/or the
> very end of the list. I decided to allocate trash buffer only when it's
> really needed, so that's why it's creation is "hidden" inside exlude_grease
> function.
> - Now ssl_capture (next to ciphersuite) contains data about extensions, ec
> ciphers etc. One of the reasons I decided to merge all those values in a
> single ssl_capture buffer is easier control of buffer size limit. I think
> it's beneficial to have a single buffer limit for all those values rather
> than separate values for each. Having said that probably
> tune.ssl.capture-cipherlist-size needs to change it's name to eg.
> tune.ssl.capture-buffer-limit to better reflect it's function.
> - Instead of creating a new converter I decided to extend existing hex
> conveter to provide a similar functionality to bin2int. I thought this
> makes more sense as extended hex converter is fully backward compatible. It
> has to be noted that extended hex converter is not strictly necessary to
> produce JA3 TLS Fingerprint, but but might useful in some other scenarios.
>
> Example usage:
> http-request set-header X-SSL-JA3
> %[ssl_fc_protocol_hello_id],%[ssl_fc_cipherlist_bin(1),bin2int(-,2)],%[ssl_fc_extlist_bin(1),bin2int(-,2)],%[ssl_fc_eclist_bin(1),bin2int(-,2)],%[ssl_fc_ecformats_bin,bin2int(-,1)]
> http-request set-header X-SSL-JA3-Hash
> %[req.fhdr(x-ssl-ja3),digest(md5),hex]
>
> Question: I noticed that during Client Hello parsing we calculate xxh64
> right away and store it. Isn't better to calculate it when it's actually
> used?
> Regards,
>
> Marcin Deranek
>
> [1] https://github.com/salesforce/ja3
>
>


Re: Proposal about new default SSL log format

2021-07-03 Thread Илья Шипицин
сб, 3 июл. 2021 г. в 16:22, Aleksandar Lazic :

> Hi Remi.
>
> On 02.07.21 16:26, Remi Tricot-Le Breton wrote:
> > Hello list,
> >
> > Some work in ongoing to ease connection error and SSL handshake error
> logging.
> > This will rely on some new sample fetches that could be added to a custom
> > log-format string.
> > In order to ease SSL logging and debugging, we will also add a new
> default log
> > format for SSL connections. Now is then the good time to find the best
> format
> > for everyone.
> > The proposed format looks like the HTTP one to which the SSL specific
> > information is added. But if anybody sees a missing information that
> could be
> > beneficial for everybody, feel free to tell it, nothing is set in stone
> yet.
> >
> > The format would look like this :
> >  >>> Jul  1 18:11:31 haproxy[143338]: 127.0.0.1:37740
> [01/Jul/2021:18:11:31.517] \
> >ssl_frontend~ ssl_frontend/s2 0/0/0/7/+7 \
> >0/0/0/0 2750  1/1/1/1/0 0/0 TLSv1.3 TLS_AES_256_GCM_SHA384
> >
> >Field   FormatExtract from the
> example above
> >1   process_name '[' pid ']:'
>  haproxy[143338]:
> >2   client_ip ':' client_port
> 127.0.0.1:37740
> >3   '[' request_date ']'
> [01/Jul/2021:18:11:31.517]
> >4   frontend_name
> ssl_frontend~
> >5   backend_name '/' server_name
>  ssl_frontend/s2
> >6   TR '/' Tw '/' Tc '/' Tr '/' Ta*
> 0/0/0/7/+7
> >7 *conn_status '/' SSL hsk error '/' SSL vfy '/' SSL CA vfy*
> 0/0/0/0
> >8 bytes_read*
>2750
> >9 termination_state
>
> >   10   actconn '/' feconn '/' beconn '/' srv_conn '/' retries*
> 1/1/1/1/0
> >   11   srv_queue '/'
> backend_queue  0/0
> >   12 *ssl_version*
> TLSv1.3
> >   13 *ssl_ciphers*
>  TLS_AES_256_GCM_SHA384
> >
> >
> > The equivalent log-format string would be the following :
> >  "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta \
> > %[conn_err_code]/%[ssl_fc_hsk_err]/%[ssl_c_err]/%[ssl_c_ca_err] \
> >  %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq %sslv %sslc
> >
> > The fields in bold are the SSL specific ones and the statuses ones will
> come
> > from a not yet submitted code so the names and format might slightly
> change.
> >
> > Feel free to suggest any missing data, which could come from log-format
> > specific fields or already existing sample fetches.
>
> How about to combine ssl_version/ssl_ciphers in one line.
>
> It would be helpful to see also the backend status.
> Maybe add a 14th and 15th line with following fields
>
> *backend_name '/' conn_status '/' SSL hsk error '/' SSL vfy '/' SSL CA vfy*
> *backend_name '/' ssl_version '/' ssl_ciphers*
>
> I had in the past several issues with the backend where the backend CA
> wasn't in the CA File which was quite
> difficult to debug.
>
> +1 to the suggestion from Илья Шипицин to use iso8601 which is already in
> haproxy since 2019/10/01:2.1-dev2.
>
> I haven't found sub second format parameter in strftime call therefore I
> assume the strftime call have this
> ".00" as fix value.
>
> ```
> strftime(iso_time_str, sizeof(iso_time_str),
> "%Y-%m-%dT%H:%M:%S.00+00:00", )
> ```
>
> Maybe another option is to use TAI for timestamps.
>


many analysis tools, for example Microsoft LogParser, ClickHouse, can
perform queries right on top of TSV files with iso8601 time.





>
> https://en.wikipedia.org/wiki/International_Atomic_Time
> https://cr.yp.to/proto/utctai.html
> http://www.madore.org/~david/computers/unix-leap-seconds.html
>
> > Thanks
> >
> > Rémi
>
> Jm2c
>
> Alex
>
>


Re: Proposal about new default SSL log format

2021-07-02 Thread Илья Шипицин
also, "process name" is something that is prior knowledge. no need to log
it every time (for millions of requests)

пт, 2 июл. 2021 г. в 19:52, Илья Шипицин :

> I worked with log formats a lot, couple of thoughts
>
> 1) tab separated is better for any log import tool (mixing spaces and "/"
> is terrible for import)
> 2) time should be iso8601
>
> пт, 2 июл. 2021 г. в 19:29, Remi Tricot-Le Breton :
>
>> Hello list,
>>
>> Some work in ongoing to ease connection error and SSL handshake error
>> logging.
>> This will rely on some new sample fetches that could be added to a custom
>> log-format string.
>> In order to ease SSL logging and debugging, we will also add a new
>> default log
>> format for SSL connections. Now is then the good time to find the best
>> format
>> for everyone.
>> The proposed format looks like the HTTP one to which the SSL specific
>> information is added. But if anybody sees a missing information that
>> could be
>> beneficial for everybody, feel free to tell it, nothing is set in stone
>> yet.
>>
>> The format would look like this :
>> >>> Jul  1 18:11:31 haproxy[143338]: 127.0.0.1:37740
>> [01/Jul/2021:18:11:31.517] \
>>   ssl_frontend~ ssl_frontend/s2 0/0/0/7/+7 \
>>   0/0/0/0 2750  1/1/1/1/0 0/0 TLSv1.3 TLS_AES_256_GCM_SHA384
>>
>>   Field   FormatExtract from the example
>> above
>>   1   process_name '[' pid ']:'
>> haproxy[143338]:
>>   2   client_ip ':' client_port
>> 127.0.0.1:37740
>>   3   '[' request_date ']'
>> [01/Jul/2021:18:11:31.517]
>>   4   frontend_name
>> ssl_frontend~
>>   5   backend_name '/' server_name
>> ssl_frontend/s2
>>   6   TR '/' Tw '/' Tc '/' Tr '/' Ta*
>> 0/0/0/7/+7
>>   7   *conn_status '/' SSL hsk error '/' SSL vfy '/' SSL CA vfy*
>> 0/0/0/0
>>   8   bytes_read*
>> 2750
>>   9   termination_state
>> 
>>  10   actconn '/' feconn '/' beconn '/' srv_conn '/' retries*
>> 1/1/1/1/0
>>  11   srv_queue '/'
>> backend_queue  0/0
>>  12   *ssl_version*
>> TLSv1.3
>>  13   *ssl_ciphers*
>> TLS_AES_256_GCM_SHA384
>>
>>
>> The equivalent log-format string would be the following :
>> "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta \
>> %[conn_err_code]/%[ssl_fc_hsk_err]/%[ssl_c_err]/%[ssl_c_ca_err] \
>> %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq %sslv %sslc
>>
>> The fields in bold are the SSL specific ones and the statuses ones will
>> come
>> from a not yet submitted code so the names and format might slightly
>> change.
>>
>> Feel free to suggest any missing data, which could come from log-format
>> specific fields or already existing sample fetches.
>>
>> Thanks
>>
>> Rémi
>>
>


Re: Proposal about new default SSL log format

2021-07-02 Thread Илья Шипицин
I understand that everybody can redefine its own format.

but I saw several times that default (not very comfortable) format was
later adopted as industry wide.

пт, 2 июл. 2021 г. в 19:52, Илья Шипицин :

> I worked with log formats a lot, couple of thoughts
>
> 1) tab separated is better for any log import tool (mixing spaces and "/"
> is terrible for import)
> 2) time should be iso8601
>
> пт, 2 июл. 2021 г. в 19:29, Remi Tricot-Le Breton :
>
>> Hello list,
>>
>> Some work in ongoing to ease connection error and SSL handshake error
>> logging.
>> This will rely on some new sample fetches that could be added to a custom
>> log-format string.
>> In order to ease SSL logging and debugging, we will also add a new
>> default log
>> format for SSL connections. Now is then the good time to find the best
>> format
>> for everyone.
>> The proposed format looks like the HTTP one to which the SSL specific
>> information is added. But if anybody sees a missing information that
>> could be
>> beneficial for everybody, feel free to tell it, nothing is set in stone
>> yet.
>>
>> The format would look like this :
>> >>> Jul  1 18:11:31 haproxy[143338]: 127.0.0.1:37740
>> [01/Jul/2021:18:11:31.517] \
>>   ssl_frontend~ ssl_frontend/s2 0/0/0/7/+7 \
>>   0/0/0/0 2750  1/1/1/1/0 0/0 TLSv1.3 TLS_AES_256_GCM_SHA384
>>
>>   Field   FormatExtract from the example
>> above
>>   1   process_name '[' pid ']:'
>> haproxy[143338]:
>>   2   client_ip ':' client_port
>> 127.0.0.1:37740
>>   3   '[' request_date ']'
>> [01/Jul/2021:18:11:31.517]
>>   4   frontend_name
>> ssl_frontend~
>>   5   backend_name '/' server_name
>> ssl_frontend/s2
>>   6   TR '/' Tw '/' Tc '/' Tr '/' Ta*
>> 0/0/0/7/+7
>>   7   *conn_status '/' SSL hsk error '/' SSL vfy '/' SSL CA vfy*
>> 0/0/0/0
>>   8   bytes_read*
>> 2750
>>   9   termination_state
>> 
>>  10   actconn '/' feconn '/' beconn '/' srv_conn '/' retries*
>> 1/1/1/1/0
>>  11   srv_queue '/'
>> backend_queue  0/0
>>  12   *ssl_version*
>> TLSv1.3
>>  13   *ssl_ciphers*
>> TLS_AES_256_GCM_SHA384
>>
>>
>> The equivalent log-format string would be the following :
>> "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta \
>> %[conn_err_code]/%[ssl_fc_hsk_err]/%[ssl_c_err]/%[ssl_c_ca_err] \
>> %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq %sslv %sslc
>>
>> The fields in bold are the SSL specific ones and the statuses ones will
>> come
>> from a not yet submitted code so the names and format might slightly
>> change.
>>
>> Feel free to suggest any missing data, which could come from log-format
>> specific fields or already existing sample fetches.
>>
>> Thanks
>>
>> Rémi
>>
>


Re: Proposal about new default SSL log format

2021-07-02 Thread Илья Шипицин
I worked with log formats a lot, couple of thoughts

1) tab separated is better for any log import tool (mixing spaces and "/"
is terrible for import)
2) time should be iso8601

пт, 2 июл. 2021 г. в 19:29, Remi Tricot-Le Breton :

> Hello list,
>
> Some work in ongoing to ease connection error and SSL handshake error
> logging.
> This will rely on some new sample fetches that could be added to a custom
> log-format string.
> In order to ease SSL logging and debugging, we will also add a new default
> log
> format for SSL connections. Now is then the good time to find the best
> format
> for everyone.
> The proposed format looks like the HTTP one to which the SSL specific
> information is added. But if anybody sees a missing information that could
> be
> beneficial for everybody, feel free to tell it, nothing is set in stone
> yet.
>
> The format would look like this :
> >>> Jul  1 18:11:31 haproxy[143338]: 127.0.0.1:37740
> [01/Jul/2021:18:11:31.517] \
>   ssl_frontend~ ssl_frontend/s2 0/0/0/7/+7 \
>   0/0/0/0 2750  1/1/1/1/0 0/0 TLSv1.3 TLS_AES_256_GCM_SHA384
>
>   Field   FormatExtract from the example
> above
>   1   process_name '[' pid ']:'
> haproxy[143338]:
>   2   client_ip ':' client_port
> 127.0.0.1:37740
>   3   '[' request_date ']'
> [01/Jul/2021:18:11:31.517]
>   4   frontend_name
> ssl_frontend~
>   5   backend_name '/' server_name
> ssl_frontend/s2
>   6   TR '/' Tw '/' Tc '/' Tr '/' Ta*
> 0/0/0/7/+7
>   7   *conn_status '/' SSL hsk error '/' SSL vfy '/' SSL CA vfy*
> 0/0/0/0
>   8   bytes_read*
> 2750
>   9   termination_state
> 
>  10   actconn '/' feconn '/' beconn '/' srv_conn '/' retries*
> 1/1/1/1/0
>  11   srv_queue '/' backend_queue
> 0/0
>  12   *ssl_version*
> TLSv1.3
>  13   *ssl_ciphers*
> TLS_AES_256_GCM_SHA384
>
>
> The equivalent log-format string would be the following :
> "%ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta \
> %[conn_err_code]/%[ssl_fc_hsk_err]/%[ssl_c_err]/%[ssl_c_ca_err] \
> %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq %sslv %sslc
>
> The fields in bold are the SSL specific ones and the statuses ones will
> come
> from a not yet submitted code so the names and format might slightly
> change.
>
> Feel free to suggest any missing data, which could come from log-format
> specific fields or already existing sample fetches.
>
> Thanks
>
> Rémi
>


Re: SSL Labs says my server isn't doing ssl session resumption

2021-06-20 Thread Илья Шипицин
вс, 20 июн. 2021 г. в 11:43, Shawn Heisey :

> On 6/17/2021 1:01 AM, Willy Tarreau wrote:
> > I don't know if the config is responsible for this but I've just tested
> > on haproxy.org and it does work there:
> >
> >Session resumption (caching)   Yes
> >Session resumption (tickets)   Yes
>
> Many thanks to everyone who replied, and countless people who published
> comments and articles on the Internet.  You're awesome.
>
> I've managed to get my SSL Labs grade to A+.  I really like testssl.sh.



it is something loved by compliance people. for example, if you have to
certify for PCI DSS,
you have to setup A+ actually


>
>   I've only got one remaining yellow item on the testssl report:
>
>   BREACH (CVE-2013-3587)potentially NOT ok, "gzip"
> HTTP compression detected. - only supplied "/" tested
>
> This is what SSL Labs now says for the thing that started this thread:
>
> Session resumption (caching)No (IDs assigned but not accepted)
> Session resumption (tickets)Yes
>
> I'd like to get the caching item fixed, but I haven't figured that out
> yet.  Hoping that getting the tickets working is enough for most
> clients.  Not that I feel TLS is slow at all, seems zippy enough.  Any
> chance that the problem is my openssl library rather than haproxy?  It's
> stock Ubuntu 18.  Here's the openssl info from haproxy -vv:
>
> Built with OpenSSL version : OpenSSL 1.1.1  11 Sep 2018
> Running on OpenSSL version : OpenSSL 1.1.1  11 Sep 2018
>
> As for the list of functional clients that SSL Labs generates, people
> using IE on anything older than Windows 10 are out of luck with my web
> pages, as are those using Safari 8 and below.  Java 6 and 7 are also
> unsupported.  I really do not care about people using those clients.
> Upgrade your Java version or get a better browser.
>


unfortunately, TLS does not allow you to signal such properly. client will
only see "unable to establish connection"
without any clear reason for them.


but you are right, it is completely up to you whether to choose
compatibility and bigger client coverage or high SSL Labs rating.


>
> Thanks,
> Shawn
>
>


Re: Speeding up opentracing build in CI ?

2021-06-17 Thread Илья Шипицин
чт, 17 июн. 2021 г. в 19:39, Willy Tarreau :

> On Thu, Jun 17, 2021 at 04:31:57PM +0200, Tim Düsterhus wrote:
> > Willy, William,
> >
> > On 6/17/21 3:55 PM, William Lallemand wrote:
> > > > OK that's a net win, openssl-3.0.0-alpha17 dropped from 8'29 to 2'55.
> > > > I've just excluded versions 1.x from both the parallel build and the
> > > > build_sw target and that's good now.
> > > >
> > > > Willy
> > >
> > > Great improvement, thanks!
> > >
> >
> > Here's a full filtered overview:
> >
> >
> https://github.com/haproxy/haproxy/actions/workflows/vtest.yml?query=branch%3Amaster
> >
> > It's down from ~11 minutes until the last build finishes to ~9 minutes.
>
> Ah indeed. Well, it even suffers variations that are larger than the
> potential savings, ranging from 10'45 to 15'00 over the last few builds.
> So we don't really know how much we're saving, we can only hope it saves
> a little bit.
>

we are in control of that.
what if we'll add "sleep 30" and remove it after while ?
nobody will tell it is "a little bit speedup"


>
> Thanks!
> Willy
>


Re: SSL Labs says my server isn't doing ssl session resumption

2021-06-16 Thread Илья Шипицин
ср, 16 июн. 2021 г. в 20:27, Lukas Tribus :

> On Wed, 16 Jun 2021 at 17:03, Илья Шипицин  wrote:
> >
> > ssl sessions are for tls1.0  (disabled in your config)
> > tls1.2 uses tls tickets for resumption
>
> That is not true, you can disable TLS tickets and still get resumption
> on TLSv1.2. Disabling TLSv1.0 does not mean disabling Session ID
> caching.
>
>
ok, I need to refresh that in memory


>
> What do you see with testssl.sh ?
>
>
> Lukas
>


Re: SSL Labs says my server isn't doing ssl session resumption

2021-06-16 Thread Илья Шипицин
ssl sessions are for tls1.0  (disabled in your config)
tls1.2 uses tls tickets for resumption

what does ssl labs say on tls tickets ?

сб, 12 июн. 2021 г. в 05:51, Shawn Heisey :

> I'm fiddling with ssl labs to see how I can improve my TLS setup.
>
> Here's what they say about a site I have behind haproxy with TLS:
>
>
> https://www.elyograg.org/foo/haproxy-ssllabs-session-resumption-not-working.png
>
> They claim that session resumption isn't working.  I'm hoping that I've
> just done something wrong... which will be bad for my ego but great for
> getting problems fixed.  I did have the option to disable tls tickets,
> but when I took it out, my ssl labs grade didn't go down, so it's still
> not there.
>
> This is what I have in the global section:
>
> global
>  log 127.0.0.1   len 65535 format rfc5424 local0
>  log 127.0.0.1   len 65535 format rfc5424 local1 notice
>  maxconn 4096
>  daemon
>  spread-checks   2
>  tune.bufsize65536
>  tune.http.logurilen 49152
>  tune.ssl.cachesize 10
>  tune.ssl.lifetime   900
>  ssl-server-verify   none
>  tune.ssl.default-dh-param   2048
>  ssl-default-bind-ciphers EECDH+AESGCM:EDH+AESGCM
>  ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11
>  ssl-default-server-ciphers
> RC4-MD5:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-SHA:AES256-SHA:AES256-SHA256
>  stats socket /etc/haproxy/stats.socket
>
> I don't think there's anything else in the config that could affect
> this, but if there is something that would help diagnose, let me know.
>
> Here's info about haproxy, which I compiled from source:
>
> root@smeagol:~# haproxy -vv
> HAProxy version 2.4.0-6cbbecf 2021/05/14 - https://haproxy.org/
> Status: long-term supported branch - will stop receiving fixes around Q2
> 2026.
> Known bugs: http://www.haproxy.org/bugs/bugs-2.4.0.html
> Running on: Linux 5.8.0-55-generic #62~20.04.1-Ubuntu SMP Wed Jun 2
> 08:55:04 UTC 2021 x86_64
> Build options :
>TARGET  = linux-glibc
>CPU = native
>CC  = cc
>CFLAGS  = -O2 -march=native -g -Wall -Wextra
> -Wdeclaration-after-statement -fwrapv -Wno-address-of-packed-member
> -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered
> -Wno-missing-field-initializers -Wno-cast-function-type -Wtype-limits
> -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond
> -Wnull-dereference
>OPTIONS = USE_PCRE_JIT=1 USE_OPENSSL=1 USE_ZLIB=1 USE_SYSTEMD=1
>DEBUG   =
>
> Feature list : +EPOLL -KQUEUE +NETFILTER -PCRE +PCRE_JIT -PCRE2
> -PCRE2_JIT +POLL -PRIVATE_CACHE +THREAD -PTHREAD_PSHARED +BACKTRACE
> -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT
> +CRYPT_H +GETADDRINFO +OPENSSL -LUA +FUTEX +ACCEPT4 -CLOSEFROM +ZLIB
> -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL
> +SYSTEMD -OBSOLETE_LINKER +PRCTL +THREAD_DUMP -EVPORTS -OT -QUIC -PROMEX
> -MEMORY_PROFILING
>
> Default settings :
>bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
>
> Built with multi-threading support (MAX_THREADS=64, default=4).
> Built with OpenSSL version : OpenSSL 1.1.1f  31 Mar 2020
> Running on OpenSSL version : OpenSSL 1.1.1f  31 Mar 2020
> OpenSSL library supports TLS extensions : yes
> OpenSSL library supports SNI : yes
> OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
> Built with network namespace support.
> Built with zlib version : 1.2.11
> Running on zlib version : 1.2.11
> Compression algorithms supported : identity("identity"),
> deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
> Built with transparent proxy support using: IP_TRANSPARENT
> IPV6_TRANSPARENT IP_FREEBIND
> Built with PCRE version : 8.39 2016-06-14
> Running on PCRE version : 8.39 2016-06-14
> PCRE library supports JIT : yes
> Encrypted password support via crypt(3): yes
> Built with gcc compiler version 9.3.0
>
> Available polling systems :
>epoll : pref=300,  test result OK
> poll : pref=200,  test result OK
>   select : pref=150,  test result OK
> Total: 3 (3 usable), will use epoll.
>
> Available multiplexer protocols :
> (protocols marked as  cannot be specified using 'proto'
> keyword)
>h2 : mode=HTTP   side=FE|BE mux=H2
> flags=HTX|CLEAN_ABRT|HOL_RISK|NO_UPG
>  fcgi : mode=HTTP   side=BEmux=FCGI
> flags=HTX|HOL_RISK|NO_UPG
>  : mode=HTTP   side=FE|BE mux=H1   flags=HTX
>h1 : mode=HTTP   side=FE|BE mux=H1
> flags=HTX|NO_UPG
>  : mode=TCPside=FE|BE mux=PASS flags=
>  none : mode=TCPside=FE|BE mux=PASS
> flags=NO_UPG
>
> Available services : none
>
> Available filters :
>  [SPOE] spoe
>  [CACHE] cache
>  [FCGI] fcgi-app
>  [COMP] compression
>  [TRACE] trace
>
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-14 Thread Илья Шипицин
That's why Travis badge in README.md makes sense

On Mon, Jun 14, 2021, 5:16 PM Willy Tarreau  wrote:

> On Mon, Jun 14, 2021 at 02:33:20PM +0200, William Lallemand wrote:
> > On Mon, Jun 14, 2021 at 12:01:11PM +0200, Tim Düsterhus wrote:
> > > We only run Travis once weekly, because of the limited credits we
> have.
> > > Thus only the newest commit at time of running will have a Travis CI
> > > Status integrated into GitHub.
> > >
> > > The most recent one is this:
> > >
> https://github.com/haproxy/haproxy/commit/1b095cac9468d0c3eeb157e9b1a2947487bd3c83
> >
> >
> > I thought it disappeared completely from the interface, good to know!
>
> Yes, it's still useful to know whether a week ago arm64, ppc64 and s390x
> were working or not. So once a week is better than nothing at all.
>
> Willy
>


Re: [PATCH 0/4] Use 'feature cmd' in regtests

2021-06-14 Thread Илья Шипицин
I believe William means conditions like "openssl is 1.1.0 or higher", but
that's possible using grep

пн, 14 июн. 2021 г. в 15:33, Tim Düsterhus :

> William,
>
> On 6/14/21 12:12 PM, William Lallemand wrote:
> >>
> >>  feature cmd "$HAPROXY_PROGRAM -vv |grep -vq BoringSSL"
> >>
> >> and then they won't run for BoringSSL (the example is untested).
> >>
> >
> > That's very interesting, I had a discussion with Willy last week about
> > this kind of problems. I think that's clearly the cleanest soluton to
> > the problem.
> >
> > In my opinion it only lacks a way to check the OpenSSL version from
> > 'haproxy -cc' so we can enable ssl/set_ssl_cert_bundle.vtc again.
> >
>
> Note that it doesn't necessarily need to be part of '-cc'. If a simple
> 'grep' is sufficient, then that works as well. Of course if you need to
> check exact versions instead of just product names a 'grep' probably is
> not sufficient.
>
> >> After this series is applied the output of 'make reg-tests' will
> change. Tests
> >> that are excluded using 'feature cmd' will still be listed as "Add
> test" in
> >> the test gathering section. However the final line will show that a few
> tests
> >> have been skipped:
> >>
> >>  0 tests failed, 4 tests skipped, 105 tests passed
> >>
> >> I don't think this is going to be an issue. But if it is, please
> complain!
> >>
> >
> > Hm the only problem I have with this, is that we won't be able to see
> why a
> > test was excluded.
> >
>
> VTest logs the reason for the SKIP in the verbose output, but it's true
> that it's not readily available (not that I bypass 'make' for this
> example):
>
> > env HAPROXY_PROGRAM=$(pwd)/haproxy /VTest/vtest -v -k -t 1
> reg-tests/ssl/new_del_ssl_crlfile.vtc reg-tests/ssl/set_ssl_crlfile.vtc
> reg-tests/ssl/set_ssl_cafile.vtc reg-tests/ssl/new_del_ssl_cafile.vtc
> reg-tests/ssl/show_ssl_ocspresponse.vtc
> reg-tests/http-messaging/http_abortonclose.vtc
> reg-tests/startup/check_condition.vtc |grep SKIPPING
>
> Results in:
>
> > *top   SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc
> 'feature(OPENSSL)'
> > *top   SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc
> 'feature(OPENSSL)'
> > *top   SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc
> 'feature(OPENSSL)'
> > *top   SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc
> 'feature(OPENSSL)'
> > *top   SKIPPING test, lacking feature: $HAPROXY_PROGRAM -cc
> 'feature(OPENSSL)'
> > 0 tests failed, 5 tests skipped, 2 tests passed
>
> Best regards
> Tim Düsterhus
>


Re: [PATCH] CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'

2021-06-13 Thread Илья Шипицин
Anyway, ACK from me.
ulimit is less evil than sudo

On Sun, Jun 13, 2021, 4:25 PM Tim Düsterhus  wrote:

> Ilya,
>
> On 6/13/21 3:18 PM, Илья Шипицин wrote:
> > It was in my to do list as well. I recall I ran tests without increasing
> > limits.
> >
> > Which test requires 5 open files? Maybe the one currently disabled?
> >
> > Also, it does not like a good pattern to open so many files. If we really
> > use as many files, shouldn't we revisit that area?
>
> The issue is that HAProxy attempts to increase the soft ulimit to the
> hard ulimit during startup. This fails on macOS which has a soft limit
> of ~10k and a hard limit of ~500k, but does not allow to actually
> increase the soft limit without being root.
>
> That's why I'm updating the ulimit manually before starting HAProxy.
> This will update both the soft as well as the hard limit, resulting in
> HAProxy not needing to do anything, thus succeeding on macOS.
>
> I surely could've used an even smaller number, but 5k is something that
> worked just fine on my first attempt, so that one it is. In fact the
> limit I set is even lower than the default of macOS.
>
> Best regards
> Tim Düsterhus
>


Re: [PATCH] CI: Replace the requirement for 'sudo' with a call to 'ulimit -n'

2021-06-13 Thread Илья Шипицин
It was in my to do list as well. I recall I ran tests without increasing
limits.

Which test requires 5 open files? Maybe the one currently disabled?

Also, it does not like a good pattern to open so many files. If we really
use as many files, shouldn't we revisit that area?

On Sun, Jun 13, 2021, 4:02 PM Tim Duesterhus  wrote:

> Using 'sudo' required quite a few workarounds in various places. Setting an
> explicit 'ulimit -n' removes the requirement for 'sudo', resulting in a
> cleaner
> workflow configuration.
> ---
>  .github/workflows/vtest.yml | 11 ++-
>  1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/.github/workflows/vtest.yml b/.github/workflows/vtest.yml
> index 786713374..1dc216eeb 100644
> --- a/.github/workflows/vtest.yml
> +++ b/.github/workflows/vtest.yml
> @@ -99,13 +99,14 @@ jobs:
>run: echo "::add-matcher::.github/vtest.json"
>  - name: Run VTest for HAProxy ${{ steps.show-version.outputs.version
> }}
>id: vtest
> -  # sudo is required, because macOS fails due to an open files limit.
> -  run: sudo make reg-tests VTEST_PROGRAM=../vtest/vtest
> REGTESTS_TYPES=default,bug,devel
> +  run: |
> +# This is required for macOS which does not actually allow to
> increase
> +# the '-n' soft limit to the hard limit, thus failing to run.
> +ulimit -n 5000
> +make reg-tests VTEST_PROGRAM=../vtest/vtest
> REGTESTS_TYPES=default,bug,devel
>  - name: Show results
>if: ${{ failure() }}
> -  # The chmod / sudo is necessary due to the `sudo` while running the
> tests.
>run: |
> -sudo chmod a+rX ${TMPDIR}/haregtests-*/
>  for folder in ${TMPDIR}/haregtests-*/vtc.*; do
>printf "::group::"
>cat $folder/INFO
> @@ -115,6 +116,6 @@ jobs:
>  shopt -s nullglob
>  for asan in asan.log*; do
>echo "::group::$asan"
> -  sudo cat $asan
> +  cat $asan
>echo "::endgroup::"
>  done
> --
> 2.32.0
>
>


[PATCH] typo fixes

2021-06-12 Thread Илья Шипицин
Hello,

yet more typo and spelling fixes.

Ilya
From 3328f8814e7d1d5c1d708c3d5c6a472e645c9d71 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 12 Jun 2021 15:55:27 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

This is 24th iteration of typo fixes
---
 src/backend.c  | 2 +-
 src/cfgparse.c | 4 ++--
 src/h1_htx.c   | 8 
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/src/backend.c b/src/backend.c
index 65beb84b7..da879eda3 100644
--- a/src/backend.c
+++ b/src/backend.c
@@ -2253,7 +2253,7 @@ void back_handle_st_cer(struct stream *s)
 * redispatch). Thus we must release the SI endpoint on the server side
 * an close the attached connection. It is especially important to do it
 * now if the retry is not immediately performed, to be sure to release
-* ressources as soon as possible and to not catch errors from the lower
+* resources as soon as possible and to not catch errors from the lower
 * layers in an unexpected state (i.e < ST_CONN).
 *
 * Note: the stream-interface will be switched to ST_REQ, ST_ASS or
diff --git a/src/cfgparse.c b/src/cfgparse.c
index 906ab076f..eda126bff 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -2155,7 +2155,7 @@ int readcfgfile(const char *file)
}
else if (strcmp(args[0], ".else") == 0) {
if (*args[1]) {
-   ha_alert("parsing [%s:%d]: Unxpected 
argument '%s' for '%s'.\n",
+   ha_alert("parsing [%s:%d]: Unexpected 
argument '%s' for '%s'.\n",
 file, linenum, args[1], 
args[0]);
err_code |= ERR_ALERT | ERR_FATAL | 
ERR_ABORT;
break;
@@ -2187,7 +2187,7 @@ int readcfgfile(const char *file)
}
else if (strcmp(args[0], ".endif") == 0) {
if (*args[1]) {
-   ha_alert("parsing [%s:%d]: Unxpected 
argument '%s' for '%s'.\n",
+   ha_alert("parsing [%s:%d]: Unexpected 
argument '%s' for '%s'.\n",
 file, linenum, args[1], 
args[0]);
err_code |= ERR_ALERT | ERR_FATAL | 
ERR_ABORT;
break;
diff --git a/src/h1_htx.c b/src/h1_htx.c
index 878c45db8..6ff69d0aa 100644
--- a/src/h1_htx.c
+++ b/src/h1_htx.c
@@ -463,7 +463,7 @@ static const char hextable[] = {
 };
 
 /* Generic function to parse the current HTTP chunk. It may be used to parsed
- * any kind of chunks, including incomplete HTTP chunks or splitted chunks
+ * any kind of chunks, including incomplete HTTP chunks or split chunks
  * because the buffer wraps. This version tries to performed zero-copy on large
  * chunks if possible.
  */
@@ -547,7 +547,7 @@ static size_t h1_parse_chunk(struct h1m *h1m, struct htx 
**dsthtx,
 
 /* Parses full contiguous HTTP chunks. This version is optimized for small
  * chunks and does not performed zero-copy. It must be called in
- * H1_MSG_CHUNK_SIZE state. Be carefull if you change something in this
+ * H1_MSG_CHUNK_SIZE state. Be careful if you change something in this
  * function. It is really sensitive, any change may have an impact on
  * performance.
  */
@@ -689,7 +689,7 @@ static size_t h1_parse_full_contig_chunks(struct h1m *h1m, 
struct htx **dsthtx,
}
 
/* Now check if the whole chunk is here (including the CRLF at
-* the end), otherise we switch in H1_MSG_DATA stae.
+* the end), otherwise we switch in H1_MSG_DATA state.
 */
if (chksz + 2 > -ridx) {
h1m->curr_len = chksz;
@@ -769,7 +769,7 @@ static size_t h1_parse_msg_chunks(struct h1m *h1m, struct 
htx **dsthtx,
}
 
/* If some data remains, try to parse it using the generic
-* function handling incomplete chunks and splitted chunks
+* function handling incomplete chunks and split chunks
 * because of a wrapping buffer.
 */
if (h1m->state < H1_MSG_TRAILERS && ofs < b_data(srcbuf)) {
-- 
2.29.2.windows.2



Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-12 Thread Илья Шипицин
final apline/musl patch attached

пт, 11 июн. 2021 г. в 23:02, Илья Шипицин :

> it works :)
>
> oops · chipitsine/haproxy@2ce9681 (github.com)
> <https://github.com/chipitsine/haproxy/runs/2805388200>
>
> I'll polish it a bit and will send final patch tomorrow
>
> пт, 11 июн. 2021 г. в 20:42, Илья Шипицин :
>
>>
>>
>> пт, 11 июн. 2021 г. в 20:34, William Lallemand :
>>
>>> On Fri, Jun 11, 2021 at 08:14:49PM +0500, Илья Шипицин wrote:
>>> > I've found ubuntu musl package, so we can just link to it in CI, for
>>> > example (I'll try)
>>> >
>>>
>>>
>>> Well, that won't give you the same environnement as a docker image,
>>> with the same versions. I'll honestly prefer if we could do it with the
>>> alpine image.
>>>
>>
>> there's always gap between "what is tested" and "what is packaged".
>> package maintainers add untested selinux patches and link against
>> various libs.
>>
>> I agree with you that we should prefer to test on alpine natively.
>>
>>
>>>
>>> I you want to build with the musl-gcc wrapper you will need to link the
>>> linux headers in the musl headers directory otherwise it won't work the
>>> way their package is done.
>>>
>>
>>
>>
>>
>>
>>>
>>> --
>>> William Lallemand
>>>
>>
From 5b0db964a9328279d1c301b4c582a20983f7c329 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 12 Jun 2021 15:46:29 +0500
Subject: [PATCH] CI: github actions: enable alpine/musl builds

on push builds are added. based on cirrus-ci patch sent by William Lallemand
---
 .github/workflows/musl.yml | 38 ++
 1 file changed, 38 insertions(+)
 create mode 100644 .github/workflows/musl.yml

diff --git a/.github/workflows/musl.yml b/.github/workflows/musl.yml
new file mode 100644
index 0..9f6b0f52a
--- /dev/null
+++ b/.github/workflows/musl.yml
@@ -0,0 +1,38 @@
+name: alpine/musl
+
+on: [push]
+
+jobs:
+  musl:
+  name: gcc
+  runs-on: ubuntu-latest
+  container:
+image: alpine:latest
+  steps:
+- uses: actions/checkout@master
+- name: Install dependencies
+  run: apk add gcc make tar git python3 libc-dev linux-headers 
pcre-dev pcre2-dev openssl-dev lua5.3-dev grep socat curl
+- name: Install VTest
+  run: scripts/build-vtest.sh
+- name: Build
+  run: make -j$(nproc) CC=cc V=1 TARGET=linux-musl USE_LUA=1 
LUA_INC=/usr/include/lua5.3 LUA_LIB=/usr/lib/lua5.3 USE_OPENSSL=1 USE_PCRE2=1 
USE_PCRE2_JIT=1 USE_PROMEX=1
+- name: Show version
+  run: ./haproxy -vv
+- name: Show linked libraries
+  run: ldd haproxy
+- name: Install problem matcher for VTest
+  # This allows one to more easily see which tests fail.
+  run: echo "::add-matcher::.github/vtest.json"
+- name: Run VTest
+  id: vtest
+  run: make reg-tests VTEST_PROGRAM=../vtest/vtest 
REGTESTS_TYPES=default,bug,devel
+- name: Show results
+  if: ${{ failure() }}
+  run: |
+for folder in /tmp/haregtests-*/vtc.*; do
+  printf "::group::"
+  cat $folder/INFO
+  cat $folder/LOG
+  echo "::endgroup::"
+done
+shopt -s nullglob
-- 
2.29.2.windows.2



Re: [PATCH 0/4] Use 'feature cmd' in regtests

2021-06-11 Thread Илья Шипицин
haproxy/regression-testing.txt at master · haproxy/haproxy (github.com)
<https://github.com/haproxy/haproxy/blob/master/doc/regression-testing.txt>

can it be converted into md/rst ?? please please ?

пт, 11 июн. 2021 г. в 23:19, Илья Шипицин :

> there's reg-test documentation for beginners.
> should it be updated as well ?
>
> пт, 11 июн. 2021 г. в 22:56, Tim Duesterhus :
>
>> Hi!
>>
>> I hope I added all the active developers that touch the reg-tests to the
>> 'CC'
>> list :-)
>>
>> This series updates the regtests to make use of VTest's 'feature cmd'
>> syntax
>> to skip tests that are not supported in the current environment.
>>
>> In the long run this will should result in much cleaner tests, because
>> they
>> don't need to be parsed by run-regtests.sh to extract these marker
>> comments. It
>> also allows to easily add test specific requirements without needing to
>> invent
>> an entirely new syntax. An example might be the recently added tests that
>> are
>> not supported on BoringSSL. These should be able to get a condition like:
>>
>> feature cmd "$HAPROXY_PROGRAM -vv |grep -vq BoringSSL"
>>
>> and then they won't run for BoringSSL (the example is untested).
>>
>> After this series is applied the output of 'make reg-tests' will change.
>> Tests
>> that are excluded using 'feature cmd' will still be listed as "Add test"
>> in
>> the test gathering section. However the final line will show that a few
>> tests
>> have been skipped:
>>
>> 0 tests failed, 4 tests skipped, 105 tests passed
>>
>> I don't think this is going to be an issue. But if it is, please complain!
>>
>> If all of you are happy with the series then it might be merged. Just
>> remember
>> to use 'feature cmd' after it is applied :-)
>>
>> Best regards
>>
>> Tim Düsterhus (4):
>>   REGTESTS: Replace REQUIRE_VERSION=2.5 with 'haproxy -cc'
>>   REGTESTS: Replace REQUIRE_OPTIONS with 'haproxy -cc' for 2.5+ tests
>>   REGTESTS: Replace REQUIRE_BINARIES with 'command -v'
>>   REGTESTS: Remove support for REQUIRE_BINARIES
>>
>>  reg-tests/http-messaging/http_abortonclose.vtc |  2 +-
>>  reg-tests/mcli/mcli_start_progs.vtc|  2 +-
>>  reg-tests/ssl/add_ssl_crt-list.vtc |  2 +-
>>  reg-tests/ssl/new_del_ssl_cafile.vtc   |  6 +++---
>>  reg-tests/ssl/new_del_ssl_crlfile.vtc  |  6 +++---
>>  reg-tests/ssl/set_ssl_cafile.vtc   |  6 +++---
>>  reg-tests/ssl/set_ssl_cert.vtc |  2 +-
>>  reg-tests/ssl/set_ssl_cert_bundle.vtc  |  2 +-
>>  reg-tests/ssl/set_ssl_cert_noext.vtc   |  2 +-
>>  reg-tests/ssl/set_ssl_crlfile.vtc  |  6 +++---
>>  reg-tests/ssl/set_ssl_server_cert.vtc  |  2 +-
>>  reg-tests/ssl/show_ssl_ocspresponse.vtc|  6 +++---
>>  reg-tests/startup/check_condition.vtc  |  1 -
>>  scripts/run-regtests.sh| 12 
>>  14 files changed, 22 insertions(+), 35 deletions(-)
>>
>> --
>> 2.32.0
>>
>>


Re: [PATCH 0/4] Use 'feature cmd' in regtests

2021-06-11 Thread Илья Шипицин
there's reg-test documentation for beginners.
should it be updated as well ?

пт, 11 июн. 2021 г. в 22:56, Tim Duesterhus :

> Hi!
>
> I hope I added all the active developers that touch the reg-tests to the
> 'CC'
> list :-)
>
> This series updates the regtests to make use of VTest's 'feature cmd'
> syntax
> to skip tests that are not supported in the current environment.
>
> In the long run this will should result in much cleaner tests, because they
> don't need to be parsed by run-regtests.sh to extract these marker
> comments. It
> also allows to easily add test specific requirements without needing to
> invent
> an entirely new syntax. An example might be the recently added tests that
> are
> not supported on BoringSSL. These should be able to get a condition like:
>
> feature cmd "$HAPROXY_PROGRAM -vv |grep -vq BoringSSL"
>
> and then they won't run for BoringSSL (the example is untested).
>
> After this series is applied the output of 'make reg-tests' will change.
> Tests
> that are excluded using 'feature cmd' will still be listed as "Add test" in
> the test gathering section. However the final line will show that a few
> tests
> have been skipped:
>
> 0 tests failed, 4 tests skipped, 105 tests passed
>
> I don't think this is going to be an issue. But if it is, please complain!
>
> If all of you are happy with the series then it might be merged. Just
> remember
> to use 'feature cmd' after it is applied :-)
>
> Best regards
>
> Tim Düsterhus (4):
>   REGTESTS: Replace REQUIRE_VERSION=2.5 with 'haproxy -cc'
>   REGTESTS: Replace REQUIRE_OPTIONS with 'haproxy -cc' for 2.5+ tests
>   REGTESTS: Replace REQUIRE_BINARIES with 'command -v'
>   REGTESTS: Remove support for REQUIRE_BINARIES
>
>  reg-tests/http-messaging/http_abortonclose.vtc |  2 +-
>  reg-tests/mcli/mcli_start_progs.vtc|  2 +-
>  reg-tests/ssl/add_ssl_crt-list.vtc |  2 +-
>  reg-tests/ssl/new_del_ssl_cafile.vtc   |  6 +++---
>  reg-tests/ssl/new_del_ssl_crlfile.vtc  |  6 +++---
>  reg-tests/ssl/set_ssl_cafile.vtc   |  6 +++---
>  reg-tests/ssl/set_ssl_cert.vtc |  2 +-
>  reg-tests/ssl/set_ssl_cert_bundle.vtc  |  2 +-
>  reg-tests/ssl/set_ssl_cert_noext.vtc   |  2 +-
>  reg-tests/ssl/set_ssl_crlfile.vtc  |  6 +++---
>  reg-tests/ssl/set_ssl_server_cert.vtc  |  2 +-
>  reg-tests/ssl/show_ssl_ocspresponse.vtc|  6 +++---
>  reg-tests/startup/check_condition.vtc  |  1 -
>  scripts/run-regtests.sh| 12 
>  14 files changed, 22 insertions(+), 35 deletions(-)
>
> --
> 2.32.0
>
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
it works :)

oops · chipitsine/haproxy@2ce9681 (github.com)
<https://github.com/chipitsine/haproxy/runs/2805388200>

I'll polish it a bit and will send final patch tomorrow

пт, 11 июн. 2021 г. в 20:42, Илья Шипицин :

>
>
> пт, 11 июн. 2021 г. в 20:34, William Lallemand :
>
>> On Fri, Jun 11, 2021 at 08:14:49PM +0500, Илья Шипицин wrote:
>> > I've found ubuntu musl package, so we can just link to it in CI, for
>> > example (I'll try)
>> >
>>
>>
>> Well, that won't give you the same environnement as a docker image,
>> with the same versions. I'll honestly prefer if we could do it with the
>> alpine image.
>>
>
> there's always gap between "what is tested" and "what is packaged".
> package maintainers add untested selinux patches and link against
> various libs.
>
> I agree with you that we should prefer to test on alpine natively.
>
>
>>
>> I you want to build with the musl-gcc wrapper you will need to link the
>> linux headers in the musl headers directory otherwise it won't work the
>> way their package is done.
>>
>
>
>
>
>
>>
>> --
>> William Lallemand
>>
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
пт, 11 июн. 2021 г. в 20:34, William Lallemand :

> On Fri, Jun 11, 2021 at 08:14:49PM +0500, Илья Шипицин wrote:
> > I've found ubuntu musl package, so we can just link to it in CI, for
> > example (I'll try)
> >
>
>
> Well, that won't give you the same environnement as a docker image,
> with the same versions. I'll honestly prefer if we could do it with the
> alpine image.
>

there's always gap between "what is tested" and "what is packaged".
package maintainers add untested selinux patches and link against
various libs.

I agree with you that we should prefer to test on alpine natively.


>
> I you want to build with the musl-gcc wrapper you will need to link the
> linux headers in the musl headers directory otherwise it won't work the
> way their package is done.
>





>
> --
> William Lallemand
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
пт, 11 июн. 2021 г. в 20:18, Willy Tarreau :

> On Fri, Jun 11, 2021 at 08:14:49PM +0500,  ??? wrote:
> > @Willy Tarreau  , do you think it is good idea to display libc
> > variant in "haproxy -vv" ?
>
> If needed we can (for those that are detectable), but I'm not convinced
> of the benefits. If it's in order to exclude some tests, I'd rather
> merge this into new config predicates and use Tim's "-cc" instead. This
> will give us way more flexibility over the long term.
>


same as we display openssl variant in "haproxy -vv".
visibility only



>
> Cheers,
> Willy
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
@Willy Tarreau  , do you think it is good idea to display libc
variant in "haproxy -vv" ?
(not sure actually whether musl represent itself in recognizable way)

I've found ubuntu musl package, so we can just link to it in CI, for
example (I'll try)


пт, 11 июн. 2021 г. в 20:03, Илья Шипицин :

>
>
> пт, 11 июн. 2021 г. в 19:43, William Lallemand :
>
>> On Fri, Jun 11, 2021 at 07:09:14PM +0500, Илья Шипицин wrote:
>> > I'm against expanding cirrus matrix. cirrus is overloaded already, I'm
>> > afraid they will not stay for long time.
>> > using custom images in github actions is straightforward, have a look
>> >
>> > centos 6 · chipitsine/haproxy@20fabcd (github.com)
>> > <
>> https://github.com/chipitsine/haproxy/commit/20fabcd005dc9e3bac54a84bf44631f177fa79c2
>> >
>> >
>> >
>> > the same way you can specify either alpine or even special prepared
>> image.
>> > as I recall last time, we decided not to add alpine because "it is
>> tested
>> > anyway when docker images are created"
>> >
>>
>> As far as I know, it is only built, not tested. We encountered a few
>> problems that could have been caught before being built by Docker.
>> And that will prevent us to release a version that doesn't work
>> correctly with docker before they are built on the docker hub.
>>
>>
>> > also, there's small caveat, github actions runs agent inside docker
>> > container, it might have issues with older libc (or musl).
>> > but it worth a try
>> >
>>
>> Let's hope it works in this case.
>>
>
> ok, you made my weekend :)
> I will try tomorrow.
>
> also, I can contact github, no idea why they do not link they agents
> statically.
>
>
>>
>>
>> --
>> William Lallemand
>>
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
пт, 11 июн. 2021 г. в 19:43, William Lallemand :

> On Fri, Jun 11, 2021 at 07:09:14PM +0500, Илья Шипицин wrote:
> > I'm against expanding cirrus matrix. cirrus is overloaded already, I'm
> > afraid they will not stay for long time.
> > using custom images in github actions is straightforward, have a look
> >
> > centos 6 · chipitsine/haproxy@20fabcd (github.com)
> > <
> https://github.com/chipitsine/haproxy/commit/20fabcd005dc9e3bac54a84bf44631f177fa79c2
> >
> >
> >
> > the same way you can specify either alpine or even special prepared
> image.
> > as I recall last time, we decided not to add alpine because "it is tested
> > anyway when docker images are created"
> >
>
> As far as I know, it is only built, not tested. We encountered a few
> problems that could have been caught before being built by Docker.
> And that will prevent us to release a version that doesn't work
> correctly with docker before they are built on the docker hub.
>
>
> > also, there's small caveat, github actions runs agent inside docker
> > container, it might have issues with older libc (or musl).
> > but it worth a try
> >
>
> Let's hope it works in this case.
>

ok, you made my weekend :)
I will try tomorrow.

also, I can contact github, no idea why they do not link they agents
statically.


>
>
> --
> William Lallemand
>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
William, if you do not have a time, I can try to create github action based
on your cirrus patch ... tomorrow ?

пт, 11 июн. 2021 г. в 19:09, Илья Шипицин :

> I'm against expanding cirrus matrix. cirrus is overloaded already, I'm
> afraid they will not stay for long time.
> using custom images in github actions is straightforward, have a look
>
> centos 6 · chipitsine/haproxy@20fabcd (github.com)
> <https://github.com/chipitsine/haproxy/commit/20fabcd005dc9e3bac54a84bf44631f177fa79c2>
>
>
> the same way you can specify either alpine or even special prepared image.
> as I recall last time, we decided not to add alpine because "it is tested
> anyway when docker images are created"
>
> also, there's small caveat, github actions runs agent inside docker
> container, it might have issues with older libc (or musl).
> but it worth a try
>
>
> пт, 11 июн. 2021 г. в 19:02, William Lallemand :
>
>> This commit adds a CI job to cirrus-ci which builds HAProxy on Alpine
>> Linux, allowing to build and test HAProxy with musl.
>>
>> OpenSSL, PCRE2, Lua 5.3 as well as the prometheus exporter are enabled.
>>
>> GNU grep was purposely installed to run the reg-test script.
>> ---
>>  .cirrus.yml | 13 +
>>  1 file changed, 13 insertions(+)
>>
>> diff --git a/.cirrus.yml b/.cirrus.yml
>> index 9b83e6169..392a3abc5 100644
>> --- a/.cirrus.yml
>> +++ b/.cirrus.yml
>> @@ -11,3 +11,16 @@ FreeBSD_task:
>>  - ./haproxy -vv
>>  - ldd haproxy
>>  - env VTEST_PROGRAM=../vtest/vtest gmake reg-tests
>> REGTESTS_TYPES=default,bug,devel || (for folder in /tmp/*regtest*/vtc.*; do
>> cat $folder/INFO $folder/LOG; done && exit 1)
>> +
>> +alpine_task:
>> +  container:
>> +image: alpine:latest
>> +  only_if: $CIRRUS_BRANCH =~ 'master|next'
>> +  script:
>> +- apk add gcc make tar git python3 libc-dev linux-headers pcre-dev
>> pcre2-dev openssl-dev lua5.3-dev grep socat curl
>> +- git clone https://github.com/VTest/VTest.git ../vtest
>> +- make -C ../vtest FLAGS="-O2 -s -Wall"
>> +- make CC=cc V=1 TARGET=linux-musl USE_LUA=1
>> LUA_INC=/usr/include/lua5.3 LUA_LIB=/usr/lib/lua5.3 USE_OPENSSL=1
>> USE_PCRE2=1 USE_PCRE2_JIT=1 USE_PROMEX=1
>> +- ./haproxy -vv
>> +- ldd haproxy
>> +- env VTEST_PROGRAM=../vtest/vtest make reg-tests
>> REGTESTS_TYPES=default,bug,devel || (for folder in /tmp/*regtest*/vtc.*; do
>> cat $folder/INFO $folder/LOG; done && exit 1)
>> --
>> 2.17.1
>>
>>


Re: [PATCH] CI: cirrus: add alpine linux to the jobs

2021-06-11 Thread Илья Шипицин
I'm against expanding cirrus matrix. cirrus is overloaded already, I'm
afraid they will not stay for long time.
using custom images in github actions is straightforward, have a look

centos 6 · chipitsine/haproxy@20fabcd (github.com)



the same way you can specify either alpine or even special prepared image.
as I recall last time, we decided not to add alpine because "it is tested
anyway when docker images are created"

also, there's small caveat, github actions runs agent inside docker
container, it might have issues with older libc (or musl).
but it worth a try


пт, 11 июн. 2021 г. в 19:02, William Lallemand :

> This commit adds a CI job to cirrus-ci which builds HAProxy on Alpine
> Linux, allowing to build and test HAProxy with musl.
>
> OpenSSL, PCRE2, Lua 5.3 as well as the prometheus exporter are enabled.
>
> GNU grep was purposely installed to run the reg-test script.
> ---
>  .cirrus.yml | 13 +
>  1 file changed, 13 insertions(+)
>
> diff --git a/.cirrus.yml b/.cirrus.yml
> index 9b83e6169..392a3abc5 100644
> --- a/.cirrus.yml
> +++ b/.cirrus.yml
> @@ -11,3 +11,16 @@ FreeBSD_task:
>  - ./haproxy -vv
>  - ldd haproxy
>  - env VTEST_PROGRAM=../vtest/vtest gmake reg-tests
> REGTESTS_TYPES=default,bug,devel || (for folder in /tmp/*regtest*/vtc.*; do
> cat $folder/INFO $folder/LOG; done && exit 1)
> +
> +alpine_task:
> +  container:
> +image: alpine:latest
> +  only_if: $CIRRUS_BRANCH =~ 'master|next'
> +  script:
> +- apk add gcc make tar git python3 libc-dev linux-headers pcre-dev
> pcre2-dev openssl-dev lua5.3-dev grep socat curl
> +- git clone https://github.com/VTest/VTest.git ../vtest
> +- make -C ../vtest FLAGS="-O2 -s -Wall"
> +- make CC=cc V=1 TARGET=linux-musl USE_LUA=1
> LUA_INC=/usr/include/lua5.3 LUA_LIB=/usr/lib/lua5.3 USE_OPENSSL=1
> USE_PCRE2=1 USE_PCRE2_JIT=1 USE_PROMEX=1
> +- ./haproxy -vv
> +- ldd haproxy
> +- env VTEST_PROGRAM=../vtest/vtest make reg-tests
> REGTESTS_TYPES=default,bug,devel || (for folder in /tmp/*regtest*/vtc.*; do
> cat $folder/INFO $folder/LOG; done && exit 1)
> --
> 2.17.1
>
>


Re: Speeding up opentracing build in CI ?

2021-06-10 Thread Илья Шипицин
I was mistaken. LibreSSL does not like parallel install

libressl fails on `make -j4 install` · Issue #461 ·
libressl-portable/portable (github.com)



anyway, if CI works, I'm ok with changes

чт, 10 июн. 2021 г. в 20:49, William Lallemand :

> On Thu, Jun 10, 2021 at 07:52:23AM +0200, Willy Tarreau wrote:
> > Subject: Re: Speeding up opentracing build in CI ?
> >
> > On Thu, Jun 10, 2021 at 07:19:37AM +0200, Willy Tarreau wrote:
> > > On Thu, Jun 10, 2021 at 10:15:46AM +0500,  ??? wrote:
> > > > OT takes about 30 sec (it is built with almost everything disabled).
> the
> > > > biggest time eater is openssl-3.0.0
> > >
> > > Maybe that one could be sped up too, I haven't checked if it uses
> parallel
> > > builds.
> >
> > So I checked. Good news, it wasn't parallel either, and this alone:
> >
> > --- a/scripts/build-ssl.sh
> > +++ b/scripts/build-ssl.sh
> > @@ -21,7 +21,8 @@ build_openssl_linux () {
> >  (
> >  cd "openssl-${OPENSSL_VERSION}/"
> >  ./config shared --prefix="${HOME}/opt"
> --openssldir="${HOME}/opt" -DPURIFY
> > -make all install_sw
> > +make -j$(nproc) all
> > +make install_sw
> >  )
> >  }
> >
> > Is enough to drop from 4:52 to 1:28 on my machine. About 1/4 of this time
> > is used to build man and HTML pages that we don't use. Instead of the
> "all"
> > target, we should use "build_sw"
> >
> > --- a/scripts/build-ssl.sh
> > +++ b/scripts/build-ssl.sh
> > @@ -21,7 +21,8 @@ build_openssl_linux () {
> >  (
> >  cd "openssl-${OPENSSL_VERSION}/"
> >  ./config shared --prefix="${HOME}/opt"
> --openssldir="${HOME}/opt" -DPURIFY
> > -make all install_sw
> > +make -j$(nproc) build_sw
> > +make install_sw
> >  )
> >  }
> >
> > this further downs the time to 1:9, hence more than 4 times faster than
> > the initial one. It should probably be tested on macos to be certain it's
> > OK there as well, and I don't know how to get the CPU count there (or
> > maybe we could just force it to a low value like 2 or 4).
> >
> > Willy
> >
>
> Looks fine to me, but from what I remember when debugging some reg-tests
> there was only one CPU available, I hope I'm wrong.
>
> --
> William Lallemand
>


Re: Speeding up opentracing build in CI ?

2021-06-10 Thread Илья Шипицин
openssl does not support -j for make install

I filed a bug on them, they told me "OK, just don't use it"

On Thu, Jun 10, 2021, 8:52 AM Willy Tarreau  wrote:

> On Thu, Jun 10, 2021 at 07:19:37AM +0200, Willy Tarreau wrote:
> > On Thu, Jun 10, 2021 at 10:15:46AM +0500,  ??? wrote:
> > > OT takes about 30 sec (it is built with almost everything disabled).
> the
> > > biggest time eater is openssl-3.0.0
> >
> > Maybe that one could be sped up too, I haven't checked if it uses
> parallel
> > builds.
>
> So I checked. Good news, it wasn't parallel either, and this alone:
>
> --- a/scripts/build-ssl.sh
> +++ b/scripts/build-ssl.sh
> @@ -21,7 +21,8 @@ build_openssl_linux () {
>  (
>  cd "openssl-${OPENSSL_VERSION}/"
>  ./config shared --prefix="${HOME}/opt" --openssldir="${HOME}/opt"
> -DPURIFY
> -make all install_sw
> +make -j$(nproc) all
> +make install_sw
>  )
>  }
>
> Is enough to drop from 4:52 to 1:28 on my machine. About 1/4 of this time
> is used to build man and HTML pages that we don't use. Instead of the "all"
> target, we should use "build_sw"
>
> --- a/scripts/build-ssl.sh
> +++ b/scripts/build-ssl.sh
> @@ -21,7 +21,8 @@ build_openssl_linux () {
>  (
>  cd "openssl-${OPENSSL_VERSION}/"
>  ./config shared --prefix="${HOME}/opt" --openssldir="${HOME}/opt"
> -DPURIFY
> -make all install_sw
> +make -j$(nproc) build_sw
> +make install_sw
>  )
>  }
>
> this further downs the time to 1:9, hence more than 4 times faster than
> the initial one. It should probably be tested on macos to be certain it's
> OK there as well, and I don't know how to get the CPU count there (or
> maybe we could just force it to a low value like 2 or 4).
>
> Willy
>


Re: Speeding up opentracing build in CI ?

2021-06-09 Thread Илья Шипицин
OT takes about 30 sec (it is built with almost everything disabled). the
biggest time eater is openssl-3.0.0

hopefully, OT will speed up to 10 sec by using parallel builds

чт, 10 июн. 2021 г. в 10:05, Willy Tarreau :

> On Thu, Jun 10, 2021 at 07:55:17AM +0300,  ??? wrote:
> > First one is supposed to be cached one day together with "opt" folder.
> >
> > However, we can indeed use parallel builds until cache is enabled for
> > github actions
>
> OK. The other use case I saw was that it's more convenient for developers
> than having to go through each component's readme to figure how to build
> that :-)
>
> I'll make a patch then, thank you!
> Willy
>


Re: Speeding up opentracing build in CI ?

2021-06-09 Thread Илья Шипицин
First one is supposed to be cached one day together with "opt" folder.

However, we can indeed use parallel builds until cache is enabled for
github actions

On Thu, Jun 10, 2021, 7:49 AM Willy Tarreau  wrote:

> Tim, Ilya,
>
> while testing Miroslav's fix, I found the opentracing build to be quite
> slow and figured it doesn't use parallel builds. Do you have any objection
> against patching the script like this ?
>
> diff --git a/scripts/build-ot.sh b/scripts/build-ot.sh
> index 59d6af587..1c296b64b 100755
> --- a/scripts/build-ot.sh
> +++ b/scripts/build-ot.sh
> @@ -19,7 +19,7 @@ if [ "$(cat ${HOME}/opt/.ot-cpp-version)" !=
> "${OT_CPP_VERSION}" ]; then
>  mkdir build
>  cd build
>  cmake -DCMAKE_INSTALL_PREFIX=${HOME}/opt -DBUILD_STATIC_LIBS=OFF
> -DBUILD_MOCKTRACER=OFF -DBUILD_TESTING=OFF >
> -make
> +make -j$(nproc)
>  make install
>  echo "${OT_CPP_VERSION}" > "${HOME}/opt/.ot-cpp-version"
>  fi
> @@ -28,7 +28,7 @@ git clone
> https://github.com/haproxytech/opentracing-c-wrapper.git
>  cd opentracing-c-wrapper
>   ./scripts/bootstrap
>   ./configure --prefix=${HOME}/opt --with-opentracing=${HOME}/opt
> - make
> + make -j$(nproc)
>   make install
>
> I'm assuming it's already made to build only in environments we support
> so we shouldn't care about the presence of the nproc utility there.
>
> Thanks,
> Willy
>


Re: enaling cache in github actions

2021-06-08 Thread Илья Шипицин
Tim, maybe you have an idea how to make it work.
I gave up

(the idea was to cache "opt" and "download-cache" folders)

enabling cache
https://github.com/chipitsine/haproxy/commit/fcb5f130c44f1efb50c2aaf1842a86c7f37f0cca

initially cache
https://github.com/chipitsine/haproxy/runs/2773929840


dumb commit to test cache (all SSL variants should be taken from cache)
https://github.com/chipitsine/haproxy/runs/2774363928

but ...

+ cat /home/runner/opt/.libressl-version
cat: /home/runner/opt/.libressl-version: No such file or directory
+ [  != 2.9.2 ]

вс, 16 мая 2021 г. в 16:59, Tim Düsterhus :

> Ilya,
>
> On 5/15/21 5:30 PM, Илья Шипицин wrote:
> > I've found that we do not cache "download-cache" and "opt" folders,
> > thus we build BoringSSL on every build (no cache).
> >
> > I tried to enable cache
> >
> >
> https://github.com/chipitsine/haproxy/blob/master/.github/workflows/vtest.yml#L46-L53
> >
> > github does not like such caching keys:
> >
> > Error: Key Validation Error: Linux-Ubuntu, gcc, no features cannot
> contain
> > commas.
> >
> >
> > Tim, do you have an idea how to fix this ?
> >
>
> I suggest the following:
>
> - Cache based off the matrix.ssl value, because that determines what
> type of SSL lib we download and build.
> - If we want to cache something else in the future, then we use a
> separate cache head for that specifically.
> - And then use `steps.XXX.outputs.cache-hit != 'true'` to check whether
> you need to build anything or not.
>
> See also: https://github.com/actions/cache#example-workflow
>
> Best regards
> Tim Düsterhus
>


Re: [PATCH] CI: Make matrix.py executable and add shebang

2021-06-08 Thread Илья Шипицин
ack from me.

вт, 8 июн. 2021 г. в 18:17, Tim Duesterhus :

> It's a script, allow executing this as a script without needing to invoke
> `python3` manually.
> ---
>  .github/matrix.py | 2 ++
>  1 file changed, 2 insertions(+)
>  mode change 100644 => 100755 .github/matrix.py
>
> diff --git a/.github/matrix.py b/.github/matrix.py
> old mode 100644
> new mode 100755
> index 473524848..cfef53c9e
> --- a/.github/matrix.py
> +++ b/.github/matrix.py
> @@ -1,3 +1,5 @@
> +#!/usr/bin/python3
> +
>  # Copyright 2019 Ilya Shipitsin 
>  # Copyright 2020 Tim Duesterhus 
>  #
> --
> 2.31.1
>
>
>


Re: [PATCH] CI: enable openssl-3.0.0 builds

2021-06-07 Thread Илья Шипицин
пн, 7 июн. 2021 г. в 18:50, William Lallemand :

> On Mon, Jun 07, 2021 at 02:17:24PM +0200, William Lallemand wrote:
> >
> > On Mon, Jun 07, 2021 at 02:09:33PM +0200, Tim Düsterhus wrote:
> > >
> > > William,
> > >
> > > On 6/7/21 1:30 PM, William Lallemand wrote:
> > > > On Mon, Jun 07, 2021 at 04:02:00PM +0500, Илья Шипицин wrote:
> > > >> sorry, I do not have much spare time to implement that in short time
> > > >> perspective.
> > > >> I think of 2-3 month timeframe.
> > > >>
> > > >
> > > > Isn't it possible to pass a make argument for one specific CI job?
> > > >
> > > > This way we could just do something like:
> > > >
> > > >make DEBUG_CFLAGS="-g -Wno-deprecated-declarations"
> > > >
> > >
> > > Yes, this would be possible. It's already done to enable QUIC for
> BoringSSL:
> > >
> > >
> https://github.com/haproxy/haproxy/blob/a9334df5a9bee2bf17b22f965b08df8d47f2f63e/.github/matrix.py#L116-L117
> > >
> > > It would need an extra case for OpenSSL 3. You can test whether it
> works
> > > locally using:
> > >
> > > $ python3 .github/matrix.py push
> > >
> > > Best regards
> > > Tim Düsterhus
> >
> > Looks easy enough, I'll try that, thanks.
> >
>
> It worked:
>
> https://github.com/haproxy/haproxy/runs/2764512334
>
> Thanks,
>
>



> --
> William Lallemand
>


Re: [PATCH] CI: enable openssl-3.0.0 builds

2021-06-07 Thread Илья Шипицин
пн, 7 июн. 2021 г. в 16:31, William Lallemand :

> On Mon, Jun 07, 2021 at 04:02:00PM +0500, Илья Шипицин wrote:
> > sorry, I do not have much spare time to implement that in short time
> > perspective.
> > I think of 2-3 month timeframe.
> >
>
> Isn't it possible to pass a make argument for one specific CI job?
>
> This way we could just do something like:
>
>   make DEBUG_CFLAGS="-g -Wno-deprecated-declarations"
>

please send a patch


>
> --
> William Lallemand
>


Re: [PATCH] CI: enable openssl-3.0.0 builds

2021-06-07 Thread Илья Шипицин
sorry, I do not have much spare time to implement that in short time
perspective.
I think of 2-3 month timeframe.

пн, 7 июн. 2021 г. в 13:07, William Lallemand :

> On Sun, Jun 06, 2021 at 05:36:45AM +0200, Willy Tarreau wrote:
> > On Sun, Jun 06, 2021 at 12:51:53AM +0200, Tim Düsterhus wrote:
> > > Ilya,
> > >
> > > On 6/5/21 5:10 AM,  ??? wrote:
> > > > here are two patches:
> > > >
> > > > - deprecated warnings suppressed
> > > > - openssl-3.0.0 enabled
> > > >
> > >
> > > In the second patch you forgot the 'CI:' prefix in the commit message.
> > >
> > > Other than that the patches LGTM with regard to the CI changes.
> >
> > OK thanks Tim, I'll take it and adjust the subject.
> >
> > Willy
> >
>
> Hello,
>
> I don't think that's a good idea to do it this way, the first intent was
> to be able to build OpenSSL 3.0.0, but the -Wdeprecated-declarations was
> preventing to build with -Werror.
>
> With this patch we are hiding all these warnings for all users and they
> can be relevant at some point, not only for OpenSSL, but for the other
> libs that are linked with haproxy.
>
> In my opinion we should only disable them for this specific build of
> OpenSSL 3.0.0 on the CI, not for everyone in the Makefile.
>
> --
> William Lallemand
>


Re: Official ubuntu 20 repository

2021-06-07 Thread Илья Шипицин
пн, 7 июн. 2021 г. в 12:20, Valters Jansons :

> On Mon, Jun 7, 2021 at 12:34 AM Ismail Azerty 
> wrote:
> >  For some security reasons, our security teams want us to use the
> official repository, or recompile the whole project on ubuntu 20.
>
> Official Ubuntu repositories are "slow" to update due to LTS policies,
> ensuring no potentially breaking changes. Focal (20.04) is on 2.0
> series, and will not be getting an update to 2.2.
>

term "official" maybe treated as "ubuntu official" or "haproxy official".
while "ubuntu official" are indeed slow, vbernat PPA is considered as
"haproxy official".


>
> If you want the latest version, then that goes against the official
> LTS policy, and therefore you need to either use someone else's build
> or build locally.
>
> >  Do you have any ansible playbook, or shell script, that we can use ?
>
> The PPA in question can be seen on
>
> https://launchpad.net/~vbernat/+archive/ubuntu/haproxy-2.4/+packages?field.series_filter=focal
> and in package details you can see the .debian.tar.xz file. This
> contains relevant modifications so that standard Debian/Ubuntu build
> process is successful - with dpkg tools, and debuild, and the likes.
> If you have internal build processes in place for Ubuntu packages,
> this should be simple to integrate.
>
> Replace 2.4 in the link above with whichever series you are interested
> in. If you are rebuilding by hand and/or want to have manual review
> processes in place, you might want to opt for an older series - say,
> 2.2 - which will have less changes over time.
>
> There are considerations for proper internal distribution, such as
> needing your own signing keys internally. However, further explanation
> of the Debian/Ubuntu build processes falls outside of the scope of the
> mailing list -- there are plenty of resources online for those
> particular tasks.
>
>


Re: [PATCH] CI: enable openssl-3.0.0 builds

2021-06-04 Thread Илья Шипицин
here are two patches:

- deprecated warnings suppressed
- openssl-3.0.0 enabled

ср, 2 июн. 2021 г. в 21:51, Tim Düsterhus :

> Dinko,
>
> On 6/2/21 1:34 PM, Dinko Korunic wrote:
> > Yes, GItHub Actions has support for flagging some of the jobs in matrix
> as experimental (which will permit job to fail and other jobs in the matrix
> will continue to execute), for instance:
> >
> >
> https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-including-new-combinations
> <
> https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-including-new-combinations
> >
> >
> https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-preventing-a-specific-failing-matrix-job-from-failing-a-workflow-run
> <
> https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#example-preventing-a-specific-failing-matrix-job-from-failing-a-workflow-run
> >
> >
>
> Oh, continue-on-error also exists on the job level. IMO it's badly
> named, but it appears to be what I was looking for.
>
> Based off the other responses we might not need it after all, but it's
> good to know that it is possible. Thanks!
>
> Best regards
> Tim Düsterhus
>
>
From 4f6467bfe678479cfeb69bac1ec7ffd01177988b Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 5 Jun 2021 03:06:59 +
Subject: [PATCH 2/2] github actions: add OpenSSL-3.0.0 builds

OpenSSL-3.0.0 is getting close to its release, let us add it to build matrix
---
 .github/matrix.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.github/matrix.py b/.github/matrix.py
index 9e9b395..1ff48aa 100644
--- a/.github/matrix.py
+++ b/.github/matrix.py
@@ -108,6 +108,7 @@ for CC in ["gcc", "clang"]:
 for ssl in [
 "stock",
 "OPENSSL_VERSION=1.0.2u",
+"OPENSSL_VERSION=3.0.0-alpha17",
 "LIBRESSL_VERSION=2.9.2",
 "LIBRESSL_VERSION=3.3.3",
 "BORINGSSL=yes",
-- 
1.8.3.1

From 83235f06da6b731f01216fa3ee4e0221496844f6 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 5 Jun 2021 03:06:03 +
Subject: [PATCH 1/2] BUILD: suppress deprecated warnings

this is temporary workaround to enable OpenSSL-3.0.0 builds
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index 6571dc6..8fa3b7c 100644
--- a/Makefile
+++ b/Makefile
@@ -203,6 +203,7 @@ SPEC_CFLAGS += $(call cc-nowarn,clobbered)
 SPEC_CFLAGS += $(call cc-nowarn,missing-field-initializers)
 SPEC_CFLAGS += $(call cc-nowarn,cast-function-type)
 SPEC_CFLAGS += $(call cc-nowarn,string-plus-int)
+SPEC_CFLAGS += $(call cc-nowarn,deprecated-declarations)
 SPEC_CFLAGS += $(call cc-opt,-Wtype-limits)
 SPEC_CFLAGS += $(call cc-opt,-Wshift-negative-value)
 SPEC_CFLAGS += $(call cc-opt,-Wshift-overflow=2)
-- 
1.8.3.1



Re: Modify POST data (with lua)?

2021-06-02 Thread Илья Шипицин
ср, 2 июн. 2021 г. в 18:00, Elias Abacioglu <
elias.abacio...@deltaprojects.com>:

>
> Hi Tim
>
> On Wed, Jun 2, 2021 at 12:38 PM Tim Düsterhus  wrote:
>
>> Elias,
>>
>> On 6/2/21 12:16 PM, Elias Abacioglu wrote:
>> > I'm planning on placing an API behind HAProxy and want a way to modify
>> the
>> > times/intervals in the POST data to increase cacheability to the API.
>>
>> While POST requests may in fact be cached under certain circumstances as
>> I learned 2 minutes ago, it is certainly very unusual and also not
>> supported by HAProxy's cache.
>>
>
> Oh, that was bad news that haproxy cannot cache POST requests.
> I guess I can use nuster (https://github.com/jiangwenyuan/nuster), a
> HAProxy fork that adds more caching support (seems to be able to cache POST
> requests).
> Otherwise I would have to use Nginx or OpenResty, however I do prefer
> HAProxy.
>
> I would advise against caching POST requests and instead move the
>> parameters into the URL or a request header (make sure to set 'Vary' on
>> the response) and instead use a GET request.
>>
>
> It makes perfect sense to cache POSTS in certain scenarios, for example,
> Grafana speaks with POST requests to OpenTSDB. Also Druid uses POST
> requests for querying, and a bunch of other time series databases. I can't
> really control that some third party tools have opted for POST instead of
> URL parameters or headers.
> Sending a POST request that is repeated, for instance a user pressing F5
> in their browser a bunch of times, can cause an underlying API to do some
> heavy processing repeatedly. Some of these applications might have some
> query cache, but not always optimal. I would rather just avoid talking to
> the API in a situation like this and just presenting a response from a
> cache.
>

application usually knows the best whether its requests might be cached or
not.
for example, html5 manifest can prevent from F5 storms.

it is always better to shift caching activity to app level


>
> However I still want to know if it's possible to transform a POST's json
> data as I mentioned in my original post (even if I cannot cache the
> response).
>
> Thanks
> Elias
>


Re: [PATCH] CI: enable openssl-3.0.0 builds

2021-06-02 Thread Илья Шипицин
ср, 2 июн. 2021 г. в 16:27, Tim Düsterhus :

> Ilya,
>
> On 6/2/21 12:58 PM, Илья Шипицин wrote:
> > as openssl-3.0.0 is getting close to release, let us add it to build
> matrix.
> >
>
> I dislike that this is going to report all commits in red, until the
> build failures with OpenSSL 3 are fixed. I did a quick research, whether
> some job could be marked as "experimental" / "allowed failure" with
> GitHub Actions, but could not find anything.
>

@William Lallemand   has an appetite to make it
green ;)


>
> Do you know whether this is possible?
>


That's a good question. I was sure it would be possible, I'll have a look.


>
> The patch LGTM other than that concern.
>
> Best regards
> Tim Düsterhus
>


[PATCH] CI: enable openssl-3.0.0 builds

2021-06-02 Thread Илья Шипицин
Hello,

as openssl-3.0.0 is getting close to release, let us add it to build matrix.

thanks,
Ilya
From 1651766c81e141c1a077499e151e6c619289883a Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 2 Jun 2021 10:42:09 +
Subject: [PATCH] CI: github actions: add OpenSSL-3.0.0 builds

OpenSSL-3.0.0 is getting close to its release, let us add it to build matrix
---
 .github/matrix.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/.github/matrix.py b/.github/matrix.py
index 9e9b395..1ff48aa 100644
--- a/.github/matrix.py
+++ b/.github/matrix.py
@@ -108,6 +108,7 @@ for CC in ["gcc", "clang"]:
 for ssl in [
 "stock",
 "OPENSSL_VERSION=1.0.2u",
+"OPENSSL_VERSION=3.0.0-alpha17",
 "LIBRESSL_VERSION=2.9.2",
 "LIBRESSL_VERSION=3.3.3",
 "BORINGSSL=yes",
-- 
1.8.3.1



Re: how to write to a file safely in haproxy

2021-05-26 Thread Илья Шипицин
Not clear what recommendation did you read.
you might consider dataplane api for config management:
haproxytech/dataplaneapi:
HAProxy Data Plane API (github.com)


On Wed, May 26, 2021, 4:32 PM reshma r  wrote:

> Hello all,
> Periodically I need to write some configuration data to a file. However I
> came across documentation that warned against writing to a file at runtime.
> Can someone give me advice on how I can achieve this safely?
> Appreciate any help
>
> Thank you
>


Re: [PATCH] CI: enable OpenTracing feature

2021-05-18 Thread Илья Шипицин
Miroslav,


can you please keep an eye on patch from OpenTracing build failures using
clang · Issue #1242 · haproxy/haproxy (github.com)
 ?

вт, 18 мая 2021 г. в 21:38, Willy Tarreau :

> On Tue, May 18, 2021 at 01:32:56PM +0200, Tim Düsterhus wrote:
> > Willy,
> >
> > On 5/18/21 12:42 PM,  ??? wrote:
> > > this enables OpenTracing for CI builds.
> >
> > This one looks good to me.
>
> Applied, thanks guys.
>
> Willy
>


[PATCH] CI: enable OpenTracing feature

2021-05-18 Thread Илья Шипицин
Hello,

this enables OpenTracing for CI builds.


thanks.
Ilya
From 9ca15c0d95b1c4fa5dc040116f9ec006712095f1 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 18 May 2021 09:46:43 +
Subject: [PATCH] CI: github actions: add OpenTracing builds

let us add OpenTracing module to "all features" builds
---
 .github/matrix.py   |  8 
 .github/workflows/vtest.yml |  3 +++
 scripts/build-ot.sh | 34 ++
 3 files changed, 45 insertions(+)
 create mode 100755 scripts/build-ot.sh

diff --git a/.github/matrix.py b/.github/matrix.py
index 067ce93..9e9b395 100644
--- a/.github/matrix.py
+++ b/.github/matrix.py
@@ -71,6 +71,10 @@ for CC in ["gcc", "clang"]:
 "CC": CC,
 "FLAGS": [
 "USE_ZLIB=1",
+"USE_OT=1",
+"OT_INC=${HOME}/opt/include",
+"OT_LIB=${HOME}/opt/lib",
+"OT_RUNPATH=1",
 "USE_PCRE=1",
 "USE_PCRE_JIT=1",
 "USE_LUA=1",
@@ -139,6 +143,10 @@ matrix.append(
 "FLAGS": get_asan_flags(CC)
 + [
 "USE_ZLIB=1",
+"USE_OT=1",
+"OT_INC=${HOME}/opt/include",
+"OT_LIB=${HOME}/opt/lib",
+"OT_RUNPATH=1",
 "USE_PCRE=1",
 "USE_PCRE_JIT=1",
 "USE_LUA=1",
diff --git a/.github/workflows/vtest.yml b/.github/workflows/vtest.yml
index f7f34d7..7867133 100644
--- a/.github/workflows/vtest.yml
+++ b/.github/workflows/vtest.yml
@@ -64,6 +64,9 @@ jobs:
 - name: Install SSL ${{ matrix.ssl }}
   if: ${{ matrix.ssl && matrix.ssl != 'stock' }}
   run: env ${{ matrix.ssl }} scripts/build-ssl.sh
+- name: Install OpenTracing libs
+  if: ${{ contains(matrix.FLAGS, 'USE_OT=1') }}
+  run: scripts/build-ot.sh
 - name: Build WURFL
   if: ${{ contains(matrix.FLAGS, 'USE_WURFL=1') }}
   run: make -C addons/wurfl/dummy
diff --git a/scripts/build-ot.sh b/scripts/build-ot.sh
new file mode 100755
index 000..59d6af5
--- /dev/null
+++ b/scripts/build-ot.sh
@@ -0,0 +1,34 @@
+#!/bin/bash
+
+#
+# OT helper. script built from documentation: 
https://github.com/haproxytech/opentracing-c-wrapper
+#
+
+set -e
+
+export OT_CPP_VERSION=1.5.0
+
+if [ ! -f "download-cache/v${OT_CPP_VERSION}.tar.gz" ]; then
+wget -P download-cache/ \
+
"https://github.com/opentracing/opentracing-cpp/archive/v${OT_CPP_VERSION}.tar.gz;
+fi
+
+if [ "$(cat ${HOME}/opt/.ot-cpp-version)" != "${OT_CPP_VERSION}" ]; then
+tar xf download-cache/v${OT_CPP_VERSION}.tar.gz
+cd opentracing-cpp-${OT_CPP_VERSION}
+mkdir build
+cd build
+cmake -DCMAKE_INSTALL_PREFIX=${HOME}/opt -DBUILD_STATIC_LIBS=OFF 
-DBUILD_MOCKTRACER=OFF -DBUILD_TESTING=OFF ..
+make
+make install
+echo "${OT_CPP_VERSION}" > "${HOME}/opt/.ot-cpp-version"
+fi
+
+git clone https://github.com/haproxytech/opentracing-c-wrapper.git
+cd opentracing-c-wrapper
+ ./scripts/bootstrap
+ ./configure --prefix=${HOME}/opt --with-opentracing=${HOME}/opt
+ make
+ make install
+
+
-- 
1.8.3.1



Re: [PATCH] move VTest installation to scripts/build-vtest.sh

2021-05-18 Thread Илья Шипицин
вт, 18 мая 2021 г. в 13:54, Willy Tarreau :

> On Sat, May 15, 2021 at 05:53:57PM +0200, Tim Düsterhus wrote:
> > Willy,
> > Ilya,
> >
> > On 5/15/21 8:58 AM,  ??? wrote:
> > > I attached a patch that uses "curl". on a distance it seems to be
> faster
> > > for 50%
> > >
> >
> > This one looks good to me.
>
> Now merged, thanks guys.
>
> A quick comment about this, since we're trying to be fast, the following:
>
>   curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz -o
> VTest.tar.gz
>   mkdir ../vtest
>   tar xvf VTest.tar.gz -C ../vtest --strip-components=1
>
> Can be more optimally done this way by paralleling the download and the
> extraction:
>
>   mkdir ../vtest
>   curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz | \
>  tar -C ../vtest -zxf - --strip-components=1
>
> However due to vtest being very small (180 kB) it will not make any
> difference on this one.
>

I wonder if we can benefit from dependent steps, i.e. build VTest just once
and deliver binary to other containers (instead of building it every time).


>
> Cheers,
> Willy
>


Re: unexpected asan behaviour

2021-05-18 Thread Илья Шипицин
Maybe sudo drops env variables

On Tue, May 18, 2021, 12:29 PM Tim Düsterhus  wrote:

> Ilya,
>
> On 5/17/21 3:57 PM, Илья Шипицин wrote:
> > FYI, this is not what it supposed to be
> >
> > BUILD/MINOR: ssl: Fix compilation with SSL enabled ·
> > haproxy/haproxy@d75b99e (github.com)
> > <https://github.com/haproxy/haproxy/runs/2599654016>
> >
> > according to global ASAN_OPTIONS it was supposed to go to separate log.
> > but something went wrong :)
>
> I'm aware of that, I believe we discussed this back when migrating to
> GitHub Actions. But we considered the current state acceptable, as ASAN
> generally works and one can find the output, even if it is a bit hidden.
> I was unable to find out why it didn't work back then.
>
> > it is almost all time ok to break on ASAN errors, the only degradation it
> > fails on the first error and execution stops (that is ok almost all
> time).
> >
> > I'll try to figure out why ASAN_OPTIONS did not work.
> >
>
> Thanks!
>
> Best regards
> Tim Düsterhus
>


unexpected asan behaviour

2021-05-17 Thread Илья Шипицин
Tim,

FYI, this is not what it supposed to be

BUILD/MINOR: ssl: Fix compilation with SSL enabled ·
haproxy/haproxy@d75b99e (github.com)


according to global ASAN_OPTIONS it was supposed to go to separate log.
but something went wrong :)

it is almost all time ok to break on ASAN errors, the only degradation it
fails on the first error and execution stops (that is ok almost all time).

I'll try to figure out why ASAN_OPTIONS did not work.


Ilya


enaling cache in github actions

2021-05-15 Thread Илья Шипицин
I've found that we do not cache "download-cache" and "opt" folders,
thus we build BoringSSL on every build (no cache).

I tried to enable cache

https://github.com/chipitsine/haproxy/blob/master/.github/workflows/vtest.yml#L46-L53

github does not like such caching keys:

Error: Key Validation Error: Linux-Ubuntu, gcc, no features cannot contain
commas.


Tim, do you have an idea how to fix this ?

thanks,
Ilya


Re: DNS service discovery and consistent hashing

2021-05-15 Thread Илья Шипицин
maybe using dataplane api https://github.com/haproxytech/dataplaneapi
will give you required flexibility


пт, 14 мая 2021 г. в 01:58, Andrew Rodland :

> At Vimeo we have a custom tool since 2015 that monitors the membership of
> clusters of servers, templates out a config with servers assigned to
> backends, and manages reloading haproxy. We're looking into replacing this
> with something a bit more off-the-shelf, and one of the options is
> HAProxy's own DNS service discovery support.
>
> We're also using URI-based load balancing with consistent hashing, and the
> stability of that mapping is important to us. Temporary disagreements while
> membership is changing are inevitable, but we want the portion of the hash
> space that a backend server sees to change as little as possible during its
> lifetime, and for multiple haproxies running the same config, against the
> same cluster, to converge on the same mapping. Our existing tool assigns a
> persistent ID to each server, which is mapped to an "id" option in the
> server line, which has worked quite well.
>
> From what we've seen in testing so far, using "server-template" with DNS
> *doesn't* give us the behavior we want — the assignment of servers to slots
> seems inconsistent, maybe depending on some combination of the order of
> answers in the DNS packet or the order that new server appearances are
> observed by haproxy.
>
> Long story short:
>
> 1. Is my interpretation right?
>
> 2. Would you be open to a patch to change that? I'm thinking of something
> like setting puid from a hash of the SRV name or the A address, "open
> addressing" style, with who goes first in case of a collision determined by
> lexicographic order — but I'm quite open to guidance.
>
> Or should I just look somewhere other than the DNS service discovery?
>
> Thanks,
>
> Andrew Rodland
>
> (Please CC, I'm not on the list.)
>
>
>


Re: [PATCH] move VTest installation to scripts/build-vtest.sh

2021-05-15 Thread Илья Шипицин
сб, 15 мая 2021 г. в 01:14, Илья Шипицин :

>
>
> сб, 15 мая 2021 г. в 00:56, Nicolas CARPi :
>
>> > I compared several times, git clone is faster
>> In my hands curl is the fastest option compared to git clone.
>>
>
> ok, I'll make more comparison tomorrow
>


I attached a patch that uses "curl". on a distance it seems to be faster
for 50%


>
>
>>
>> Wouldn't "git clone --depth 1" be a good option to consider here? It
>> avoids getting the full history.
>>
>> ~Nico
>>
>
From 7d933d88dc03782bab2985081bb9629d3a3389fd Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 15 May 2021 11:46:15 +0500
Subject: [PATCH] CI: introduce scripts/build-vtest.sh for installing VTest

we install VTest for all CI systems, let us unify instalation
---
 .cirrus.yml|  3 +--
 .github/workflows/openssl-nodeprecated.yml |  3 +--
 .github/workflows/vtest.yml|  8 ++--
 .travis.yml|  4 +---
 scripts/build-vtest.sh | 10 ++
 5 files changed, 15 insertions(+), 13 deletions(-)
 create mode 100755 scripts/build-vtest.sh

diff --git a/.cirrus.yml b/.cirrus.yml
index fdabfdcdd..9b83e6169 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -6,8 +6,7 @@ FreeBSD_task:
   install_script:
 - pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat pcre
   script:
-- git clone https://github.com/VTest/VTest.git ../vtest
-- make -C ../vtest
+- scripts/build-vtest.sh
 - gmake CC=clang V=1 ERR=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
 - ./haproxy -vv
 - ldd haproxy
diff --git a/.github/workflows/openssl-nodeprecated.yml b/.github/workflows/openssl-nodeprecated.yml
index b853fe233..6833911e4 100644
--- a/.github/workflows/openssl-nodeprecated.yml
+++ b/.github/workflows/openssl-nodeprecated.yml
@@ -23,8 +23,7 @@ jobs:
 - uses: actions/checkout@v1
 - name: prepare VTest
   run: |
-git clone https://github.com/VTest/VTest.git ../vtest
-make -C ../vtest FLAGS="-O2 -s -Wall"
+scripts/build-vtest.sh
 - name: build haproxy
   run: |
 make DEFINE="-DOPENSSL_API_COMPAT=0x1010L -DOPENSSL_NO_DEPRECATED" -j3 CC=gcc ERR=1 TARGET=linux-glibc USE_OPENSSL=1
diff --git a/.github/workflows/vtest.yml b/.github/workflows/vtest.yml
index cb52f27d6..f7f34d720 100644
--- a/.github/workflows/vtest.yml
+++ b/.github/workflows/vtest.yml
@@ -60,11 +60,7 @@ jobs:
 brew install lua
 - name: Install VTest
   run: |
-curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz -o VTest.tar.gz
-mkdir VTest
-tar xvf VTest.tar.gz -C VTest --strip-components=1
-make -C VTest -j$(nproc) FLAGS="-O2 -s -Wall"
-sudo install -m755 VTest/vtest /usr/local/bin/vtest
+scripts/build-vtest.sh
 - name: Install SSL ${{ matrix.ssl }}
   if: ${{ matrix.ssl && matrix.ssl != 'stock' }}
   run: env ${{ matrix.ssl }} scripts/build-ssl.sh
@@ -101,7 +97,7 @@ jobs:
 - name: Run VTest for HAProxy ${{ steps.show-version.outputs.version }}
   id: vtest
   # sudo is required, because macOS fails due to an open files limit.
-  run: sudo make reg-tests REGTESTS_TYPES=default,bug,devel
+  run: sudo make reg-tests VTEST_PROGRAM=../vtest/vtest REGTESTS_TYPES=default,bug,devel
 - name: Show results
   if: ${{ failure() }}
   # The chmod / sudo is necessary due to the `sudo` while running the tests.
diff --git a/.travis.yml b/.travis.yml
index 1aa415aa8..3083e302c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -38,9 +38,7 @@ matrix:
 if: type == cron
 
 install:
-  - git clone https://github.com/VTest/VTest.git ../vtest
-  # Special flags due to: https://github.com/vtest/VTest/issues/12
-  - make -C ../vtest FLAGS="-O2 -s -Wall"
+  - scripts/build-vtest.sh
 
 script:
   - make -j$(nproc) ERR=1 TARGET=linux-glibc CC=$CC DEBUG=-DDEBUG_STRICT=1 $FLAGS
diff --git a/scripts/build-vtest.sh b/scripts/build-vtest.sh
new file mode 100755
index 0..4db35d6ee
--- /dev/null
+++ b/scripts/build-vtest.sh
@@ -0,0 +1,10 @@
+#!/bin/sh
+
+set -eux
+
+curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz -o VTest.tar.gz
+mkdir ../vtest
+tar xvf VTest.tar.gz -C ../vtest --strip-components=1
+# Special flags due to: https://github.com/vtest/VTest/issues/12
+make -C ../vtest FLAGS="-O2 -s -Wall"
+
-- 
2.31.1



Re: [PATCH] move VTest installation to scripts/build-vtest.sh

2021-05-14 Thread Илья Шипицин
сб, 15 мая 2021 г. в 00:56, Nicolas CARPi :

> > I compared several times, git clone is faster
> In my hands curl is the fastest option compared to git clone.
>

ok, I'll make more comparison tomorrow


>
> Wouldn't "git clone --depth 1" be a good option to consider here? It
> avoids getting the full history.
>
> ~Nico
>


Re: [PATCH] move VTest installation to scripts/build-vtest.sh

2021-05-14 Thread Илья Шипицин
сб, 15 мая 2021 г. в 00:20, Tim Düsterhus :

> Ilya,
>
> On 5/14/21 9:02 PM, Илья Шипицин wrote:
> > let us unify VTest installation magic.
>
> I disagree with using 'git clone' here, cloning the repository with full
> history is wasteful. Please use the tarball approach from vtest.yml
>

I think, if we download from github releases, indeed those are
pre-packaged. Master seems to be created on the fly.
I compared several times, git clone is faster

[ilia@localhost temp]$ time git clone https://github.com/VTest/VTest.git
Cloning into 'VTest'...
remote: Enumerating objects: 1072, done.
remote: Counting objects: 100% (320/320), done.
remote: Compressing objects: 100% (171/171), done.
remote: Total 1072 (delta 213), reused 241 (delta 149), pack-reused 752
Receiving objects: 100% (1072/1072), 588.23 KiB | 223.00 KiB/s, done.
Resolving deltas: 100% (740/740), done.

real 0m6.045s
user 0m0.287s
sys 0m0.087s
[ilia@localhost temp]$ time curl -fsSL
https://github.com/vtest/VTest/archive/master.tar.gz -o VTest.tar.gz

real 0m8.185s
user 0m0.071s
sys 0m0.020s
[ilia@localhost temp]$

I can swich to "curl" ifthere's more justification of that



>
> Best regards
> Tim Düsterhus
>


[PATCH] move VTest installation to scripts/build-vtest.sh

2021-05-14 Thread Илья Шипицин
Hello,

let us unify VTest installation magic.

Ilya
From 9f85fe25ad9ee6bf753311a44a5089a6ee2096c9 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Fri, 14 May 2021 23:59:35 +0500
Subject: [PATCH] CI: introduce scripts/build-vtest.sh for installing VTest

we install VTest for all CI systems, let us unify instalation
---
 .cirrus.yml| 3 +--
 .github/workflows/openssl-nodeprecated.yml | 3 +--
 .github/workflows/vtest.yml| 8 ++--
 .travis.yml| 4 +---
 scripts/build-vtest.sh | 8 
 5 files changed, 13 insertions(+), 13 deletions(-)
 create mode 100755 scripts/build-vtest.sh

diff --git a/.cirrus.yml b/.cirrus.yml
index fdabfdcdd..9b83e6169 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -6,8 +6,7 @@ FreeBSD_task:
   install_script:
 - pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat pcre
   script:
-- git clone https://github.com/VTest/VTest.git ../vtest
-- make -C ../vtest
+- scripts/build-vtest.sh
 - gmake CC=clang V=1 ERR=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
 - ./haproxy -vv
 - ldd haproxy
diff --git a/.github/workflows/openssl-nodeprecated.yml b/.github/workflows/openssl-nodeprecated.yml
index b853fe233..6833911e4 100644
--- a/.github/workflows/openssl-nodeprecated.yml
+++ b/.github/workflows/openssl-nodeprecated.yml
@@ -23,8 +23,7 @@ jobs:
 - uses: actions/checkout@v1
 - name: prepare VTest
   run: |
-git clone https://github.com/VTest/VTest.git ../vtest
-make -C ../vtest FLAGS="-O2 -s -Wall"
+scripts/build-vtest.sh
 - name: build haproxy
   run: |
 make DEFINE="-DOPENSSL_API_COMPAT=0x1010L -DOPENSSL_NO_DEPRECATED" -j3 CC=gcc ERR=1 TARGET=linux-glibc USE_OPENSSL=1
diff --git a/.github/workflows/vtest.yml b/.github/workflows/vtest.yml
index cb52f27d6..f7f34d720 100644
--- a/.github/workflows/vtest.yml
+++ b/.github/workflows/vtest.yml
@@ -60,11 +60,7 @@ jobs:
 brew install lua
 - name: Install VTest
   run: |
-curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz -o VTest.tar.gz
-mkdir VTest
-tar xvf VTest.tar.gz -C VTest --strip-components=1
-make -C VTest -j$(nproc) FLAGS="-O2 -s -Wall"
-sudo install -m755 VTest/vtest /usr/local/bin/vtest
+scripts/build-vtest.sh
 - name: Install SSL ${{ matrix.ssl }}
   if: ${{ matrix.ssl && matrix.ssl != 'stock' }}
   run: env ${{ matrix.ssl }} scripts/build-ssl.sh
@@ -101,7 +97,7 @@ jobs:
 - name: Run VTest for HAProxy ${{ steps.show-version.outputs.version }}
   id: vtest
   # sudo is required, because macOS fails due to an open files limit.
-  run: sudo make reg-tests REGTESTS_TYPES=default,bug,devel
+  run: sudo make reg-tests VTEST_PROGRAM=../vtest/vtest REGTESTS_TYPES=default,bug,devel
 - name: Show results
   if: ${{ failure() }}
   # The chmod / sudo is necessary due to the `sudo` while running the tests.
diff --git a/.travis.yml b/.travis.yml
index 1aa415aa8..3083e302c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -38,9 +38,7 @@ matrix:
 if: type == cron
 
 install:
-  - git clone https://github.com/VTest/VTest.git ../vtest
-  # Special flags due to: https://github.com/vtest/VTest/issues/12
-  - make -C ../vtest FLAGS="-O2 -s -Wall"
+  - scripts/build-vtest.sh
 
 script:
   - make -j$(nproc) ERR=1 TARGET=linux-glibc CC=$CC DEBUG=-DDEBUG_STRICT=1 $FLAGS
diff --git a/scripts/build-vtest.sh b/scripts/build-vtest.sh
new file mode 100755
index 0..6a974951e
--- /dev/null
+++ b/scripts/build-vtest.sh
@@ -0,0 +1,8 @@
+#!/bin/sh
+
+set -eux
+
+git clone https://github.com/VTest/VTest.git ../vtest
+# Special flags due to: https://github.com/vtest/VTest/issues/12
+make -C ../vtest FLAGS="-O2 -s -Wall"
+
-- 
2.31.1



Re: adding Coverity badge to some visible place

2021-05-14 Thread Илья Шипицин
some tools (e.g. Coverity, Travis-CI) are not attached to commits directly.
travis runs once a week, badge would be nice as well

пт, 14 мая 2021 г. в 20:45, Daniel Corbett :

> Hello Ilya,
>
> > From: Илья Шипицин 
> > Sent: Friday, May 14, 2021 11:10 AM
> > To: HAProxy 
> > Subject: adding Coverity badge to some visible place
>
> > Hello,
> >
> > I would like to improve code quality visibility.
> > what if we convert README --> README.md and put Coverity badge there ?
> >
>
> Funny, I had a similar thought earlier today about having a README.md.
>
> I would like to help improve it from a visual perspective.
>
> +1 from me and very interested!
>
> Thanks,
> -- Daniel
>
>


adding Coverity badge to some visible place

2021-05-14 Thread Илья Шипицин
Hello,

I would like to improve code quality visibility.
what if we convert README --> README.md and put Coverity badge there ?

Ilya


Re: [PATCH] BUILD/MINOR: opentracing: fixed compilation with filter, enabled

2021-05-12 Thread Илья Шипицин
I did not mean to postpone Miroslav's patch :)

On Wed, May 12, 2021, 10:39 AM Willy Tarreau  wrote:

> Hi Ilya,
>
> On Tue, May 11, 2021 at 10:59:46PM +0500,  ??? wrote:
> > I tried opentracing ci, I estimate 1-2 weeks to stabilize. Can we delay
> 2.4
> > after that?
>
> If you mean delaying 2.4 after some extra CI is ready, not at all. What
> would delay it is a big last minute regression, but postponing a release
> just to wait for something never works as stuff continues to accumulate.
> Believe me, that's what turned the planed 6-months cycle of 1.5 into 4.5
> years. In addition, the feedback from users just after a release is always
> faster and more valuable than anything we can try to imagine at the last
> minute.
>
> Do not worry, we do have time to continue to improve the CI (which is
> already great), there's no emergency and whatever you provide can be
> merged at any moment since it focuses on the development tree, so do
> not feel pressured by an upcoming version nor any timing in general.
>
> Cheers,
> Willy
>


Re: [PATCH] BUILD/MINOR: opentracing: fixed compilation with filter, enabled

2021-05-11 Thread Илья Шипицин
I tried opentracing ci, I estimate 1-2 weeks to stabilize. Can we delay 2.4
after that?

On Tue, May 11, 2021, 10:37 PM Miroslav Zagorac 
wrote:

> Hello,
>
> The inclusion of header files proxy.h and tools.h was added to the
> addons/ot/include/include.h file.  Without this HAProxy cannot be
> compiled if the OpenTracing filter is to be used.
>
> Best regards,
>
> --
> Zaga
>
> What can change the nature of a man?
>


Re: [PATCH] CI: Build VTest with clang

2021-05-10 Thread Илья Шипицин
There are vtest build in cirrus and travis as well.

What if we move vtest building into "scripts/build-vtest.sh" ?

On Tue, May 11, 2021, 1:54 AM Tim Duesterhus  wrote:

> Willy,
> Ilya,
>
> not tested, but it should be simple enough to not mess it up.
>
> Best regards
> Tim Düsterhus
>
> Apply with `git am --scissors` to automatically cut the commit message.
>
> -- >8 --
> Current VTest master fails to build using gcc, see vtest/VTest#27.
>
> This patch is to be reverted once VTest is fixed.
> ---
>  .github/workflows/vtest.yml | 2 +-
>  .travis.yml | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/.github/workflows/vtest.yml b/.github/workflows/vtest.yml
> index cb52f27d6..1d62f98f3 100644
> --- a/.github/workflows/vtest.yml
> +++ b/.github/workflows/vtest.yml
> @@ -63,7 +63,7 @@ jobs:
>  curl -fsSL https://github.com/vtest/VTest/archive/master.tar.gz
> -o VTest.tar.gz
>  mkdir VTest
>  tar xvf VTest.tar.gz -C VTest --strip-components=1
> -make -C VTest -j$(nproc) FLAGS="-O2 -s -Wall"
> +make -C VTest -j$(nproc) FLAGS="-O2 -s -Wall" CC=clang
>  sudo install -m755 VTest/vtest /usr/local/bin/vtest
>  - name: Install SSL ${{ matrix.ssl }}
>if: ${{ matrix.ssl && matrix.ssl != 'stock' }}
> diff --git a/.travis.yml b/.travis.yml
> index 1aa415aa8..37b667bc1 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -40,7 +40,7 @@ matrix:
>  install:
>- git clone https://github.com/VTest/VTest.git ../vtest
># Special flags due to: https://github.com/vtest/VTest/issues/12
> -  - make -C ../vtest FLAGS="-O2 -s -Wall"
> +  - make -C ../vtest FLAGS="-O2 -s -Wall" CC=clang
>
>  script:
>- make -j$(nproc) ERR=1 TARGET=linux-glibc CC=$CC
> DEBUG=-DDEBUG_STRICT=1 $FLAGS
> --
> 2.31.1
>
>


[PATCH] spell check fixes

2021-05-10 Thread Илья Шипицин
Hello,

yet another spell check improvements.

Ilya
From 58ea7d81c586a609aa7bdea44d0c33a7de500fda Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Mon, 10 May 2021 12:50:00 +0500
Subject: [PATCH 2/2] CLEANUP: assorted typo fixes in the code and comments

This is 23rd iteration of typo fixes
---
 doc/configuration.txt | 2 +-
 src/activity.c| 4 ++--
 src/haproxy.c | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 964bc04ce..a716c3481 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -12247,7 +12247,7 @@ tcp-request content  [{if | unless} ]
   "tcp-request content" rules are not evaluated. This upgrade method should be
   preferred to the implicit one consisting to rely on the backend mode. When
   used, it is possible to set HTTP directives in a frontend without any
-  warning. These directives will be conditionaly evaluated if the HTTP upgrade
+  warning. These directives will be conditionally evaluated if the HTTP upgrade
   is performed. However, an HTTP backend must still be selected. It remains
   unsupported to route an HTTP connection (upgraded or not) to a TCP server.
 
diff --git a/src/activity.c b/src/activity.c
index 179ff1f32..ec230da82 100644
--- a/src/activity.c
+++ b/src/activity.c
@@ -112,7 +112,7 @@ static __attribute__((noreturn)) void memprof_die(const char *msg)
  * Worse, we have to account for the risk of reentrance from dlsym() when
  * it tries to prepare its error messages. Here its ahndled by in_memprof
  * that makes allocators return NULL. dlsym() handles it gracefully. An
- * alternate approch consists in calling aligned_alloc() from these places
+ * alternate approach consists in calling aligned_alloc() from these places
  * but that would mean not being able to intercept it later if considered
  * useful to do so.
  */
@@ -411,7 +411,7 @@ static int cli_parse_set_profiling(char **args, char *payload, struct appctx *ap
 	}
 
 	if (strcmp(args[2], "tasks") != 0)
-		return cli_err(appctx, "Expects etiher 'tasks' or 'memory'.\n");
+		return cli_err(appctx, "Expects either 'tasks' or 'memory'.\n");
 
 	if (strcmp(args[3], "on") == 0) {
 		unsigned int old = profiling;
diff --git a/src/haproxy.c b/src/haproxy.c
index c13beb487..1fd4a6be6 100644
--- a/src/haproxy.c
+++ b/src/haproxy.c
@@ -2623,7 +2623,7 @@ void run_poll_loop()
 			int i;
 
 			if (stopping) {
-/* stop muxes before acknowleding stopping */
+/* stop muxes before acknowledging stopping */
 if (!(stopping_thread_mask & tid_bit)) {
 	task_wakeup(mux_stopping_data[tid].task, TASK_WOKEN_OTHER);
 	wake = 1;
-- 
2.31.1

From 58064260ab0f4b65bad595e9e987d4d4223016a0 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Mon, 10 May 2021 12:45:18 +0500
Subject: [PATCH 1/2] CI: extend spellchecker whitelist, add "ists" as well

codespell does not handle plurals, we already whitelusted "ist", let us
whitelist "ists" as well
---
 .github/workflows/codespell.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml
index c2130ebf9..de49f4343 100644
--- a/.github/workflows/codespell.yml
+++ b/.github/workflows/codespell.yml
@@ -14,4 +14,4 @@ jobs:
 - name: install prerequisites
   run: sudo pip install codespell
 - name: check
-  run: codespell -c -q 2 --ignore-words-list ist,hist,wan,ca,cas,que,ans,te,nd,referer,ot,uint,iif,fo,keep-alives,dosen --skip="CHANGELOG,Makefile,*.fig,*.pem"
+  run: codespell -c -q 2 --ignore-words-list ist,ists,hist,wan,ca,cas,que,ans,te,nd,referer,ot,uint,iif,fo,keep-alives,dosen --skip="CHANGELOG,Makefile,*.fig,*.pem"
-- 
2.31.1



[PATCH] enable QUIC for CI builds when appropriate

2021-05-09 Thread Илья Шипицин
Hello,

USE_QUIC=1 added for BoringSSL builds.

thanks,
Ilya
From 2a79e92bdac01d926e7f85583f455db17f1515bb Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 9 May 2021 21:27:28 +0500
Subject: [PATCH] CI: Github Actions: enable USE_QUIC=1 for BoringSSL builds

if haproxy is built against BoringSSL, let us add USE_QUIC=1 dynamically
---
 .github/matrix.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/.github/matrix.py b/.github/matrix.py
index 577ad5b72..067ce93a3 100644
--- a/.github/matrix.py
+++ b/.github/matrix.py
@@ -109,6 +109,8 @@ for CC in ["gcc", "clang"]:
 "BORINGSSL=yes",
 ]:
 flags = ["USE_OPENSSL=1"]
+if ssl == "BORINGSSL=yes":
+flags.append("USE_QUIC=1")
 if ssl != "stock":
 flags.append("SSL_LIB=${HOME}/opt/lib")
 flags.append("SSL_INC=${HOME}/opt/include")
-- 
2.31.1



Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-05-08 Thread Илья Шипицин
I've created "proof of concept" OT support for CI

https://github.com/chipitsine/haproxy/runs/2535689580

Tim, are you ok with that approach ? If yes, I will polish things a bit.


currently "building OT and satellites" takes 2min 11sec. seem, I will add
similar caching level as for SSL variants (i.e. checking version in "opt"
and skip build if version is there).

also, I caught an error on clang:
https://github.com/haproxy/haproxy/issues/1242

I'm not very happy with unconditionally adding OT flags
https://github.com/chipitsine/haproxy/commit/dcf1812b17bd2907939eaa8d3310378d419d90c5#diff-b525f86b1f4925509959f857496dff18a9c4ed34fcc7f49357c5a6d00fb64d17R91
Tim, do you know how to add it using ${{ contains(matrix.FLAGS, 'USE_OT=1')
}} condition ?



чт, 8 апр. 2021 г. в 14:44, Илья Шипицин :

>
>
> чт, 8 апр. 2021 г. в 14:25, Willy Tarreau :
>
>> On Wed, Apr 07, 2021 at 05:26:24PM +0500,  ??? wrote:
>> > we run "all features anebled" gcc and clang builds, for example
>> > BUG/MINOR: tools: fix parsing "us" unit for timers ·
>> > haproxy/haproxy@a683805 (github.com)
>> > <
>> https://github.com/haproxy/haproxy/runs/2275440914?check_suite_focus=true
>> >
>> >
>> > <
>> https://github.com/haproxy/haproxy/runs/2275440914?check_suite_focus=true
>> >
>> > if
>> > additional libraries are easy to install (building will increase total
>> time
>> > a lot), I'd add opentracing to those "all features" builds
>>
>> I managed to build it myself so it's reasonably accessible, however we'd
>> possibly need to cache the builds, or we'll really start to spend a lot
>> of time building dependencies.
>>
>
> That is what I meant. if build is cheap - ok.
> if packages are available - ok.
>
> other options would be either caching dependencies (github ci supports
> caches) or provisioning custom docker images for our builds
>
>
>>
>> Willy
>>
>


[PATCH] CI: switch to the latest stable LibreSSL-3.3.3

2021-05-04 Thread Илья Шипицин
Hello,

LibreSSL-3.3.3 just released. patch attached.

thanks,
Ilya
From bddc3bb1f9726b1bee964a66e77575f054faae2b Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 5 May 2021 09:01:03 +0500
Subject: [PATCH] CI: Github Actions: switch to LibreSSL-3.3.3

stable LibreSSL-3.3.3 released, let us switch to it
---
 .github/matrix.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/matrix.py b/.github/matrix.py
index 22c33f0df..577ad5b72 100644
--- a/.github/matrix.py
+++ b/.github/matrix.py
@@ -105,7 +105,7 @@ for CC in ["gcc", "clang"]:
 "stock",
 "OPENSSL_VERSION=1.0.2u",
 "LIBRESSL_VERSION=2.9.2",
-"LIBRESSL_VERSION=3.2.5",
+"LIBRESSL_VERSION=3.3.3",
 "BORINGSSL=yes",
 ]:
 flags = ["USE_OPENSSL=1"]
-- 
2.31.1



[PATCH] typo fixes

2021-04-24 Thread Илья Шипицин
hello,

one more typo fixing.

Ilya
From 95a5f29e573ef02bc31c74d426f2db8fbdc1f57d Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 24 Apr 2021 13:25:42 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

This is 22nd iteration of typo fixes
---
 doc/configuration.txt| 6 +++---
 doc/management.txt   | 2 +-
 include/haproxy/action.h | 2 +-
 include/haproxy/cpuset.h | 2 +-
 include/import/slz.h | 2 +-
 src/action.c | 2 +-
 src/cfgdiag.c| 2 +-
 src/check.c  | 2 +-
 src/fd.c | 2 +-
 src/http_act.c   | 4 ++--
 src/http_ana.c   | 6 +++---
 src/sink.c   | 4 ++--
 src/slz.c| 8 
 src/tcpcheck.c   | 4 ++--
 14 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 375eedafa..db78c0704 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -6794,7 +6794,7 @@ http-request wait-for-body time  [ at-least  ]
   follows the HAProxy time format and is expressed in milliseconds.
 
is optional. It is the minimum payload size to receive to stop to
-  wait. It fallows the HAProxy size format and is expressed in
+  wait. It follows the HAProxy size format and is expressed in
   bytes.
 
   Example:
@@ -7248,7 +7248,7 @@ http-response wait-for-body time  [ at-least  ]
   follows the HAProxy time format and is expressed in milliseconds.
 
is optional. It is the minimum payload size to receive to stop to
-  wait. It fallows the HAProxy size format and is expressed in
+  wait. It follows the HAProxy size format and is expressed in
   bytes.
 
   Example:
@@ -12100,7 +12100,7 @@ tcp-request content  [{if | unless} ]
   "tcp-request content" rules are not evaluated. This upgrade method should be
   preferred to the implicit one consisting to rely on the backend mode. When
   used, it is possible to set HTTP directives in a frontend without any
-  warning. These directives will be conditionnaly evaluated if the HTTP upgrade
+  warning. These directives will be conditionaly evaluated if the HTTP upgrade
   is performed. However, an HTTP backend must still be selected. It remains
   unsupported to route an HTTP connection (upgraded or not) to a TCP server.
 
diff --git a/doc/management.txt b/doc/management.txt
index cee26839a..1e79eb788 100644
--- a/doc/management.txt
+++ b/doc/management.txt
@@ -1793,7 +1793,7 @@ set var  
may only involve "internal" sample fetch keywords and converters
   even though the most likely useful ones will be str('something') or int().
   Note that the command line parser doesn't know about quotes, so any space in
-  the expression must be preceeded by a backslash. This command requires levels
+  the expression must be preceded by a backslash. This command requires levels
   "operator" or "admin". This command is only supported on a CLI connection
   running in experimental mode (see "experimental-mode on").
 
diff --git a/include/haproxy/action.h b/include/haproxy/action.h
index 62fba7ed3..39f756f16 100644
--- a/include/haproxy/action.h
+++ b/include/haproxy/action.h
@@ -77,7 +77,7 @@ static inline void action_build_list(struct list *keywords,
 }
 
 /* Check an action ruleset validity. It returns the number of error encountered
- * andd err_code is updated if a warning is emitted.
+ * and err_code is updated if a warning is emitted.
  */
 int check_action_rules(struct list *rules, struct proxy *px, int *err_code);
 
diff --git a/include/haproxy/cpuset.h b/include/haproxy/cpuset.h
index f6cea4325..6e1627131 100644
--- a/include/haproxy/cpuset.h
+++ b/include/haproxy/cpuset.h
@@ -27,7 +27,7 @@ int ha_cpuset_count(const struct hap_cpuset *set);
 
 /* Returns the first index set plus one in  starting from the lowest.
  * Returns 0 if no index set.
- * Do not forget to substract the result by one if using it for set/clr.
+ * Do not forget to subtract the result by one if using it for set/clr.
  */
 int ha_cpuset_ffs(const struct hap_cpuset *set);
 
diff --git a/include/import/slz.h b/include/import/slz.h
index 0251a855f..0f284e303 100644
--- a/include/import/slz.h
+++ b/include/import/slz.h
@@ -151,7 +151,7 @@ static inline long slz_encode(struct slz_stream *strm, void *out,
  * It returns the number of bytes emitted. The trailer consists in flushing the
  * possibly pending bits from the queue (up to 24 bits), rounding to the next
  * byte, then 4 bytes for the CRC when doing zlib/gzip, then another 4 bytes
- * for the input length for gzip. That may abount to 4+4+4 = 12 bytes, that the
+ * for the input length for gzip. That may about to 4+4+4 = 12 bytes, that the
  * caller must ensure are available before calling the function.
  */
 static inline int slz_finish(struct slz_stream *strm, void *buf)
diff --git a/src/action.c b/src/action.c
index 5e430f23a..98359badd 

[PATCH] CI: travis-ci: enable graviton2 builds

2021-04-15 Thread Илья Шипицин
Hello,

as described in travis-ci docs:
https://blog.travis-ci.com/2020-09-11-arm-on-aws

it is next generation armv8 available on AWS.

cheers,
Ilya
From 4365c936396a5c7c4f710ed8267b06a73fd5bbcc Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 15 Apr 2021 19:16:09 +0500
Subject: [PATCH] CI: travis-ci: enable weekly graviton2 builds

as documented in https://blog.travis-ci.com/2020-09-11-arm-on-aws
we can build on graviton2, let us expand travis-ci matrix to that
machine type as well
---
 .travis.yml | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 2cb2d25a6..1aa415aa8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -26,6 +26,12 @@ matrix:
 arch: arm64
 compiler: gcc
 if: type == cron
+  - os: linux
+arch: arm64-graviton2
+group: edge
+virt: vm
+compiler: gcc
+if: type == cron
   - os: linux
 arch: s390x
 compiler: gcc
-- 
2.30.2



[PATCH] fix cirrus-ci builds by installing "pcre" package

2021-04-14 Thread Илья Шипицин
Hello,

something changed in freebsd packaging. we now need to install pcre
directly.

Ilya
From 406a5b8cfee330cc74c18f0eca1811195e7eff6d Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 14 Apr 2021 21:47:34 +0500
Subject: [PATCH] CI: cirrus: install "pcre" package

it turned out that our cirrus-ci freebsd builds got broken because
of missing "pcre". Most probably it was installed earlier as a dependency.
let us install it directly.
---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 0ac21bf0d..fdabfdcdd 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -4,7 +4,7 @@ FreeBSD_task:
   image_family: freebsd-12-2
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat
+- pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat pcre
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- 
2.30.2



Re: [ANNOUNCE] haproxy-2.4-dev16

2021-04-12 Thread Илья Шипицин
Dear Team,

can we address at least #1112, #1119 before 2.4 is released ?

пт, 9 апр. 2021 г. в 20:52, Willy Tarreau :

> Hi,
>
> HAProxy 2.4-dev16 was released on 2021/04/09. It added 37 new commits
> after version 2.4-dev15.
>
> This one is particularly calm, I even hesitated between making it or not.
> But there are a few updates that may affect configuration so I figured it
> was better to emit a new one.
>
> A large part of the patch is essentially caused by the renaming of a few
> atomic ops that were causing some confusion. Now HA_ATOMIC_FOO() will be
> a void statement so that if you want to read from the location you use
> either HA_ATOMIC_FETCH_FOO() or HA_ATOMIC_FOO_FETCH() (pre- or post-
> fetch).  The output code shouldn't change however, and given that it was
> essentially sed-work, as soon as it started to work I was confident in it.
>
> A few changes in the FD code to clean up that messy fdtab structure cause
> another noticeable part of the diff. I obviously managed to break
> something once (hence the BUG/MAJOR fix) but now it's OK. Mistakes at this
> level are never forgiving anyway, either it fully works or it fully fails.
>
> The nice part that makes me think we're progressively approaching what
> could look like the release is that Emeric finally performed the few
> changes in the DNS and log code. For the DNS, the TCP servers in the
> "resolvers" section do not need to be referred to as "server" anymore,
> they are "nameserver" just like the other ones, except that you can
> mention "tcp@" in front of their address to force them to be TCP
> nameservers. No more configuration mixing there. And for the log servers,
> similarly, now you can specify "tcp@" in front of an address on a "log"
> statement, and it will automatically create the ring for you with default
> settings. Previously it was still required to manually declare the ring, I
> found this too cumbersome, and Emeric figured how to handle this.
>
> The rest is essentially small bug fixes and code cleanups from a bunch
> of contributors.
>
> Now speaking about the pending stuff I'm still aware of:
>
>   - I'd like to rename LIST_ADD/LIST_ADDQ to LIST_INSERT/LIST_APPEND
> (or maybe LIST_ADD_HEAD/LIST_ADD_TAIL ?). I've already been trapped
> in using LIST_ADD() when I wanted to append and I know the confusion
> is easy. That's just a quick "sed" once we know what we want.
>
>   - I identified some undesired cache line sharing around some pointers,
> that slow down FD operations and possibly other stuff. I see how to
> arrange this, it just needs to be done (and tested on MacOS since we
> noticed once that section names differ there).
>
>   - we've had a recent discussion about the opportunity to import SLZ
> and to always enable compression by default using it if no other lib
> is specified. I think it could be useful for some users (especially
> those for whom memory usage is important). I'll have a look at this,
> maybe next week, that's only two files to include.
>
>   - regarding the quick discussion two weeks ago about optimization for
> modern ARM CPUs, I saw that one solution could be to build using
> gcc 10.2+ which outline their atomics into functions. That's ugly
> but the performance impact is small (about 3% in my tests), while
> it provides a tremendous improvement for many-core machines. But if
> we rely on this do this I'll probably add two new CPU entries to
> force to use only an old one (v8.0) or only a modern one (v8.2) so
> that those who build themselves can shave the last few percent.
>
>   - no progress made on the default-path, but we'll have to reread the
> issue to figure the best choice. I'd like to see it done for the
> release to ease config packaging and deployments.
>
>   - I'd like to add a warning when we detect that nbthreads is forced
> to a higher value than the number of bound CPUs. It's not the first
> time that I see this in a config and the result is catastrophic, so
> a warning is deserved. It just needs to be set at the right place.
>
>   - the shared memory pools cleanup must be completed before the release,
> as the situation is not good for those with a slow malloc() function
> at the moment. I know what to do, I just need to stay focused on it.
>
>   - the date & time cleanups would be nice to have for the long term and
> are not particularly hard to do so better finish them before 2.4.
>
>   - Tim sent a patch series to implement URI normalization. That's
> something
> I'd like to see merged if possible, as it may improve security for some
> users, and at least improve reliability for others.
>
>   - I also noticed Alek's mjson import with a new converter, but didn't
> have a look yet. Maybe it will open opportunities for more converters,
> that's definitely something which deserves being considered before the
> release.
>
>   - Amaury has almost finished some 

Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-04-08 Thread Илья Шипицин
чт, 8 апр. 2021 г. в 14:25, Willy Tarreau :

> On Wed, Apr 07, 2021 at 05:26:24PM +0500,  ??? wrote:
> > we run "all features anebled" gcc and clang builds, for example
> > BUG/MINOR: tools: fix parsing "us" unit for timers ·
> > haproxy/haproxy@a683805 (github.com)
> > <
> https://github.com/haproxy/haproxy/runs/2275440914?check_suite_focus=true>
> >
> > <
> https://github.com/haproxy/haproxy/runs/2275440914?check_suite_focus=true>
> > if
> > additional libraries are easy to install (building will increase total
> time
> > a lot), I'd add opentracing to those "all features" builds
>
> I managed to build it myself so it's reasonably accessible, however we'd
> possibly need to cache the builds, or we'll really start to spend a lot
> of time building dependencies.
>

That is what I meant. if build is cheap - ok.
if packages are available - ok.

other options would be either caching dependencies (github ci supports
caches) or provisioning custom docker images for our builds


>
> Willy
>


Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-04-07 Thread Илья Шипицин
On Wed, Apr 7, 2021, 2:16 PM Miroslav Zagorac  wrote:

> On 04/07/2021 12:13 PM, Илья Шипицин wrote:
> > do you consider adding opentracing to "github actions" CI ?
> >
>
> Hello Илья,
>
> I don't know how to add it because I never used it.  The filter uses
> specific libraries that are not part of the system but need to be
> installed and/or compiled independently.
>

we run "all features anebled" gcc and clang builds, for example
BUG/MINOR: tools: fix parsing "us" unit for timers ·
haproxy/haproxy@a683805 (github.com)
<https://github.com/haproxy/haproxy/runs/2275440914?check_suite_focus=true>

<https://github.com/haproxy/haproxy/runs/2275440914?check_suite_focus=true>if
additional libraries are easy to install (building will increase total time
a lot), I'd add opentracing to those "all features" builds

>
> Best regards,
>
> --
> Zaga
>
> What can change the nature of a man?
>


Re: [PATCH] MINOR: opentracing: register config file and line number on log servers

2021-04-07 Thread Илья Шипицин
do you consider adding opentracing to "github actions" CI ?

ср, 7 апр. 2021 г. в 14:35, Miroslav Zagorac :

> Hello,
>
> due to the modified function declaration, the opentracing filter can no
> longer be compiled.
>
> In commit 9533a7038 new parameters have been added to the declaration
> of function parse_logsrv().
>
> This patch should be backported to all branches where the OpenTracing
> filter is located.
>
> --
> Zaga
>
> What can change the nature of a man?
>


Re: zlib vs slz (perfoarmance)

2021-03-30 Thread Илья Шипицин
вт, 30 мар. 2021 г. в 17:44, Willy Tarreau :

> On Tue, Mar 30, 2021 at 02:25:14PM +0500,  ??? wrote:
> > I would wait for feedback from some other high load projects. From my
> > observation it is significant benefit.
>
> Originally slz was written for use in haproxy to solve the huge memory
> consumption that comes with zlib (~288kB of context per stream IIRC).
> SLZ needs much less and uses ~40 bytes or so per stream, and by keeping
> less context, has less opportunities for high compression ratios. This
> simplified its lookup algorithms, and as a byproduct made it significantly
> faster.
>


for example, silesia xml, Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz

lzbench suite

[root@localhost lzbench]# ./lzbench -ezlib,1/slz_zlib,1  silezia/xml
lzbench 1.8 (64-bit Linux)   Assembled by P.Skibinski
Compressor name Compress. Decompress. Compr. size  Ratio Filename
memcpy  10948 MB/s 15766 MB/s 5345280 100.00 silezia/xml
zlib 1.2.11 -1125 MB/s   429 MB/s  965248  18.06 silezia/xml
slz_zlib 1.2.0 -1 329 MB/s   331 MB/s 1294302  24.21 silezia/xml
done... (cIters=1 dIters=1 cTime=1.0 dTime=2.0 chunkSize=1706MB cSpeed=0MB)
[root@localhost lzbench]#


I observe that slz is more than 2 times faster compared to zlib, while
compressed file size is 20% bigger.
but I did not yet checked zlib-ng.

also, I read that slz stops compressing when CPU level reaches some
threshold. is it related to all gzip, including zlib ?
if so, we can safely stick with any compression lib (but I agree that
having benchmarks would help people)


>
> The only sane way I've seen to use zlib is to limit its CPU usage. When it
> reaches a target ratio (typically 80%), compression automatically disables
> itself, so overall the savings-to-cpu usage makes it less interesting for
> zlib in our use cases. The only case where zlib is more interesting is for
> those having a small line and who prefer to spend more CPU cycles to get
> more savings.
>
> One reasonable approach we could think of would consist in importing SLZ
> directly into haproxy now. I didn't want to do it initially because the
> code was young and I wanted it to live its own life and get its own fixes.
> Now the burn-in test is long finished, and the only updates over the last
> 4 years were for performance improvements or fixes for use cases outside
> of haproxy. That's only 1500 lines of code to integrate and it would
> certainly simplify a lot of things. We could even imagine to always build
> with compression enabled, SLZ when not specified otherwise zlib when
> asked for it.
>
> Just a few ideas.
> Willy
>


Re: zlib vs slz (perfoarmance)

2021-03-30 Thread Илья Шипицин
I would wait for feedback from some other high load projects. From my
observation it is significant benefit.

On Tue, Mar 30, 2021, 12:18 PM Dinko Korunic 
wrote:

> On 30.03.2021., at 08:17, Илья Шипицин  wrote:
> >
> > I would really like to know whether zlib was chosen for purpose or by
> chance.
> >
> > And yes, some marketing campaign makes sense
> >
>
> Hi Илья,
>
> People tend to spawn Docker images in dozens and/or even hundreds and we
> always have to consider that adding a single library on top of what’s
> already present in the minimal Docker base image (libz is pretty much
> always present in base images) will result in additional size which is in
> general frown upon by Docker users.
>
> But then again, if the community (aka you) thinks that pros (performance)
> outweigh the cons (increased target size), I’ll take care of it for
> haproxytech images and these changes would be also propagated to Ingress
> Controller image as well.
>
>
> Kind regards,
> D.
>
> --
> Dinko Korunic   ** Standard disclaimer applies **
> Sent from OSF1 osf1v4b V4.0 564 alpha
>
>
>


Re: zlib vs slz (perfoarmance)

2021-03-30 Thread Илья Шипицин
I would really like to know whether zlib was chosen for purpose or by
chance.

And yes, some marketing campaign makes sense

On Tue, Mar 30, 2021, 10:35 AM Dinko Korunic 
wrote:

>
> > On 29.03.2021., at 23:06, Lukas Tribus  wrote:
> >
>
> […]
>
> > Like I said last year, this needs a marketing campaign:
> > https://www.mail-archive.com/haproxy@formilux.org/msg38044.html
> >
> >
> > What about the docker images from haproxytech? Are those zlib or slz
> > based? Perhaps that would be a better starting point?
> >
> > https://hub.docker.com/r/haproxytech/haproxy-alpine
>
>
>
> Hi Lukas,
>
> I am maintaining haproxytech Docker images and I can easily make that (slz
> being used) happen, if that’s what community would like to see.
>
>
> Kind regards,
> D.
>
> --
> Dinko Korunic   ** Standard disclaimer applies **
> Sent from OSF1 osf1v4b V4.0 564 alpha
>
>
>


Re: zlib vs slz (perfoarmance)

2021-03-29 Thread Илья Шипицин
пн, 29 мар. 2021 г. в 19:35, Lukas Tribus :

> Hi Ilya,
>
>
> On Mon, 29 Mar 2021 at 15:34, Илья Шипицин  wrote:
> >
> > Dear list,
> >
> > on browser load (html + js + css) I observe 80% of cpu spent on gzip.
> > also, I observe that zlib is probably one of the slowest implementation
> > my personal benchmark correlate with https://github.com/inikep/lzbench
> >
> > if so, should'n we switch to slz by default ? or am I missing something ?
>
> There is no default.
>
> zlib is optional.
> slz is optional.
>
> Haproxy is compiled with either zlib, slz or no compression library at all.
>
>
> What specifically are you suggesting to change in the haproxy source tree?
>

for example, docker image
https://hub.docker.com/_/haproxy


>
>
> Regards,
> Lukas
>


  1   2   3   4   5   6   7   8   9   >