Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

2020-01-23 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 15:20, Willy Tarreau :

> On Wed, Jan 22, 2020 at 11:10:05PM +0100, Tim Düsterhus wrote:
> > > both travis and github actions do offer 4 parallel builds, while
> cirrus and
> > > app veyor offer 1 parallel build.
> >
> > Parallel Builds just improve test speed. I don't consider that an
> > important selling point for us. The development process is fairly
> > asynchronous anyway and the important thing is that there are results
> > for more obscure configurations, not that there results within 1 minute.
>
> Sometimes it's nice to get a low-latency feedback on recent changes. But
> lately travis has become amazingly slow, turning from tens of seconds to
> several minutes, and regtests occasionally failing on timeouts, so it's
> clear that this low-latency argument is not true anymore. I think they're
> victim of their success and the spawned VMs are simply overloaded by too
> many users doing lots of builds there. We do try to be good citizens by
> limiting our combinations but it's possibly not the case for everyone.
>
> I still think we should not be too demanding on these tools. They do
> help us, 2.0 and 2.1's quality at the release date was far above
> anything we've done in the past, so I'm fine if we continue to use a
> diversity of tools depending on their capabilities. For sure it's less
> work to always look at a single place but that's no big deal overall.
>


we can simplify things a bit.
if we turn README into README.md , so we can add badges and links to builds.
it saves time, badges are either green or red. if it is green, ok, no need
to look deeper (amd mo need to remember links to CI)


like that

[image: Screenshot from 2020-01-24 10-59-18.png]



>
> Willy
>


Re: arm64 builds?

2020-01-23 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 15:14, Willy Tarreau :

> On Thu, Jan 23, 2020 at 01:09:22PM +0500,  ??? wrote:
> > hello,
> >
> > I tried to build using cross compiler (arm32 on amd64).  sorry for
> > screenshot.
> > Willy, do you mean errors like that ?
>
> So for those not seeing the screenshot, it says:
> warning: argument 1 range [2147483648, 4294967295] exceeds maximum object
> size 2147483647 :
>
>  new->idle_orphan_conns = calloc((unsigned int)global.nbthread,
> sizeof(*new->idle_orphan_conns));
>
> Thus the cast to unsigned int was likely placed there to shut it up
> once. Looking at the man page, it says:
>
>void *calloc(size_t nmemb, size_t size);
>
> size_t is unsigned (unless you're using an older gcc < 2.4 on a unusual
> OS). I don't see how it can be said to have a maximum size of 2147483647.
>
> This is exactly the type of stupid warnings that causes us to add casts
> that hide real bugs :-/
>
> Could you please try to use (size_t) instead of (unsigned int) ? If it's
> enough to shut it up, I'm fine with doing that change. Otherwise we'll
> probably get rid of that stupid warning.
>

yep.
I'll play with it a liitle bit, and hopefully we will add cross builds to
Github Actions.


also, can we treat warnings as errors for CI builds ? it would save a bunch
of time, instead of
looking at build log, we can watch for build status.


>
> Thanks!
> Willy
>


Re: arm64 builds?

2020-01-23 Thread Илья Шипицин
hello,

I tried to build using cross compiler (arm32 on amd64).  sorry for
screenshot.
Willy, do you mean errors like that ?


[image: Screenshot from 2020-01-23 13-03-49.png]


ср, 6 нояб. 2019 г. в 12:26, Willy Tarreau :

> Hi Ilya,
>
> On Tue, Nov 05, 2019 at 07:20:43PM +0500,  ??? wrote:
> > only arm64 are available.
> > we can try arm using cross build, for example.
>
> Then don't bother with this, it's likely then that they will not
> have all the environment available. Maybe even their hardware does
> not support arm32 at all. It was just a suggestion to try to optimize
> the solution but even arm64 is already nice to have.
>
> > is it common way to use cross build for building haproxy ?
>
> Yes it is. Actually I don't think I've built it natively for a very
> long time now, as even on my laptop I'm used to stick to the cross-
> compilers which are distributed on the distcc build farm running on
> the lab load generators :-)
>
> In practice just pass "CC=/path/to/gcc" and let it do its job. It will
> automatically use LD=$(CC) if you don't override it. You may have to
> pass PCRE_INC/PCRE_LIB, SSL_INC/SSL_LIB, LUA_INC/LUA_LIB but that's
> about all.
>
> Just for reference here's the command line I'm using when building
> (and a variant with -O0 which builds in 3.5 seconds). It may look
> large because I force all debugging options but it's in a script so
> I don't have to type it :-)
>
>PATH=/f/tc/x86_64-gcc47_glibc218-linux-gnu/bin:$PATH make -j 120
> TMPDIR=/dev/shm DISTCC_FALLBACK=0 DISTCC_SKIP_LOCAL_RETRY=0
> DISTCC_BACKOFF_PERIOD=0 DISTCC_PAUSE_TIME_MSEC=50
> DISTCC_HOSTS="--localslots_cpp=120 10.0.0.235/120,lzo"
> CC=/g/public/linux/master/x86_64-gcc47_glibc218-linux-gnu-gcc
> TARGET=linux-glibc DEP= USE_PCRE=1 PCREDIR=
> DEFINE="-DDEBUG_DONT_SHARE_POOLS -DDEBUG_MEMORY_POOLS -DDEBUG_UAF
> -DDEBUG_EXPR -DDEBUG_STRICT -DDEBUG_DEV" USE_OPENSSL=1 USE_ZLIB=1 USE_LUA=1
> LUA_LIB_NAME=lua EXTRA= USE_SLZ=1 SLZ_INC=/g/public/slz/src
> SLZ_LIB=/g/public/slz USE_ZLIB= USE_DEVICEATLAS=1
> DEVICEATLAS_SRC=contrib/deviceatlas USE_51DEGREES=1
> 51DEGREES_SRC=contrib/51d/src/trie USE_WURFL=1 WURFL_INC=contrib/wurfl
> WURFL_LIB=contrib/wurfl CPU_CFLAGS="-O2" "$@"
>
> When testing with various local openssl branches I do have another variant
> of this which uses the local, native pre-processor and linkers, and remote
> compilers. It's a bit ugly since they're not on the same version but in
> practice it works fine.
>
> Cheers,
> Willy
>


Re: [PATCH v3] BUG/MINOR: dns: allow 63 char in hostname

2020-01-26 Thread Илья Шипицин
such things are fragile.  once fixed, they can silently break during
further refactoring.
on other hand, such functions are good candidates to write unit tests.

maybe we should consider things like cmocka ?

вс, 26 янв. 2020 г. в 22:44, William Dauchy :

> hostname were limited to 62 char, which is not RFC1035 compliant;
> - the parsing loop should stop when above max label char
> - fix len label test where d[i] was wrongly used
> - simplify the whole function to avoid using two extra char* variable
>
> this should fix github issue #387
>
> Signed-off-by: William Dauchy 
> ---
>  src/dns.c | 31 +--
>  1 file changed, 13 insertions(+), 18 deletions(-)
>
> diff --git a/src/dns.c b/src/dns.c
> index eefd8d0dc..212c55f0d 100644
> --- a/src/dns.c
> +++ b/src/dns.c
> @@ -1470,7 +1470,6 @@ int dns_str_to_dn_label(const char *str, int
> str_len, char *dn, int dn_len)
>   */
>  int dns_hostname_validation(const char *string, char **err)
>  {
> -   const char *c, *d;
> int i;
>
> if (strlen(string) > DNS_MAX_NAME_SIZE) {
> @@ -1479,36 +1478,32 @@ int dns_hostname_validation(const char *string,
> char **err)
> return 0;
> }
>
> -   c = string;
> -   while (*c) {
> -   d = c;
> -
> +   while (*string) {
> i = 0;
> -   while (*d != '.' && *d && i <= DNS_MAX_LABEL_SIZE) {
> -   i++;
> -   if (!((*d == '-') || (*d == '_') ||
> - ((*d >= 'a') && (*d <= 'z')) ||
> - ((*d >= 'A') && (*d <= 'Z')) ||
> - ((*d >= '0') && (*d <= '9' {
> +   while (*string && *string != '.' && i <
> DNS_MAX_LABEL_SIZE) {
> +   if (!(*string == '-' || *string == '_' ||
> + (*string >= 'a' && *string <= 'z') ||
> + (*string >= 'A' && *string <= 'Z') ||
> + (*string >= '0' && *string <= '9'))) {
> if (err)
> *err = DNS_INVALID_CHARACTER;
> return 0;
> }
> -   d++;
> +   i++;
> +   string++;
> }
>
> -   if ((i >= DNS_MAX_LABEL_SIZE) && (d[i] != '.')) {
> +   if (*string == '\0')
> +   break;
> +
> +   if (*string != '.' && i >= DNS_MAX_LABEL_SIZE) {
> if (err)
> *err = DNS_LABEL_TOO_LONG;
> return 0;
> }
>
> -   if (*d == '\0')
> -   goto out;
> -
> -   c = ++d;
> +   string++;
> }
> - out:
> return 1;
>  }
>
> --
> 2.24.1
>
>
>


Re: [PATCH v3] BUG/MINOR: dns: allow 63 char in hostname

2020-01-26 Thread Илья Шипицин
вс, 26 янв. 2020 г. в 23:12, William Dauchy :

> On Sun, Jan 26, 2020 at 7:08 PM Илья Шипицин  wrote:
> > such things are fragile.  once fixed, they can silently break during
> further refactoring.
> > on other hand, such functions are good candidates to write unit tests.
>
> I considered it but to my knowledge, this is currently not possible
> with varnishtest, as we would need to mock a dns resolution, and make
> haproxy starts. I don't know whether there are other plans for haproxy
> tests.
>


I do not mean varnishtest here.

varnishtest is "full stack functional test", it is too expensive.

I mean lightweight unit testing, for example, cmocka.


> --
> William
>


Re: Disabling regtests in Travis ?

2020-01-28 Thread Илья Шипицин
btw, we can remove "allowed failure" since this test is slow and excluded

https://travis-ci.com/haproxy/haproxy/jobs/280882138

пн, 27 янв. 2020 г. в 17:35, Martin Grigorov :

>
>
> On Fri, Jan 24, 2020 at 6:43 PM Willy Tarreau  wrote:
>
>> On Fri, Jan 24, 2020 at 09:12:58PM +0500,  ??? wrote:
>> > >> +  - make reg-tests VTEST_PROGRAM=../vtest/vtest
>> > >> REGTESTS_TYPES=default,bug,devel
>> > >>
>> > >
>> > > let us try that.
>>
>> OK, now pushed.
>>
>> > > I will have a look at "racy" tests.
>> > > Maybe we'll enable them on Github Actions.
>> > >
>> > >
>> > the good thing about Github Actions, it is possible to attach own build
>> > agents. So, if we
>> > have dedicated hardware and we not want to depend on travis-ci
>> neighbours,
>> > it might be an option.
>>
>> That's good to know, even if I doubt we'd need it, at least it
>> opens possibilities.
>>
>
> The regtests run fine on my ARM64 VM. I run them daily.
> If HAProxy team decides to move to GitHub Actions and to use an external
> build agent for ARM64 then just ping me!
>
> Regards,
> Martin
>
>
>>
>> Willy
>>
>>


Re: Lua detection on aarch64

2020-01-30 Thread Илья Шипицин
I'm stupid. I took

%{?el8:BuildRequires: lua-devel}
%{?el7:BuildRequires: lua53-static}
%{?el7:BuildRequires: lua53-devel}

obviously, it does not contain fedora

ср, 29 янв. 2020 г. в 19:54, William Lallemand :

> On Wed, Jan 29, 2020 at 02:34:19PM +0500, Илья Шипицин wrote:
> > Hello,
> >
> > I started to work on rpm packages.
> > I also applied at Linaro for arm64 vm (request is not completed yet).
> >
> >
> > interesting that Lua is not detected:
> >
> >
> https://copr-be.cloud.fedoraproject.org/results/chipitsine/haproxy-rpm/fedora-31-aarch64/01207488-haproxy/builder-live.log.gz
> >
> > Martin, do you have arm64, can you check that ?
> >
> > cheers,
> > Ilya Shipitcin
>
> You probably need a lua-devel package which contain the .a and the .h.
>
> --
> William Lallemand
>


how to properly reload haproxy (from systemd + master-worker) ?

2020-02-02 Thread Илья Шипицин
Hello,

I'm playing with RPM built here:
https://copr.fedorainfracloud.org/coprs/roidelapluie/haproxy/


it is shipped with systemd unit:

ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q
ExecReload=/bin/kill -USR2 $MAINPID


actually it works.
however, if I add dataplane api:

-
global
  master-worker

userlist controller
  user dataplaneapi insecure-password xxx

program api
  command dataplaneapi --host 0.0.0.0 --port  --haproxy-bin
/usr/sbin/haproxy --config-file /etc/haproxy/haproxy.cfg --log-to=file
--log-file=/var/log/dataplaneapi.log --log-level=trace --log-format=text
--reload-cmd "systemctl reload haproxy" --reload-delay 5
---


haproxy terminates on "systemctl reload haproxy" (which for example happens
from within dataplaneapi).

I guess, something is wrong with two above mentioned "ExecReload" commands.
any advice how to adjust them ?


thanks,
Ilya Shipitcin


Re: how to properly reload haproxy (from systemd + master-worker) ?

2020-02-02 Thread Илья Шипицин
btw, is it possible to configure haproxy "to run either with or without
configured program" ?

actually, I'm ok if it will start even if dataplane api failed.

вс, 2 февр. 2020 г. в 22:58, Tim Düsterhus :

> Ilya,
>
> Am 02.02.20 um 18:52 schrieb Илья Шипицин:
> > Feb 02 20:50:07 xxx haproxy[2851]: [NOTICE] 032/205007 (2851) : New
> worker
> > #1 (2897) forked
> > Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) : Current
> > program 'api' (2896) exited with code 1 (Exit)
>
> The 'api' program (i.e. the dataplane API) died with code 1 ...
>
> > Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) :
> > exit-on-failure: killing every processes with SIGTERM
>
> ... leading to HAProxy terminating every worker ...
>
> > Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) : Current
> > worker #1 (2897) exited with code 143 (Terminated)
> > Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : Former
> > worker #1 (2854) exited with code 143 (Terminated)
> > Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : Former
> > program 'api' (2853) exited with code 0 (Exit)
> > Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : All
> > workers exited. Exiting... (1)
>
> ... leading to the master stopping itself.
>
> > Feb 02 20:50:07 xxx systemd[1]: haproxy.service: main process exited,
> > code=exited, status=1/FAILURE
> > Feb 02 20:50:07 xxx systemd[1]: Unit haproxy.service entered failed
> state.
> > Feb 02 20:50:07 xxx systemd[1]: haproxy.service failed.
>
> ... leading to the unit being stopped.
>
> So you would have to find out with the dataplane API dies. My educated
> guess would be that it fails because the port is already being in use by
> the old dataplane API process.
>
> Best regards
> Tim Düsterhus
>


Re: how to properly reload haproxy (from systemd + master-worker) ?

2020-02-02 Thread Илья Шипицин
  global
option  httplog
option  dontlognull
option http-server-close
option forwardfor   except 127.0.0.0/8
option  redispatch
retries 3
timeout http-request10s
timeout queue   1m
timeout connect 10s
timeout client  1m
timeout server  1m
timeout http-keep-alive 10s
timeout check   10s
maxconn 3000


userlist controller
  user dataplaneapi insecure-password example

program api
  command dataplaneapi --host 127.0.0.1 --port  --haproxy-bin
/usr/sbin/haproxy --config-file /etc/haproxy/haproxy.cfg --log-to=file
--log-file=/var/log/dataplaneapi.log --log-level=trace --log-format=text
--reload-cmd "systemctl reload haproxy" --reload-delay 5
[root@xxx ~]#


[root@xxx ~]# systemctl enable haproxy
[root@xxx ~]# systemctl start haproxy
[root@xxx ~]# systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor
preset: disabled)
   Active: active (running) since Sun 2020-02-02 20:49:56 MSK; 3s ago
  Process: 2849 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q
(code=exited, status=0/SUCCESS)
 Main PID: 2851 (haproxy)
   CGroup: /system.slice/haproxy.service
   ├─2851 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid
   ├─2853 dataplaneapi --host 127.0.0.1 --port  --haproxy-bin
/usr/sbin/haproxy --config-file /etc/haproxy/haproxy.cfg --log-to=file
--log-file=/var/log/dataplaneapi.log --log-level=trace --log-format=te...
   └─2854 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p
/run/haproxy.pid

Feb 02 20:49:56 xxx systemd[1]: Starting HAProxy Load Balancer...
Feb 02 20:49:56 xxx haproxy[2851]: [NOTICE] 032/204956 (2851) : New program
'api' (2853) forked
Feb 02 20:49:56 xxx haproxy[2851]: [NOTICE] 032/204956 (2851) : New worker
#1 (2854) forked
Feb 02 20:49:56 xxx systemd[1]: Started HAProxy Load Balancer.
[root@xxx ~]# systemctl reload haproxy
[root@xxx ~]# systemctl status haproxy
● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor
preset: disabled)
   Active: failed (Result: exit-code) since Sun 2020-02-02 20:50:07 MSK; 1s
ago
  Process: 2894 ExecReload=/bin/kill -USR2 $MAINPID (code=exited,
status=0/SUCCESS)
  Process: 2892 ExecReload=/usr/sbin/haproxy -f $CONFIG -c -q (code=exited,
status=0/SUCCESS)
  Process: 2851 ExecStart=/usr/sbin/haproxy -Ws -f $CONFIG -p $PIDFILE
$OPTIONS (code=exited, status=1/FAILURE)
  Process: 2849 ExecStartPre=/usr/sbin/haproxy -f $CONFIG -c -q
(code=exited, status=0/SUCCESS)
 Main PID: 2851 (code=exited, status=1/FAILURE)

Feb 02 20:50:07 xxx haproxy[2851]: [NOTICE] 032/205007 (2851) : New worker
#1 (2897) forked
Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) : Current
program 'api' (2896) exited with code 1 (Exit)
Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) :
exit-on-failure: killing every processes with SIGTERM
Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) : Current
worker #1 (2897) exited with code 143 (Terminated)
Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : Former
worker #1 (2854) exited with code 143 (Terminated)
Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : Former
program 'api' (2853) exited with code 0 (Exit)
Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : All
workers exited. Exiting... (1)
Feb 02 20:50:07 xxx systemd[1]: haproxy.service: main process exited,
code=exited, status=1/FAILURE
Feb 02 20:50:07 xxx systemd[1]: Unit haproxy.service entered failed state.
Feb 02 20:50:07 xxx systemd[1]: haproxy.service failed.
[root@xxx ~]#


вс, 2 февр. 2020 г. в 19:18, Tim Düsterhus :

> Ilya,
>
> Am 02.02.20 um 14:54 schrieb Илья Шипицин:
> > haproxy terminates on "systemctl reload haproxy" (which for example
> happens
> > from within dataplaneapi).
>
> Any error message logged in systemd-journald? Could it be that you hit
> this systemd bug: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=936032
> ?
>
> Best regards
> Tim Düsterhus
>


Re: how to properly reload haproxy (from systemd + master-worker) ?

2020-02-02 Thread Илья Шипицин
however, I copy-pasted

--reload-cmd "systemctl reload haproxy"

from dataplane api docs. seems, at least someone (who wrote that docs) was
able to run systemd + haproxy + dataplaneapi

sure, port is used by dataplaneapi itselft.

вс, 2 февр. 2020 г. в 22:58, Tim Düsterhus :

> Ilya,
>
> Am 02.02.20 um 18:52 schrieb Илья Шипицин:
> > Feb 02 20:50:07 xxx haproxy[2851]: [NOTICE] 032/205007 (2851) : New
> worker
> > #1 (2897) forked
> > Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) : Current
> > program 'api' (2896) exited with code 1 (Exit)
>
> The 'api' program (i.e. the dataplane API) died with code 1 ...
>
> > Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) :
> > exit-on-failure: killing every processes with SIGTERM
>
> ... leading to HAProxy terminating every worker ...
>
> > Feb 02 20:50:07 xxx haproxy[2851]: [ALERT] 032/205007 (2851) : Current
> > worker #1 (2897) exited with code 143 (Terminated)
> > Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : Former
> > worker #1 (2854) exited with code 143 (Terminated)
> > Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : Former
> > program 'api' (2853) exited with code 0 (Exit)
> > Feb 02 20:50:07 xxx haproxy[2851]: [WARNING] 032/205007 (2851) : All
> > workers exited. Exiting... (1)
>
> ... leading to the master stopping itself.
>
> > Feb 02 20:50:07 xxx systemd[1]: haproxy.service: main process exited,
> > code=exited, status=1/FAILURE
> > Feb 02 20:50:07 xxx systemd[1]: Unit haproxy.service entered failed
> state.
> > Feb 02 20:50:07 xxx systemd[1]: haproxy.service failed.
>
> ... leading to the unit being stopped.
>
> So you would have to find out with the dataplane API dies. My educated
> guess would be that it fails because the port is already being in use by
> the old dataplane API process.
>
> Best regards
> Tim Düsterhus
>


Re: how to properly reload haproxy (from systemd + master-worker) ?

2020-02-03 Thread Илья Шипицин
вт, 4 февр. 2020 г. в 00:20, William Lallemand :

> On Sun, Feb 02, 2020 at 11:23:09PM +0500, Илья Шипицин wrote:
> > btw, is it possible to configure haproxy "to run either with or without
> > configured program" ?
> >
>
> It's not possible.
>
> > actually, I'm ok if it will start even if dataplane api failed.
> >
>
> Well, I don't want this to happen, because we will have to handle a failure
> status within for the program section, and start to develop a real process
> manager. If you need complicated condition like this, I suggest you to
> start
> the dataplane-api with your init system.
>

I just followed some examples taken from docs.

I'm pretty ok with using systemd for starting processes. Actually, I do not
see any benefit for starting dataplane api
from haproxy itself (taking into consideration that haproxy is not real
process manager)


>
>
> > вс, 2 февр. 2020 г. в 22:58, Tim Düsterhus :
> >
> > [...]
> > > > Feb 02 20:50:07 xxx systemd[1]: haproxy.service failed.
> > >
> > > ... leading to the unit being stopped.
> > >
> > > So you would have to find out with the dataplane API dies. My educated
> > > guess would be that it fails because the port is already being in use
> by
> > > the old dataplane API process.
> > >
>
> From what I understand of the dataplaneapi, it is reloaded upon a USR1
> signal,
> signal which is sent by the master process upon a reload.
>
> So you probably just need to add "no option start-on-reload" in your
> program
> section to prevent HAProxy to launch the dataplane-api again.
>


thank you, I'll try

maybe it should be added to docs.

>
> --
> William Lallemand
>


[PATCH] add workaround to cirrus-ci

2020-02-08 Thread Илья Шипицин
Hello,

I attached a workaround to recent cirrus-ci failures.
also, few improvements.

Cheers,
Ilya Shipitcin
From bbefa0574b0e3b8e872422f9419a458928a433d0 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 8 Feb 2020 13:30:54 +0500
Subject: [PATCH] cirrus-ci: workaround "pkg install" bug

there's a bug https://github.com/freebsd/pkg/issues/902
adding "pkg update -f && pkg upgrade -y" is workaround.

also, add ERR=1 to make, also switch to "snap" images to unify
openssl package naming. also, added freebsd-13 build
---
 .cirrus.yml | 13 -
 1 file changed, 4 insertions(+), 9 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 4143e687b..1a07c80c7 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,21 +1,16 @@
 FreeBSD_task:
   freebsd_instance:
 matrix:
-  image_family: freebsd-12-1
+  image_family: freebsd-13-0-snap
+  image_family: freebsd-12-1-snap
   image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- case `uname -r` in
-11.3*)
-  export SSL=openssl;;
-12.1*)
-  export SSL=openssl111;;
-  esac
-- pkg install -y $SSL git gmake lua53 socat
+- pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- gmake CC=clang V=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
+- gmake CC=clang V=1 ERR=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
 - ./haproxy -vv
 - ldd haproxy
 - env VTEST_PROGRAM=../vtest/vtest gmake reg-tests || (for folder in /tmp/*regtest*/vtc.*; do cat $folder/INFO $folder/LOG; done && exit 1)
-- 
2.24.1



Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

2020-01-23 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 15:20, Willy Tarreau :

> On Wed, Jan 22, 2020 at 11:10:05PM +0100, Tim Düsterhus wrote:
> > > both travis and github actions do offer 4 parallel builds, while
> cirrus and
> > > app veyor offer 1 parallel build.
> >
> > Parallel Builds just improve test speed. I don't consider that an
> > important selling point for us. The development process is fairly
> > asynchronous anyway and the important thing is that there are results
> > for more obscure configurations, not that there results within 1 minute.
>
> Sometimes it's nice to get a low-latency feedback on recent changes. But
> lately travis has become amazingly slow, turning from tens of seconds to
> several minutes, and regtests occasionally failing on timeouts, so it's
> clear that this low-latency argument is not true anymore. I think they're
> victim of their success and the spawned VMs are simply overloaded by too
> many users doing lots of builds there. We do try to be good citizens by
> limiting our combinations but it's possibly not the case for everyone.
>

those timeouts are not related to travis itself, I beleive they are mostly
related to either
real failures or tests instability (race condition).

the bigger is number of tests, the more we depend on those timeouts.

however, we can try to run tests in parallel :)


>
> I still think we should not be too demanding on these tools. They do
> help us, 2.0 and 2.1's quality at the release date was far above
> anything we've done in the past, so I'm fine if we continue to use a
> diversity of tools depending on their capabilities. For sure it's less
> work to always look at a single place but that's no big deal overall.
>
> Willy
>


Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

2020-01-23 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 20:54, Willy Tarreau :

> On Thu, Jan 23, 2020 at 08:40:19PM +0500,  ??? wrote:
> > those timeouts are not related to travis itself, I beleive they are
> mostly
> > related to either real failures or tests instability (race condition).
>
> These tests are racy by nature and some rely on short delays (i.e. health
> checks). If tests are run on an overloaded machine I don't find it
> surprising that a few will fail. Often I click restart and they succeed.
>


we started to observe asan failures

***  h1   debug|AddressSanitizer:DEADLYSIGNAL

***  h1   
debug|=

***  h1   debug|==10330==ERROR: AddressSanitizer: SEGV on unknown
address 0x0018 (pc 0x006e27b1 bp 0x0003 sp
0x7ffe0d8ee080 T0)

***  h1   debug|==10330==The signal is caused by a READ memory access.

***  h1   debug|==10330==Hint: address points to the zero page.

 dT   0.015

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

 dT   0.016

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: failed to fork (errno 11)

***  h1   debug|==10330==WARNING: Failed to use and restart external symbolizer!

***  h1   debug|#0 0x6e27b0
(/home/travis/build/haproxy/haproxy/haproxy+0x6e27b0)

***  h1   debug|#1 0x551799
(/home/travis/build/haproxy/haproxy/haproxy+0x551799)

***  h1   debug|#2 0x8218ed
(/home/travis/build/haproxy/haproxy/haproxy+0x8218ed)

***  h1   debug|#3 0x735000
(/home/travis/build/haproxy/haproxy/haproxy+0x735000)

***  h1   debug|#4 0x7328c8
(/home/travis/build/haproxy/haproxy/haproxy+0x7328c8)

***  h1   debug|#5 0x7ff91cd88b96  (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

***  h1   debug|#6 0x41bd09
(/home/travis/build/haproxy/haproxy/haproxy+0x41bd09)

***  h1   debug|

***  h1   debug|AddressSanitizer can not provide additional info.

***  h1   debug|SUMMARY: AddressSanitizer: SEGV
(/home/travis/build/haproxy/haproxy/haproxy+0x6e27b0)

***  h1   debug|==10330==ABORTING

 dT   0.017

 c1   fd=9 EOF, as expected




I will have a look. Maybe we will switch asan off for good.



> These ones often fail there and only there, so the environment definitely
> has an impact. And seeing the execution time which has become 10-30 times
> what it is on simple laptop really makes me feel like the VMs are seriously
> stressed.
>
> > the bigger is number of tests, the more we depend on those timeouts.
> >
> > however, we can try to run tests in parallel :)
>
> It would be worse!
>
> Willy
>


Re: arm64 builds?

2020-01-23 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 15:14, Willy Tarreau :

> On Thu, Jan 23, 2020 at 01:09:22PM +0500,  ??? wrote:
> > hello,
> >
> > I tried to build using cross compiler (arm32 on amd64).  sorry for
> > screenshot.
> > Willy, do you mean errors like that ?
>
> So for those not seeing the screenshot, it says:
> warning: argument 1 range [2147483648, 4294967295] exceeds maximum object
> size 2147483647 :
>
>  new->idle_orphan_conns = calloc((unsigned int)global.nbthread,
> sizeof(*new->idle_orphan_conns));
>
> Thus the cast to unsigned int was likely placed there to shut it up
> once. Looking at the man page, it says:
>
>void *calloc(size_t nmemb, size_t size);
>
> size_t is unsigned (unless you're using an older gcc < 2.4 on a unusual
> OS). I don't see how it can be said to have a maximum size of 2147483647.
>
> This is exactly the type of stupid warnings that causes us to add casts
> that hide real bugs :-/
>
> Could you please try to use (size_t) instead of (unsigned int) ? If it's
> enough to shut it up, I'm fine with doing that change. Otherwise we'll
> probably get rid of that stupid warning.
>

  CC  src/server.o
  CC  src/cfgparse.o
src/cfgparse.c: In function ‘check_config_validity’:
src/cfgparse.c:3642:33: warning: argument 1 range [2147483648, 4294967295]
exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
 newsrv->idle_orphan_conns = calloc((size_t)global.nbthread,
sizeof(*newsrv->idle_orphan_conns));

 ^~~
In file included from src/cfgparse.c:24:
/usr/arm-linux-gnueabihf/include/stdlib.h:541:14: note: in a call to
allocation function ‘calloc’ declared here
 extern void *calloc (size_t __nmemb, size_t __size)
  ^~
  CC  src/checks.o
^CMakefile:846: recipe for target 'src/checks.o' failed
make: *** [src/checks.o] Interrupt



>
> Thanks!
> Willy
>


Re: Disabling regtests in Travis ?

2020-01-24 Thread Илья Шипицин
пт, 24 янв. 2020 г. в 20:34, Willy Tarreau :

> On Fri, Jan 24, 2020 at 04:31:07PM +0100, Willy Tarreau wrote:
> > So I'm proposing that we try a last time to run with
> > "REGTESTS_TYPES=default,bug,devel"
>
> That should probably give this :
>
> diff --git a/.travis.yml b/.travis.yml
> index bf4b82aa98..a82c27327d 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -112,7 +112,7 @@ script:
>- ./haproxy -vv
>- if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
>- if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
> -  - env VTEST_PROGRAM=../vtest/vtest make reg-tests
> +  - make reg-tests VTEST_PROGRAM=../vtest/vtest
> REGTESTS_TYPES=default,bug,devel
>

let us try that.

I will have a look at "racy" tests.
Maybe we'll enable them on Github Actions.



>
>  after_failure:
>- |
>


Re: Disabling regtests in Travis ?

2020-01-24 Thread Илья Шипицин
пт, 24 янв. 2020 г. в 20:44, Илья Шипицин :

>
>
> пт, 24 янв. 2020 г. в 20:34, Willy Tarreau :
>
>> On Fri, Jan 24, 2020 at 04:31:07PM +0100, Willy Tarreau wrote:
>> > So I'm proposing that we try a last time to run with
>> > "REGTESTS_TYPES=default,bug,devel"
>>
>> That should probably give this :
>>
>> diff --git a/.travis.yml b/.travis.yml
>> index bf4b82aa98..a82c27327d 100644
>> --- a/.travis.yml
>> +++ b/.travis.yml
>> @@ -112,7 +112,7 @@ script:
>>- ./haproxy -vv
>>- if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
>>- if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
>> -  - env VTEST_PROGRAM=../vtest/vtest make reg-tests
>> +  - make reg-tests VTEST_PROGRAM=../vtest/vtest
>> REGTESTS_TYPES=default,bug,devel
>>
>
> let us try that.
>
> I will have a look at "racy" tests.
> Maybe we'll enable them on Github Actions.
>
>
the good thing about Github Actions, it is possible to attach own build
agents. So, if we
have dedicated hardware and we not want to depend on travis-ci neighbours,
it might be an option.


>
>
>>
>>  after_failure:
>>- |
>>
>


Re: arm64 builds?

2020-01-23 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 23:17, Willy Tarreau :

> On Thu, Jan 23, 2020 at 10:59:00PM +0500,  ??? wrote:
> > > Could you please try to use (size_t) instead of (unsigned int) ? If
> it's
> > > enough to shut it up, I'm fine with doing that change. Otherwise we'll
> > > probably get rid of that stupid warning.
> > >
> >
> >   CC  src/server.o
> >   CC  src/cfgparse.o
> > src/cfgparse.c: In function 'check_config_validity':
> > src/cfgparse.c:3642:33: warning: argument 1 range [2147483648,
> 4294967295]
> > exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]
> >  newsrv->idle_orphan_conns = calloc((size_t)global.nbthread,
> > sizeof(*newsrv->idle_orphan_conns));
> >
> >  ^~~
> > In file included from src/cfgparse.c:24:
> > /usr/arm-linux-gnueabihf/include/stdlib.h:541:14: note: in a call to
> > allocation function 'calloc' declared here
> >  extern void *calloc (size_t __nmemb, size_t __size)
> >   ^~
> >   CC  src/checks.o
> > ^CMakefile:846: recipe for target 'src/checks.o' failed
> > make: *** [src/checks.o] Interrupt
>
> Thanks for the test! So basically this clearly proves we respect the
> calling convention but the compiler still complains. OK I'm seeing in
> the mad that it's for functions "decorated" with the "alloc_size"
> attribute. Thus in short they enforce constraints that cannot be
> expressed with available types. This is becoming totally ridiculous.
>
> We're getting a huge collection of stupid warnings to shut up. I
> suspect that the sum of all the code written to shut them is larger
> than what haproxy 1.0 used to be :-/
>
> The man page says we can disable this crap using
> -Walloc-size-larger-than=SIZE_MAX but on my compiler (8.2) it simply
> fails to build. However when passing explicit values not even that
> large, I don't get any such warning, so I'm starting to think that
> it's a real bug of GCC 9.2, because quite frankly, aside calling
> calloc() with a char or short in argument, there's almost no way out
> of this absurdity.
>
> For me, calling it with -Walloc-size-larger-than=-1 makes it stay
> silent. is it also the case for you ? You can patch your Makefile this
> way:
>
> diff --git a/Makefile b/Makefile
> index 8399f6ca35..4757bc77e6 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -199,6 +199,7 @@ SPEC_CFLAGS += $(call cc-opt,-Wshift-negative-value)
>  SPEC_CFLAGS += $(call cc-opt,-Wshift-overflow=2)
>  SPEC_CFLAGS += $(call cc-opt,-Wduplicated-cond)
>  SPEC_CFLAGS += $(call cc-opt,-Wnull-dereference)
> +SPEC_CFLAGS += $(call cc-opt,-Walloc-size-larger-than=-1)
>


  CC  src/cfgparse.o
src/cfgparse.c: In function ‘check_config_validity’:
src/cfgparse.c:3642:33: warning: product ‘2147483648 * 8’ of arguments 1
and 2 exceeds ‘SIZE_MAX’ [-Walloc-size-larger-than=]
 newsrv->idle_orphan_conns = calloc((unsigned int)global.nbthread,
sizeof(*newsrv->idle_orphan_conns));

 ^
src/cfgparse.c:3642:33: note: argument 1 in the range [2147483648,
4294967295]
In file included from src/cfgparse.c:24:
/usr/arm-linux-gnueabihf/include/stdlib.h:541:14: note: in a call to
allocation function ‘calloc’ declared here
 extern void *calloc (size_t __nmemb, size_t __size)
  ^~
  CC  src/checks.o
  CC  src/backend.o




why is it complaining about "product ‘2147483648 * 8" ?




>
>   Memory usage tuning
>  # If small memory footprint is required, you can reduce the buffer size.
> There
>
> If it still fails, we'll have to ignore it and wait for this monstrosity
> to be fixed by their authors :-/
>
> Willy
>


Re: arm64 builds?

2020-01-23 Thread Илья Шипицин
пт, 24 янв. 2020 г. в 00:54, Willy Tarreau :

> On Fri, Jan 24, 2020 at 12:46:12AM +0500,  ??? wrote:
> > > diff --git a/Makefile b/Makefile
> > > index 8399f6ca35..4757bc77e6 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -199,6 +199,7 @@ SPEC_CFLAGS += $(call
> cc-opt,-Wshift-negative-value)
> > >  SPEC_CFLAGS += $(call cc-opt,-Wshift-overflow=2)
> > >  SPEC_CFLAGS += $(call cc-opt,-Wduplicated-cond)
> > >  SPEC_CFLAGS += $(call cc-opt,-Wnull-dereference)
> > > +SPEC_CFLAGS += $(call cc-opt,-Walloc-size-larger-than=-1)
> > >
> >
> >
> >   CC  src/cfgparse.o
> > src/cfgparse.c: In function 'check_config_validity':
> > src/cfgparse.c:3642:33: warning: product '2147483648 * 8' of arguments 1
> > and 2 exceeds 'SIZE_MAX' [-Walloc-size-larger-than=]
>
> Pfff The only good news is that it takes -1 as SIZE_MAX.
>
> >  newsrv->idle_orphan_conns = calloc((unsigned int)global.nbthread,
> > sizeof(*newsrv->idle_orphan_conns));
> >
> >
> ^
> > src/cfgparse.c:3642:33: note: argument 1 in the range [2147483648,
> > 4294967295]
> (...)
> > why is it complaining about "product '2147483648 * 8" ?
>
> because calloc multiplies the two fields and gcc decided that the largest
> value we could possibly pass to the first one if we were as stupid as it
> is is 2147483648. Interestingly it took the largest negative value turned
> to positive and ignored the positive ones that can be turned to the second
> half that are negative if nbthread was negative.
>
> I really think this test is totally bogus and that there is no way to
> express it correctly. I mean, gcc only lets us use 8, 16, 32 or 64 bits.
> If you need to calloc a few megabytes, you'll be forced to apply a mask
> to the value just to shut it up, and *create* the overflow problem yourself
> when it didn't exist.
>
> Let's give up on this one if it doesn't cause too much trouble to you.
> Otherwise we might cheat doing this :
>
> calloc((unsigned short)global.nbthread, ...)
>
> But I really despise this given that we have to make the code wrong just
> to please this shitty compiler.
>


it was ubuntu 18.04 + gcc8, I'll try 19.10 + gcc9


>
> Willy
>


Re: Disabling regtests in Travis ?

2020-01-28 Thread Илья Шипицин
вт, 28 янв. 2020 г. в 16:02, Willy Tarreau :

> On Tue, Jan 28, 2020 at 02:13:15PM +0500,  ??? wrote:
> > btw, we can remove "allowed failure" since this test is slow and excluded
> >
> > https://travis-ci.com/haproxy/haproxy/jobs/280882138
>
> My understanding of the commit message was that it was excluded because
> of issue #429. Am I wrong ?
>

it was excluded 2 times.

first, we marked openssl-1.0.2 as "allowed failure" because of #429
second, it was excluded, because it is "slow" test


>
> Willy
>


Re: Disabling regtests in Travis ?

2020-01-28 Thread Илья Шипицин
вт, 28 янв. 2020 г. в 18:15, Willy Tarreau :

> On Tue, Jan 28, 2020 at 06:02:16PM +0500,  ??? wrote:
> > ??, 28 ???. 2020 ?. ? 16:02, Willy Tarreau :
> >
> > > On Tue, Jan 28, 2020 at 02:13:15PM +0500,  ??? wrote:
> > > > btw, we can remove "allowed failure" since this test is slow and
> excluded
> > > >
> > > > https://travis-ci.com/haproxy/haproxy/jobs/280882138
> > >
> > > My understanding of the commit message was that it was excluded because
> > > of issue #429. Am I wrong ?
> > >
> >
> > it was excluded 2 times.
> >
> > first, we marked openssl-1.0.2 as "allowed failure" because of #429
> > second, it was excluded, because it is "slow" test
>
> Hmmm no, we instead disabled slow regtests, but I'm not seeing anything
>

yep.

Skip reg-tests/ssl/set_ssl_cert.vtc because its type 'slow' is excluded



> specific there under the allow_failures section :
>
>   allow_failures:
>   - os: linux
> arch: ppc64le
> if: type == cron
> compiler: gcc
> env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
>
> So my understanding is that it's still valid due to the error on
> "set ssl cert".
>
> Or maybe you mean something different ?
>
> Willy
>


Lua detection on aarch64

2020-01-29 Thread Илья Шипицин
Hello,

I started to work on rpm packages.
I also applied at Linaro for arm64 vm (request is not completed yet).


interesting that Lua is not detected:

https://copr-be.cloud.fedoraproject.org/results/chipitsine/haproxy-rpm/fedora-31-aarch64/01207488-haproxy/builder-live.log.gz

Martin, do you have arm64, can you check that ?

cheers,
Ilya Shipitcin


Re: [PATCH] CLEANUP: Remove unused static function `free_dcache`

2020-02-05 Thread Илья Шипицин
it is the last warning in travis-ci. we may consider adding ERR=1

ср, 5 февр. 2020 г. в 21:22, Tim Duesterhus :

> The function was change to be static in commit
> 6c39198b57566ddfc5368e481d2a7cf8111d, but even that commit
> no longer uses it. The purpose of the change vs. outright removal
> is unclear.
>
> see issue #301
> ---
>  src/peers.c | 12 
>  1 file changed, 12 deletions(-)
>
> diff --git a/src/peers.c b/src/peers.c
> index e2581a73b..52e2a8ba4 100644
> --- a/src/peers.c
> +++ b/src/peers.c
> @@ -2889,18 +2889,6 @@ static struct dcache *new_dcache(size_t max_entries)
> return NULL;
>  }
>
> -/*
> - * Deallocate a cache of dictionary entries.
> - */
> -static inline void free_dcache(struct dcache *dc)
> -{
> -   free_dcache_tx(dc->tx);
> -   dc->tx = NULL;
> -   free(dc->rx); dc->rx = NULL;
> -   free(dc);
> -}
> -
> -
>  /*
>   * Look for the dictionary entry with the value of  in  cache of
> dictionary
>   * entries used upon transmission.
> --
> 2.25.0
>
>


Re: [PATCH] compression/lua_validation.vtc depends on "which" utility

2020-02-18 Thread Илья Шипицин
when specifying "shell" in vtc, can we require posix /bin/sh ? or is it by
chance ?

shell {
HOST=${h1_fe1_addr}
if [ "${h1_fe1_addr}" = "::1" ] ; then
HOST="\[::1\]"
fi

md5=$(which md5 || which md5sum)


вт, 18 февр. 2020 г. в 18:24, Willy Tarreau :

> On Tue, Feb 18, 2020 at 02:15:18PM +0100, Tim Düsterhus wrote:
> > Willy,
> >
> > Am 18.02.20 um 14:06 schrieb Willy Tarreau:
> > > I haven't pushed yet Ilya's patch I've just merged, I'm fine with
> > > applying a change if preferred. Just let me know.
> >
> > `type` is required to be a builtin by POSIX [1]. The `-p` parameter does
> > not appear to be standardized, though. Nonetheless I expect it to work
> > in any `bash`.
>
> OK thanks for the info. But sadly it fails in other shells such as dash:
>
>   $ type -p curl
>   -p: not found
>   curl is /usr/local/bin/curl
>
> So we can only use "type" with no argument and adjust it. Or better,
> we can use "command -v" which looks OK:
>
>   bash$ command -v curl
>   /usr/local/bin/curl
>   dash$ command -v curl
>   /usr/local/bin/curl
>   ksh$ command -v curl
>   /usr/local/bin/curl
>
> And it's also in POSIX:
>https://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html
>
> Willy
>


[PATCH[ partially enable s390x builds in travis-ci

2020-02-19 Thread Илья Шипицин
Hello,


I enabled s390x builds except reg-tests/seamless-reload/abns_socket.vtc

Cheers,
Ilya Shipitcin
From 8924057b8a25cf8f6595929f4b18fec1a85a10c5 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 19 Feb 2020 23:47:56 +0500
Subject: [PATCH] BUILD: travis-ci: enable s390x builds

reg-tests/seamless-reload/abns_socket.vtc is skipped due to #504
---
 .travis.yml | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index fd136c980..d263cf75a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -47,6 +47,13 @@ matrix:
 if: type != cron
 compiler: clang
 env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
+  - os: linux
+arch: s390x
+if: type != cron
+compiler: gcc
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
+before_script:
+  - rm reg-tests/seamless-reload/abns_socket.vtc # please, see https://github.com/haproxy/haproxy/issues/504
   - os: linux
 if: type == cron
 compiler: clang
-- 
2.24.1



Re: haproxy 2.1.2 and 2.1.3

2020-02-21 Thread Илья Шипицин
useful things to debug would be

1) haorpxy -vv
2) config
3) bt full


can you please open a github issue at
https://github.com/haproxy/haproxy/issues  ?

пт, 21 февр. 2020 г. в 13:55, :

> Hi all,
>
>
>
> At this moment we’re using  HA-Proxy version 2.0.0 2019/06/16 -
> https://haproxy.org/
>
> And we’re running solaris 11.4 sru 950 for SPARC and I used developer
> studio version 12.6 to compile the newer versions and 12.5 to compile the
> 2.0.0 version.
>
>
>
> And because of a known bug in haproxy, we wanted to upgrade to: HA-Proxy
> version 2.1.2 2019/12/21 - https://haproxy.org/
>
> On our test-server we had no problems but on the production server it kept
> crashing.
>
> The problem is somehow related to ssl, because as soon as we add “ssl crt
> /etc/certs/self.pem” to the bind it crashes also on the test-server.
>
> So then I checked and version 2.1.3 is already available, solving lots of
> ssl bugs. So I tried that one also…but without any luck ☹
>
>
>
> Starting haproxy now gives a Bus Error (with both versons, 2.1.2 and
> 2.1.3):
>
>
>
> root@sz4203:~ # /usr/local/sbin/haproxy -p /tmp/haproxy.pid -f
> /usr/local/etc/haproxy.cfg
>
> Bus Error (core dumped)
>
>
>
> Can you please help me solving this issue ?
>
>
>
> This is the stack-dump of the 2.1.3 version I compiled:
>
> root@s00501:/var/core# pstack core_sz4203_haproxy_0_0_1582273479_18461
>
> core 'core_sz4203_haproxy_0_0_1582273479_18461' of 18461:
> /usr/local/sbin/haproxy -p /tmp/haproxy.pid -f /usr/local/etc/haproxy.
>
> 000100240c30 shctx_init (100366170, 4e20, c4, ,
> 7ec00144, 0) + b0 (shctx.c:368)
>
> 000100086e7c ssl_sock_prepare_bind_conf (1005bbb80, ,
> fffefffc, 29729948, , 100366000) + 1bc
>
> 000100135278 check_config_validity (fffefffc, 1400, 1005bbb80,
> 1005bb750, 100357238, 0) + 25b8 (cfgparse.c:3753)
>
> 000100173450 init (100357468, 1000596a0, 10005a940, 100357238,
> 10003aa90, 0) + 610 (haproxy.c:1884)
>
> 000100178058 main (5, 7688, 76b8, 10,
> 10046d680, 0) + 178 (haproxy.c:2858)
>
> 00010006caa4 _start (0, 0, 0, 0, 0, 0) + 64
>
>
>
> And this is how I compiled the 2.1.3 version:
>
>
>
> PATH=/usr/gnu/bin:${PATH}:/opt/developerstudio12.6/bin/
>
> cd ~bhr_kloos605/haproxy-2.1.3/
>
> vi Makefile
>
> Just to make sure that I can compile…changes:
>
> root@sz4203:/export/home/bhr_kloos605# diff haproxy-2.1.3/Makefile
> haproxy-2.1.3.compiled/Makefile
>
> 137c137
>
> < DESTDIR =
>
> ---
>
> > DESTDIR = /tmp/haproxy
>
> 662c662,663
>
> < OPTIONS_LDFLAGS += $(if $(PCRE_LIB),-L$(PCRE_LIB)) -Wl,-Bstatic
> -lpcreposix -lpcre -Wl,-Bdynamic
>
> ---
>
> > #OPTIONS_LDFLAGS += $(if $(PCRE_LIB),-L$(PCRE_LIB)) -Wl,-Bstatic
> -lpcreposix -lpcre -Wl,-Bdynamic
>
> > OPTIONS_LDFLAGS += $(if $(PCRE_LIB),-L$(PCRE_LIB)) -Bstatic -lpcreposix
> -lpcre
>
> make TARGET=solaris CPU=ultrasparc PCRE_LIB=/usr/lib
> PCRE_INC=/usr/include/pcre USE_STATIC_PCRE=1 USE_OPENSSL=1
>
> make install
>
>
>
> Can you please help ?
>
>
>
>
>
> Met vriendelijke groet/Kind regards,
>
>
>
> *Marcel Kloosterman*
>
> *Specialist Technisch Beheer*
>
>
>
>
>
> *KPN*
>
> N ACN P DC Cloud 1
>
> Phone: +31612413284
>
> Email: marcel.klooster...@kpn.com
>
> TeamWiki (with spoc planning):
> https://confluence.kpn.org/display/KDU/KPN+DCLOUD+SOLARIS+CLOUD+TEAM+HOMEPAGE
>
> (JIRA access is required and can be requested using IAM portal)
>
> *BTW nr.: NL0092.92.056B01*
>
> *KvK: 27124701*
>
>
>
> *The information transmitted is intended only for use by the addressee and
> may contain confidential and/or privileged material. Any review,
> re-transmission, dissemination or other use of it, or the taking of any
> action in reliance upon this information by persons and/or entities other
> than the intended recipient is prohibited. If you received this in error,
> please inform the sender and/or addressee immediately and delete the
> material. Thank you.*
>
>
>


[PATCH] travis-ci: remove "allow failures", add ERR=1, modernize build-ssl.sh script

2020-02-11 Thread Илья Шипицин
Hello,

I split patch into separate patches.

cheers,
Ilya Shipitcin
From 17b0b5fae3afe6df42e938073ccf8d7a2105fbb7 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 11 Feb 2020 14:29:41 +0500
Subject: [PATCH 1/3] BUILD: travis-ci: no more allowed failures for
 openssl-1.0.2

since 23fb037 (which fixes #429) no need to allow failures
on openssl-1.0.2
---
 .travis.yml | 6 --
 1 file changed, 6 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index a82c27327..1cce8db88 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -92,12 +92,6 @@ matrix:
 before_script:
   - git clone http://git.1wt.eu/git/libslz.git/
   - cd libslz && make && make PREFIX=${HOME}/opt install && cd ..
-  allow_failures:
-  - os: linux
-arch: ppc64le
-if: type == cron
-compiler: gcc
-env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
 
 install:
   - git clone https://github.com/VTest/VTest.git ../vtest
-- 
2.24.1

From 80803044ad103fe3815f88eb4174c55914b5ac71 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 11 Feb 2020 14:31:13 +0500
Subject: [PATCH 2/3] BUILD: travis-ci: harden builds, add ERR=1 (warning ought
 to be errors)

---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 1cce8db88..fd136c980 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -102,7 +102,7 @@ install:
 script:
   - if [ "$CC"  = "clang" ]; then export FLAGS="$FLAGS USE_OBSOLETE_LINKER=1" DEBUG_CFLAGS="-g -fsanitize=address" LDFLAGS="-fsanitize=address"; fi
   - make -C contrib/wurfl
-  - make -j3 CC=$CC V=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS -L$SSL_LIB -Wl,-rpath,$SSL_LIB" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
+  - make -j3 CC=$CC V=1 ERR=1 TARGET=$TARGET $FLAGS DEBUG_CFLAGS="$DEBUG_CFLAGS" LDFLAGS="$LDFLAGS -L$SSL_LIB -Wl,-rpath,$SSL_LIB" 51DEGREES_SRC="$FIFTYONEDEGREES_SRC" EXTRA_OBJS="$EXTRA_OBJS"
   - ./haproxy -vv
   - if [ "${TRAVIS_OS_NAME}" = "linux" ]; then ldd haproxy; fi
   - if [ "${TRAVIS_OS_NAME}" = "osx" ]; then otool -L haproxy; fi
-- 
2.24.1

From f850c0b44c52681ea94f0e42da2f0acf9baa3826 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 11 Feb 2020 14:36:23 +0500
Subject: [PATCH 3/3] BUILD: scripts/build-ssl.sh: use "uname" instead of
 ${TRAVIS_OS_NAME}

it is also useful for local builds, when ${TRAVIS_OS_NAME} is not set
---
 scripts/build-ssl.sh | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/scripts/build-ssl.sh b/scripts/build-ssl.sh
index 384438a22..60b876c5d 100755
--- a/scripts/build-ssl.sh
+++ b/scripts/build-ssl.sh
@@ -28,11 +28,14 @@ build_openssl_osx () {
 build_openssl () {
 if [ "$(cat ${HOME}/opt/.openssl-version)" != "${OPENSSL_VERSION}" ]; then
 tar zxf "download-cache/openssl-${OPENSSL_VERSION}.tar.gz"
-if [ "${TRAVIS_OS_NAME}" = "osx" ]; then
-build_openssl_osx
-elif [ "${TRAVIS_OS_NAME}" = "linux" ]; then
-build_openssl_linux
-fi
+	case `uname` in
+		'Darwin')
+			build_openssl_osx
+			;;
+		'Linux')
+			build_openssl_linux
+			;;
+	esac
 echo "${OPENSSL_VERSION}" > "${HOME}/opt/.openssl-version"
 fi
 }
-- 
2.24.1



Re: [PATCH] add workaround to cirrus-ci

2020-02-11 Thread Илья Шипицин
Hello,

here's patch split into 3 patches.

вт, 11 февр. 2020 г. в 12:17, Willy Tarreau :

> On Tue, Feb 11, 2020 at 12:14:47PM +0500,  ??? wrote:
> > I will send patch series soon.
> Thank you!
> willy
>
From e47087ecaea0aa32475593afbf460dc0cf72c1f9 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 11 Feb 2020 13:20:32 +0500
Subject: [PATCH 3/3] cirrus-ci: add ERR=1 to freebsd builds

---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 5c7c8403d..1a07c80c7 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -10,7 +10,7 @@ FreeBSD_task:
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- gmake CC=clang V=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
+- gmake CC=clang V=1 ERR=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
 - ./haproxy -vv
 - ldd haproxy
 - env VTEST_PROGRAM=../vtest/vtest gmake reg-tests || (for folder in /tmp/*regtest*/vtc.*; do cat $folder/INFO $folder/LOG; done && exit 1)
-- 
2.24.1

From 6d44d41d01fe150316b9ae9e7ba296464ad45393 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 11 Feb 2020 13:19:22 +0500
Subject: [PATCH 2/3] cirrus-ci: workaround "pkg install" bug

there's a bug https://github.com/freebsd/pkg/issues/902
adding "pkg update -f && pkg upgrade -y" is workaround.
---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index d52808358..5c7c8403d 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -6,7 +6,7 @@ FreeBSD_task:
   image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- pkg install -y openssl git gmake lua53 socat
+- pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- 
2.24.1

From 84a013d8f6c1dfe4977e2d5c21887c4ede3230ae Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 11 Feb 2020 13:16:02 +0500
Subject: [PATCH 1/3] cirrus-ci: switch to "snap" images to unify openssl
 naming

"snap" images are updated frequently, while regular images are updated quarterly.
so, mixing "snap" and regular images lead to package naming mismatch, which will occur every
quarter. we cannot use 11.3 release image, it is broken, so we have to use 11.3 "snap" image.
Thus let us use all "snap" images. 13-snap is first introduced with this commit.
---
 .cirrus.yml | 11 +++
 1 file changed, 3 insertions(+), 8 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 4143e687b..d52808358 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,17 +1,12 @@
 FreeBSD_task:
   freebsd_instance:
 matrix:
-  image_family: freebsd-12-1
+  image_family: freebsd-13-0-snap
+  image_family: freebsd-12-1-snap
   image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- case `uname -r` in
-11.3*)
-  export SSL=openssl;;
-12.1*)
-  export SSL=openssl111;;
-  esac
-- pkg install -y $SSL git gmake lua53 socat
+- pkg install -y openssl git gmake lua53 socat
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- 
2.24.1



Re: [PATCH] AIX 7.2 support

2020-02-11 Thread Илья Шипицин
there's AIX related build guide

https://github.com/haproxy/haproxy/blob/master/INSTALL#L512-L517

maybe it should be updated as well

пн, 10 февр. 2020 г. в 16:45, Chris :

> Hi Willy,
>
> thanks a lot for your input and sorry for the delay. Work is pretty
> rough right now...
>
> > Thank you for doing this work. I'm having a few questions below.
> Sure, I am glad I can help :).
>
> > > +# AIX 7.2 and above
> > > +ifeq ($(TARGET),aix72-gcc)
> > > +  set_target_defaults = $(call default_opts, \
> > > +USE_POLL USE_THREAD USE_LIBCRYPT USE_OBSOLETE_LINKER
> > > USE_GETADDRINFO USE_TFO)
> >
> > Are you really really sure about USE_TFO ? I suspect you might have
> > accidently borrowed it from another line. It stands for TCP FastOpen
> > and is really not broadly deployed, I was only aware of Linux and
> > FreeBSD, but there are likely a few other ones. And the fact that it
> > looks OK is possibly just that it's causing a different TCP socket
> > option to be set on the connection, so if you're not certain we'd
> > rather avoid it.
> I think you are right. I kept USE_TFO in there because it neither
> introduced any build-issues nor did it cause any runtime issues.
> However, it probably does nothing and should be removed as long as we
> are not certain it is actually beneficial. I will have a look at the
> corresponding AIX internals later at some point.
>
> > > src/hlua.c: In function 'hlua_panic_ljmp':
> > > src/hlua.c:128:1: warning: no return statement in function returning
> > > non-void [-Wreturn-type]
> > >  static int hlua_panic_ljmp(lua_State *L) { longjmp(safe_ljmp_env, 1);
> }
> > >  ^~
> >
> > Don't worry about this one, I'll handle it. I suspect that on linux
> > platforms the longjmp() function prototype is decorated with
> > __attribute__((noreturn)) which makes the compiler happy, but that's
> > likely not the case on any system not relying on a gcc-compatible
> > compiler by default.
> Thanks for handling this one. If there is something I shoult test
> please let me know.
>
> > Just two small extra requests :
> >   - please rebase it on top of the development branch. If you really
> > need it in 2.1, just indicate it. Given that it's well isolated,
> > I'm fine with having it backported.
> >
> >   - please have a look at CONTRIBUTING to get guidance to write a
> > subject and a commit message. It doesn't need to be very long
> > but at least indicating what options you chosed to enable/disable
> > and on what system you tested it will be enough if we need to
> > reconsider parts of it later.
> I already made a new patch based on the current development-HEAD and
> tried to follow all the guidelines from CONTRIBUTING. I would love to
> have it backported to 2.1 as the patch is pretty much identical and
> should not cause any regressions. The patch will follow shortly!
>
> > Do you have a permanent and durable access to this machine, with the
> > ability to occasionally re-run a build test in case we ask you (likely
> > no more than 2-3 times a year in the worst case) ? I'm asking because
> > I'm still keeping a very old IBM server running 5.2 on a Power3 just
> > for the sake of revalidating new releases once in a while. Given that
> > I could not upgrade it to latest OpenSSL, it cannot even be used to
> > provide complete binaries to those needing them, so it's getting very
> > obsolete and knowing that there's a better solution somewhere would
> > allow me to get rid of it.
> Well, I am not the primary administrator of this machine. However, I
> have permanent access and no problem doing the occasional build on it.
> I will have a chat with the server admin in regards to the longtime
> prospects of this server and report back to you.
>
> > thanks!
> > Willy
> You are welcome - I am happy I can contribute!
>
> thanks,
> Christian
>
> On Thu, Feb 6, 2020 at 3:36 PM Willy Tarreau  wrote:
> >
> > Hello Christian,
> >
> > On Mon, Feb 03, 2020 at 12:09:46PM +0100, Chris wrote:
> > > Hello everybody,
> > >
> > > I spent some time making haproxy compile and run successfully on AIX
> > > 7.2 using GCC 8.3 and wanted to contribute my patch in the hope that
> > > it could be merged. The patch is based on the current haproxy 2.1 head
> > > revision. I can make one for the development branch too - but it
> > > should be basically identical.
> >
> > Thank you for doing this work. I'm having a few questions below.
> >
> > > +# AIX 7.2 and above
> > > +ifeq ($(TARGET),aix72-gcc)
> > > +  set_target_defaults = $(call default_opts, \
> > > +USE_POLL USE_THREAD USE_LIBCRYPT USE_OBSOLETE_LINKER
> > > USE_GETADDRINFO USE_TFO)
> >
> > Are you really really sure about USE_TFO ? I suspect you might have
> > accidently borrowed it from another line. It stands for TCP FastOpen
> > and is really not broadly deployed, I was only aware of Linux and
> > FreeBSD, but there are likely a few other ones. And the fact that it
> > looks OK is possibly just that it's causing a different TCP 

Re: [PATCH] compression/lua_validation.vtc depends on "which" utility

2020-02-16 Thread Илья Шипицин
вс, 16 февр. 2020 г. в 01:55, Tim Düsterhus :

> Ilya,
>
> Am 15.02.20 um 20:38 schrieb Илья Шипицин:
> > that utility is not available by default in Fedora docker image.
>
> I believe we can rewrite that test to use the `type` builtin instead of
> `which`:
>


is it POSIX ?

I've just checked on freebsd, it's there. What about osx, aix, ... ?


>
> > $ type -p md5
> > $ type -p md5sum
> > /usr/bin/md5sum
> > $ md5=$(type -p md5 || type -p md5sum)
> > $ echo $md5
> > /usr/bin/md5sum
>
> Best regards
> Tim Düsterhus
>


Re: [PATCH 2/2] CLEANUP: namespace: remove uneeded ns check in my_socketat

2020-02-12 Thread Илья Шипицин
чт, 13 февр. 2020 г. в 01:26, William Dauchy :

> we check ns variable but we don't use it later
>

we "use" it.
depending on true/false we either return -1 or not


>
> Signed-off-by: William Dauchy 
> ---
>  src/namespace.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/namespace.c b/src/namespace.c
> index 89a968e36..3536629bc 100644
> --- a/src/namespace.c
> +++ b/src/namespace.c
> @@ -120,7 +120,7 @@ int my_socketat(const struct netns_entry *ns, int
> domain, int type, int protocol
>
> sock = socket(domain, type, protocol);
>
> -   if (default_namespace >= 0 && ns && setns(default_namespace,
> CLONE_NEWNET) == -1) {
> +   if (default_namespace >= 0 && setns(default_namespace,
> CLONE_NEWNET) == -1) {
> if (sock >= 0)
> close(sock);
> return -1;
> --
> 2.25.0
>
>
>


[PATCH] add ERR=1 to cygwin builds

2020-02-15 Thread Илья Шипицин
Hello,

another CI hardening.

Cheers,
Ilya Shipitcin
From 64bf8ef27486feb6e12b183d563b887473afb4cf Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 15 Feb 2020 18:58:22 +0500
Subject: [PATCH 1/2] BUILD: enable ERR=1 in github cygwin builds

---
 .github/workflows/windows-latest.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/windows-latest.yml b/.github/workflows/windows-latest.yml
index 27b16eec6..445e021a4 100644
--- a/.github/workflows/windows-latest.yml
+++ b/.github/workflows/windows-latest.yml
@@ -16,5 +16,5 @@ jobs:
 - name: fake step
   run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd'
 - name: build
-  run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
+  run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin ERR=1 USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
 
-- 
2.24.1



[PATCH] compression/lua_validation.vtc depends on "which" utility

2020-02-15 Thread Илья Шипицин
Hello,


that utility is not available by default in Fedora docker image.


Cheers,
Ilya Shipitcin
From fd8045f2f6a3c455642dc797ae6ac838f66529a6 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 16 Feb 2020 00:34:10 +0500
Subject: [PATCH] REGTESTS: add dependency compression/lua_validation.vtc on
 "which"

when running reg-tests on Fedora docker container, it turned out that
"which" utility is not installed
---
 reg-tests/compression/lua_validation.vtc | 1 +
 1 file changed, 1 insertion(+)

diff --git a/reg-tests/compression/lua_validation.vtc b/reg-tests/compression/lua_validation.vtc
index 6e8c1e879..7d0488624 100644
--- a/reg-tests/compression/lua_validation.vtc
+++ b/reg-tests/compression/lua_validation.vtc
@@ -4,6 +4,7 @@ varnishtest "Compression validation"
 #REQUIRE_VERSION=1.6
 #REQUIRE_OPTIONS=ZLIB|SLZ,LUA
 #REGTEST_TYPE=slow
+#REQUIRE_BINARIES=which
 
 feature ignore_unknown_macro
 
-- 
2.24.1



Re: [PATCH] BUG/MINOR: ssl: Stop passing dynamic strings as format arguments

2020-02-19 Thread Илья Шипицин
It happens because we now run ERR=1 in ci builds

On Wed, Feb 19, 2020, 3:41 PM Tim Duesterhus  wrote:

> gcc complains rightfully:
>
> src/ssl_sock.c: In function ‘ssl_load_global_issuers_from_path’:
> src/ssl_sock.c:9860:4: warning: format not a string literal and no format
> arguments [-Wformat-security]
> ha_warning(warn);
> ^
>
> Introduced in 70df7bf19cebd5593c0abb01923e6c9f72961da6.
> ---
>  src/ssl_sock.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/ssl_sock.c b/src/ssl_sock.c
> index e30bb8a6c..ade5ffc84 100644
> --- a/src/ssl_sock.c
> +++ b/src/ssl_sock.c
> @@ -9857,7 +9857,7 @@ static int ssl_load_global_issuers_from_path(char
> **args, int section_type, stru
> goto next;
> ssl_load_global_issuer_from_BIO(in, fp, );
> if (warn) {
> -   ha_warning(warn);
> +   ha_warning("%s", warn);
> free(warn);
> warn = NULL;
> }
> --
> 2.25.0
>
>


is it allowed to bind both http and https on the same port ?

2020-02-12 Thread Илья Шипицин
hello,
while playing with dataplane api (I copy-pasted code), accidentally I
created the following config

frontend git_example_com_https_frontend
  mode http
  bind 10.216.7.1:7080 name http
  bind 10.216.7.1:7080 name https crt /etc/haproxy/bundle.pem ssl alpn
h2,http/1.1
  default_backend git_example_com_https_backend
  redirect scheme https code 301 if !{ ssl_fc }

both bind lines include the same 7080 port. haproxy does not complain. is
that configuration correct ? should haproxy complain on such config ?

(I did not mean to put that to production, it happened accidentally).

Cheers,
Ilya Shipitcin


Re: how to properly reload haproxy (from systemd + master-worker) ?

2020-02-12 Thread Илья Шипицин
I confirm that adding "no option start-on-reload" resolves the reload issue.

I'll report to dataplane documentation.

вт, 4 февр. 2020 г. в 16:01, William Lallemand :

> On Tue, Feb 04, 2020 at 12:48:18AM +0500, Илья Шипицин wrote:
> > > > вс, 2 февр. 2020 г. в 22:58, Tim Düsterhus :
> > > >
> > > > [...]
> > > > > > Feb 02 20:50:07 xxx systemd[1]: haproxy.service failed.
> > > > >
> > > > > ... leading to the unit being stopped.
> > > > >
> > > > > So you would have to find out with the dataplane API dies. My
> educated
> > > > > guess would be that it fails because the port is already being in
> use
> > > by
> > > > > the old dataplane API process.
> > > > >
> > >
> > > From what I understand of the dataplaneapi, it is reloaded upon a USR1
> > > signal,
> > > signal which is sent by the master process upon a reload.
> > >
> > > So you probably just need to add "no option start-on-reload" in your
> > > program
> > > section to prevent HAProxy to launch the dataplane-api again.
> > >
> >
> >
> > thank you, I'll try
> >
> > maybe it should be added to docs.
> >
>
> Could you open an issue on the dataplaneapi bugtracker? Because I'm not
> sure
> this is the right behavior.
>
> The doc speaks about "no option start-on-reload" but only if used by
> docker.
>
>
> https://www.haproxy.com/documentation/hapee/1-9r1/configuration/dataplaneapi/
>
> --
> William Lallemand
>


[PATCH] CLEANUP: remove unused functions from openssl-compat level

2020-02-13 Thread Илья Шипицин
Hello,

while investigating #429, I've found some unused functions.
let us remove them.

Cheers,
Ilya Shipitcin
From 4b882141a1e8e364aa5339c5090bf40a673876aa Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 13 Feb 2020 14:07:36 +0500
Subject: [PATCH] CLEANUP: include/common/openssl-compat.h: remove unused
 functions

functions SSL_SESSION_get0_id_context, SSL_CTX_get_default_passwd_cb,
SSL_CTX_get_default_passwd_cb_userdata are not used anymore
---
 include/common/openssl-compat.h | 16 
 1 file changed, 16 deletions(-)

diff --git a/include/common/openssl-compat.h b/include/common/openssl-compat.h
index cb9caa3e9..94633aa8c 100644
--- a/include/common/openssl-compat.h
+++ b/include/common/openssl-compat.h
@@ -181,12 +181,6 @@ static inline X509_CRL *X509_OBJECT_get0_X509_CRL(const X509_OBJECT *a)
 return a->data.crl;
 }
 
-static inline const unsigned char *SSL_SESSION_get0_id_context(const SSL_SESSION *sess, unsigned int *sid_ctx_length)
-{
-	*sid_ctx_length = sess->sid_ctx_length;
-	return sess->sid_ctx;
-}
-
 static inline int SSL_SESSION_set1_id(SSL_SESSION *s, const unsigned char *sid, unsigned int sid_len)
 {
 	s->session_id_length = sid_len;
@@ -206,16 +200,6 @@ static inline const OCSP_CERTID *OCSP_SINGLERESP_get0_id(const OCSP_SINGLERESP *
 }
 #endif
 
-static inline pem_password_cb *SSL_CTX_get_default_passwd_cb(SSL_CTX *ctx)
-{
-	return ctx->default_passwd_callback;
-}
-
-static inline void *SSL_CTX_get_default_passwd_cb_userdata(SSL_CTX *ctx)
-{
-	return ctx->default_passwd_callback_userdata;
-}
-
 #ifndef OPENSSL_NO_DH
 static inline int DH_set0_pqg(DH *dh, BIGNUM *p, BIGNUM *q, BIGNUM *g)
 {
-- 
2.24.1



Re: [PATCH] MINOR: ssl: add "issuers-chain-path" directive.

2020-02-18 Thread Илья Шипицин
вт, 18 февр. 2020 г. в 21:44, Emmanuel Hocdet :

>
> > Le 18 févr. 2020 à 14:36, William Lallemand  a
> écrit :
> >
> > On Tue, Feb 18, 2020 at 01:58:39PM +0100, Emmanuel Hocdet wrote:
> >>
> >>> Le 18 févr. 2020 à 11:45, Emmanuel Hocdet  a écrit :
> >>>
>  Can you add a little bit of explanation on how the discovery of the
>  issuer is done in the documentation?
> 
> >>> ok
> >>
> >>
> >> documentation updated:
> >>
> >
> > Thanks Manu!
> >
> > Merged and pushed in master.
> >
>

src/ssl_sock.c:9860:15: error: format string is not a string literal
(potentially insecure) [-Werror,-Wformat-security]

ha_warning(warn);

   ^~~~

src/ssl_sock.c:9860:15: note: treat the string as an argument to avoid this

ha_warning(warn);

   ^

   "%s",



>
> w00t!
> Thanks
>
> Manu
>
>
>


Re: [PATCH] CLEANUP: server: remove unused err section in server_finalize_init

2020-01-08 Thread Илья Шипицин
btw, if you add "Fixes: #438", the issue will be closed automatically

https://help.github.com/en/github/managing-your-work-on-github/closing-issues-using-keywords

чт, 9 янв. 2020 г. в 01:33, William Dauchy :

> Since commit 980855bd953c ("BUG/MEDIUM: server: initialize the orphaned
> conns lists and tasks at the end"), we no longer use err section.
>
> This should fix github issue #438
>
> Signed-off-by: William Dauchy 
> ---
>  src/server.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/src/server.c b/src/server.c
> index 6212a420..14ff716a 100644
> --- a/src/server.c
> +++ b/src/server.c
> @@ -2053,8 +2053,6 @@ static int server_finalize_init(const char *file,
> int linenum, char **args, int
> srv_lb_commit_status(srv);
>
> return 0;
> -err:
> -   return ERR_ALERT | ERR_FATAL;
>  }
>
>  /*
> --
> 2.24.1
>
>
>


Re: ARM(64) builds

2020-01-16 Thread Илья Шипицин
Hello, Martin!

btw, just curious, how is Apache Foundation (or you personally) interested
in all that ?
please do not blame me, I really like to know.


чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :

> Hello HAProxy developers,
>
> At work we are going to use more and more ARM64 based servers and HAProxy
> is one of the main products we rely on.
> So I went checking whether HAProxy project has a CI environment for
> testing on ARM architecture.
>

we are looking towards
https://docs.travis-ci.com/user/multi-cpu-architectures


> I've found this recent discussion:
> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
> didn't find a way how to continue on the same mail thread, so I'm starting
> a new one. Apologies for that!).
>

I played with arm64 for a while, the issue I encountered is travis-ci cache
key, i.e. we cache openssl builds between our builds.
so travis used the same cache key for both amd64 and arm64 builds (this
might have changed recently, I did not check yet)

arm64 is in my queue (as well as recent s390x arch from travis), hope to
get back to it within month or so.


> From this discussion and from
> https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand
> that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of
> the developers run some tests locally regularly.
>

it is not completely true.
there's public CI. we do not use github PR machinery, so sometimes tests
fail after push to master branch. it is considered as ok, failures are
fixed pretty fast.
for example, see
https://www.mail-archive.com/haproxy@formilux.org/msg35910.html
it was just perfect, failure detected using CI and fixed within few days.
no customers affected.


> I've forked the project and tested on TravisCI (
> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the
> builds were not very stable:
> 1) some tests fail sometimes. I guess it is because of some timing issues
> For example:
> - https://travis-ci.org/martin-g/haproxy/jobs/636745241
> - https://travis-ci.org/martin-g/haproxy/jobs/636750676
> - https://travis-ci.org/martin-g/haproxy/jobs/636763346
>

that's very interesting. I'll have a look.



> 2) There was some weird issue on testing with LibreSSL
> The output redirect at
> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96
>  for
> some reason got stuck the build. I've removed temporarily the output
> redirects and then it passed. So, it looks like some issue with TravisCI
> environment.
>

arm64 is slower, I guess we should add "*travis*_*wait* *30" *to
build-ssl.sh script
thank for the hint


>
> In addition I've run the build and tests on one of our machines and all
> was OK!
>
> My question to you is: Are you happy with your current way of testing ARM
> architectures or you want to add more ?
> Here are some options:
> 1) enable TravisCI
>

already done

https://travis-ci.com/haproxy/haproxy


> 2) my company is willing to donate an ARM64 based VM, if you are
> interested.
>

I do not work at Haproxy Inc :)
Willy ?


> You will have a SSH access and a user with sudo permissions to install
> anything that is needed.
> The spec is aarch64 8 cores CPU 2GHz (Kunpeng), 16GB RAM, 500G disk space
> and 5M network bandwidth. The OS could be any of CentOS 7.4/7.5/7.6,
> EulerOS 2.8, Fedora 29, OpenSuse 15.0 and Ubuntu 16.04/18.04.
>
> In both cases it will be ARM64. From the earlier mail discussion I
> understand you would prefer ARM32.
>

as for myself, I prefer both arm64 and arm32.
however, both AMD64 and ARM64 are the same 64 bits. both of them are
little-endian. but you mentioned at least 3 builds with ARM64 failing (we
have those tests passing on AMD64)!



>
> Kind regards,
> Martin
>


Re: ARM(64) builds

2020-01-16 Thread Илья Шипицин
чт, 16 янв. 2020 г. в 13:26, Martin Grigorov :

> Hi Илья,
>
> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
> wrote:
>
>>
>> Hello, Martin!
>>
>> btw, just curious, how is Apache Foundation (or you personally)
>> interested in all that ?
>> please do not blame me, I really like to know.
>>
>
ok, so you work in some company that is interested in haproxy on ARM64.
you do not want to tell it's name, at least is it legal ? is it related to
some government ?
if "no" and "no", I guess most people won't ask any more questions :)

personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
contribute to it.
Please do not consider me as an "official representative".


I'm interested in testing haproxy on ARM64, I planned to do so. I can
priorierize it according to your interest to it.
and yes, I can accept hardware donation (even considering I'm not part of
Haproxy Inc).

also, from my point of view, what would be really useful in your case is
testing not just official reg-tests, but also
your configs. reg-tests cover only partially. If you enable clang asan in
your own workload there's a chance to catch
something interesting (or, to make sure your own workload is ok). I can
help with that as well.


>
>>
> @apache.org is just one of my several emails. And it is the default one
> in my email client.
> ASF is not related anyhow to my participation here.
> If I used my work email then it might look like some kind of
> advertisement. I'd like to avoid that!
> Next time I will use my @gmail.com one, as more neutral. Actually I've
> used the GMail one when registering to this mailing list, so probably the
> post from @apache has been moderated. I'll be more careful next time!
>
> Thanks, Илья!
>
> Regards,
> Martin
>
>
>>
>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :
>>
>>> Hello HAProxy developers,
>>>
>>> At work we are going to use more and more ARM64 based servers and
>>> HAProxy is one of the main products we rely on.
>>> So I went checking whether HAProxy project has a CI environment for
>>> testing on ARM architecture.
>>>
>>
>> we are looking towards
>> https://docs.travis-ci.com/user/multi-cpu-architectures
>>
>>
>>> I've found this recent discussion:
>>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
>>> didn't find a way how to continue on the same mail thread, so I'm starting
>>> a new one. Apologies for that!).
>>>
>>
>> I played with arm64 for a while, the issue I encountered is travis-ci
>> cache key, i.e. we cache openssl builds between our builds.
>> so travis used the same cache key for both amd64 and arm64 builds (this
>> might have changed recently, I did not check yet)
>>
>> arm64 is in my queue (as well as recent s390x arch from travis), hope to
>> get back to it within month or so.
>>
>>
>>> From this discussion and from
>>> https://github.com/haproxy/haproxy/blob/master/.travis.yml I understand
>>> that there is no public CI in use (i.e. TravisCI or CirrusCI) but some of
>>> the developers run some tests locally regularly.
>>>
>>
>> it is not completely true.
>> there's public CI. we do not use github PR machinery, so sometimes tests
>> fail after push to master branch. it is considered as ok, failures are
>> fixed pretty fast.
>> for example, see
>> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html
>> it was just perfect, failure detected using CI and fixed within few days.
>> no customers affected.
>>
>>
>>> I've forked the project and tested on TravisCI (
>>> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the
>>> builds were not very stable:
>>> 1) some tests fail sometimes. I guess it is because of some timing issues
>>> For example:
>>> - https://travis-ci.org/martin-g/haproxy/jobs/636745241
>>> - https://travis-ci.org/martin-g/haproxy/jobs/636750676
>>> - https://travis-ci.org/martin-g/haproxy/jobs/636763346
>>>
>>
>> that's very interesting. I'll have a look.
>>
>>
>>
>>> 2) There was some weird issue on testing with LibreSSL
>>> The output redirect at
>>> https://github.com/haproxy/haproxy/blob/bb9da0b8e23c46caab34fe6005b66fa8065fe3ac/.travis.yml#L96
>>>  for
>>> some reason got stuck the build. I've removed temporarily the output
>>> redirects and then it passed. So, it looks like some issue with TravisCI
>>> environment.
>>>
>>
>> arm64 is slower, I gue

Re: ARM(64) builds

2020-01-16 Thread Илья Шипицин
чт, 16 янв. 2020 г. в 13:37, Илья Шипицин :

>
>
> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov :
>
>> Hi Илья,
>>
>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
>> wrote:
>>
>>>
>>> Hello, Martin!
>>>
>>> btw, just curious, how is Apache Foundation (or you personally)
>>> interested in all that ?
>>> please do not blame me, I really like to know.
>>>
>>
> ok, so you work in some company that is interested in haproxy on ARM64.
> you do not want to tell it's name, at least is it legal ? is it related to
> some government ?
> if "no" and "no", I guess most people won't ask any more questions :)
>

oops. I meant "yes" and "no"


>
> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
> contribute to it.
> Please do not consider me as an "official representative".
>
>
> I'm interested in testing haproxy on ARM64, I planned to do so. I can
> priorierize it according to your interest to it.
> and yes, I can accept hardware donation (even considering I'm not part of
> Haproxy Inc).
>
> also, from my point of view, what would be really useful in your case is
> testing not just official reg-tests, but also
> your configs. reg-tests cover only partially. If you enable clang asan in
> your own workload there's a chance to catch
> something interesting (or, to make sure your own workload is ok). I can
> help with that as well.
>
>
>>
>>>
>> @apache.org is just one of my several emails. And it is the default one
>> in my email client.
>> ASF is not related anyhow to my participation here.
>> If I used my work email then it might look like some kind of
>> advertisement. I'd like to avoid that!
>> Next time I will use my @gmail.com one, as more neutral. Actually I've
>> used the GMail one when registering to this mailing list, so probably the
>> post from @apache has been moderated. I'll be more careful next time!
>>
>> Thanks, Илья!
>>
>> Regards,
>> Martin
>>
>>
>>>
>>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :
>>>
>>>> Hello HAProxy developers,
>>>>
>>>> At work we are going to use more and more ARM64 based servers and
>>>> HAProxy is one of the main products we rely on.
>>>> So I went checking whether HAProxy project has a CI environment for
>>>> testing on ARM architecture.
>>>>
>>>
>>> we are looking towards
>>> https://docs.travis-ci.com/user/multi-cpu-architectures
>>>
>>>
>>>> I've found this recent discussion:
>>>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
>>>> didn't find a way how to continue on the same mail thread, so I'm starting
>>>> a new one. Apologies for that!).
>>>>
>>>
>>> I played with arm64 for a while, the issue I encountered is travis-ci
>>> cache key, i.e. we cache openssl builds between our builds.
>>> so travis used the same cache key for both amd64 and arm64 builds (this
>>> might have changed recently, I did not check yet)
>>>
>>> arm64 is in my queue (as well as recent s390x arch from travis), hope to
>>> get back to it within month or so.
>>>
>>>
>>>> From this discussion and from
>>>> https://github.com/haproxy/haproxy/blob/master/.travis.yml I
>>>> understand that there is no public CI in use (i.e. TravisCI or CirrusCI)
>>>> but some of the developers run some tests locally regularly.
>>>>
>>>
>>> it is not completely true.
>>> there's public CI. we do not use github PR machinery, so sometimes tests
>>> fail after push to master branch. it is considered as ok, failures are
>>> fixed pretty fast.
>>> for example, see
>>> https://www.mail-archive.com/haproxy@formilux.org/msg35910.html
>>> it was just perfect, failure detected using CI and fixed within few
>>> days. no customers affected.
>>>
>>>
>>>> I've forked the project and tested on TravisCI (
>>>> https://travis-ci.org/martin-g/haproxy/builds) but unfortunately the
>>>> builds were not very stable:
>>>> 1) some tests fail sometimes. I guess it is because of some timing
>>>> issues
>>>> For example:
>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636745241
>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636750676
>>>> - https://travis-ci.org/martin-g/haproxy/jobs/636763346
>>

Re: [PATCH] BUG/MINOR: dns: Make dns_query_id_seed unsigned

2020-01-18 Thread Илья Шипицин
how noisy is ubsan ? should we add it to CI ?

сб, 18 янв. 2020 г. в 06:07, Tim Duesterhus :

> Left shifting of large signed values and negative values is undefined.
>
> In a test script clang's ubsan rightfully complains:
>
> > runtime error: left shift of 1934242336581872173 by 13 places cannot be
> represented in type 'int64_t' (aka 'long')
>
> This bug was introduced in the initial version of the DNS resolver
> in 325137d603aa81bd24cbd8c99d816dd42291daa7. The fix must be backported
> to HAProxy 1.6+.
> ---
>  src/dns.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/dns.c b/src/dns.c
> index bc68a81c0..64bb0d15f 100644
> --- a/src/dns.c
> +++ b/src/dns.c
> @@ -54,7 +54,7 @@
>  struct list dns_resolvers  = LIST_HEAD_INIT(dns_resolvers);
>  struct list dns_srvrq_list = LIST_HEAD_INIT(dns_srvrq_list);
>
> -static THREAD_LOCAL int64_t dns_query_id_seed = 0; /* random seed */
> +static THREAD_LOCAL uint64_t dns_query_id_seed = 0; /* random seed */
>
>  DECLARE_STATIC_POOL(dns_answer_item_pool, "dns_answer_item",
> sizeof(struct dns_answer_item));
>  DECLARE_STATIC_POOL(dns_resolution_pool,  "dns_resolution",
> sizeof(struct dns_resolution));
> --
> 2.25.0
>
>
>


[PATCH] improving ssl defines

2020-01-18 Thread Илья Шипицин
Hello,

let us switch to constants check, not versions.

cheers,
Ilya Shipitcin
From a8b68e746bb71c4fee65a05bea8287ad970c979c Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 18 Jan 2020 18:42:45 +0500
Subject: [PATCH] BUG/MINOR: ssl: fix build on openssl-1.1.0

while working on issue #429, I built various openssl versions,
let us improve ssl defines, switch to features, not versions
---
 include/common/openssl-compat.h | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/include/common/openssl-compat.h b/include/common/openssl-compat.h
index 72b4e2fe2..cb9caa3e9 100644
--- a/include/common/openssl-compat.h
+++ b/include/common/openssl-compat.h
@@ -284,8 +284,11 @@ static inline void EVP_PKEY_up_ref(EVP_PKEY *pkey)
 #define X509_getm_notAfter  X509_get_notAfter
 #endif
 
-#if (OPENSSL_VERSION_NUMBER < 0x101fL || defined LIBRESSL_VERSION_NUMBER)
+#if !defined(EVP_CTRL_AEAD_SET_IVLEN)
 #define EVP_CTRL_AEAD_SET_IVLEN EVP_CTRL_GCM_SET_IVLEN
+#endif
+
+#if !defined(EVP_CTRL_AEAD_SET_TAG)
 #define EVP_CTRL_AEAD_SET_TAG   EVP_CTRL_GCM_SET_TAG
 #endif
 
-- 
2.24.1



Re: ARM(64) builds

2020-01-19 Thread Илья Шипицин
вс, 19 янв. 2020 г. в 13:13, Martin Grigorov :

> Hi,
>
> On Sat, Jan 18, 2020, 22:10 Илья Шипицин  wrote:
>
>> tests on ARM64 randomly fail
>> https://travis-ci.com/chipitsine/haproxy/jobs/277236120
>>
>> (after restart there's a good chance to success)
>>
>
> I have the same observation on TravisCI. I think the reason is that their
> arm64 instances are less powerful than the amd64 ones.
> At https://docs.travis-ci.com/user/multi-cpu-architectures it is said:
> While available to all Open Source repositories, the concurrency available
> for multiple CPU arch-based jobs is limited during the alpha period.
> Not sure how much limited it is though.
>

I'd like to investigate that on dedicated ARM64 server (which I currently
does not have).
I applied for one at Linode, no answer yet.


also, it might depend on number of CPU (race condition likely reproduce on
many CPU than
on single CPU server).

also, arm64 seems not very happy about our linker options

/usr/bin/ld: src/ev_poll.o(.debug_info+0x78): R_AARCH64_ABS64 used
with TLS symbol poll_events



>
> Today this test failed once on my VM:
>
> #top  TEST reg-tests/seamless-reload/abns_socket.vtc FAILED (1.111)
> exit=2
> 1 tests failed, 0 tests skipped, 34 tests passed
> ## Gathering results ##
> ## Test case: reg-tests/seamless-reload/abns_socket.vtc ##
> ## test results in:
> "/tmp/haregtests-2020-01-19_08-06-39.45Hchw/vtc.8496.328d7f95"
>  c1   HTTP rx failed (fd:6 read: Connection reset by peer)
> Makefile:964: recipe for target 'reg-tests' failed
> make: *** [reg-tests] Error 1
>
> but the next 4 runs were successul.
>
>
>> сб, 18 янв. 2020 г. в 09:52, Martin Grigorov :
>>
>>>
>>>
>>> On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov <
>>> martin.grigo...@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Jan 17, 2020, 23:12 William Lallemand 
>>>> wrote:
>>>>
>>>>> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote:
>>>>> > Testing with haproxy version: 2.2-dev0-70c5b0-123
>>>>>
>>>>> This binary was built with code from 1 week ago, it's normal that the
>>>>> test does
>>>>> not work since the fix was made this week.
>>>>>
>>>>
>>>> I'm using the same steps to build HAProxy as from .travis-ci.yml
>>>> I guess I have to add "make clean" in the beginning.
>>>> I'll try it tomorrow! Thanks!
>>>>
>>>
>>> That was it!
>>> Everything is back to normal now!
>>> Thank you, William!
>>>
>>>
>>>>
>>>>
>>>>> --
>>>>> William Lallemand
>>>>>
>>>>


Re: ARM(64) builds

2020-01-18 Thread Илья Шипицин
tests on ARM64 randomly fail
https://travis-ci.com/chipitsine/haproxy/jobs/277236120

(after restart there's a good chance to success)

сб, 18 янв. 2020 г. в 09:52, Martin Grigorov :

>
>
> On Fri, Jan 17, 2020 at 11:17 PM Martin Grigorov <
> martin.grigo...@gmail.com> wrote:
>
>>
>>
>> On Fri, Jan 17, 2020, 23:12 William Lallemand 
>> wrote:
>>
>>> On Fri, Jan 17, 2020 at 08:50:27PM +0200, Martin Grigorov wrote:
>>> > Testing with haproxy version: 2.2-dev0-70c5b0-123
>>>
>>> This binary was built with code from 1 week ago, it's normal that the
>>> test does
>>> not work since the fix was made this week.
>>>
>>
>> I'm using the same steps to build HAProxy as from .travis-ci.yml
>> I guess I have to add "make clean" in the beginning.
>> I'll try it tomorrow! Thanks!
>>
>
> That was it!
> Everything is back to normal now!
> Thank you, William!
>
>
>>
>>
>>> --
>>> William Lallemand
>>>
>>


Re: [PATCH] BUG/MINOR: dns: Make dns_query_id_seed unsigned

2020-01-19 Thread Илья Шипицин
вс, 19 янв. 2020 г. в 14:09, Willy Tarreau :

> On Sun, Jan 19, 2020 at 12:38:43PM +0500,  ??? wrote:
> > > > how noisy is ubsan ? should we add it to CI ?
> > > >
> > >
> > > No idea. I created a dedicated test executable and copied over the
> > > implementation to avoid needing to configure HAProxy to use DNS for the
> > > verification of my patch.
> > >
> > > In short: Simply try for yourself, please :-)
> > >
> >
> > it's bloody murder
> >
> > https://travis-ci.com/chipitsine/haproxy/jobs/277286655
>
> Thanks for the test. This report is not much encouraging considering
> that it shows the test confuses pointer calculations and pointer
> dereferences. Indeed, I've checked them all and they're all the
> byproduct of calling LIST_ELEM to retrieve a list element from the
> list's pointer, which is done by subtracting an offset from this
> pointer. There is absolutely no memory access done there. Thus I
> guess this rules out any hope to use this in automated tests :-)
>

ubsan actually includes several sanitizers. I'll try them one by one.


>
> Willy
>


[PATCH] better anti replay check

2020-01-18 Thread Илья Шипицин
Hello,

let us check constants, not openssl versions.

Cheers,
Ilya Shipitcin
From eab262bda04f0f0caf8020a6837a75cdd5821e94 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 19 Jan 2020 12:20:14 +0500
Subject: [PATCH] BUILD: ssl: more elegant anti-reply feature presence check

---
 src/ssl_sock.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index 95dbe4c89..8484040f5 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -4682,7 +4682,7 @@ ssl_sock_initial_ctx(struct bind_conf *bind_conf)
 #ifdef OPENSSL_IS_BORINGSSL
 	SSL_CTX_set_select_certificate_cb(ctx, ssl_sock_switchctx_cbk);
 	SSL_CTX_set_tlsext_servername_callback(ctx, ssl_sock_switchctx_err_cbk);
-#elif (HA_OPENSSL_VERSION_NUMBER >= 0x10101000L)
+#elif defined(SSL_OP_NO_ANTI_REPLAY)
 	if (bind_conf->ssl_conf.early_data)
 		SSL_CTX_set_options(ctx, SSL_OP_NO_ANTI_REPLAY);
 	SSL_CTX_set_client_hello_cb(ctx, ssl_sock_switchctx_cbk, NULL);
-- 
2.24.1



[PATCH] introduce ARM64 travis-ci builds

2020-01-18 Thread Илья Шипицин
hello,

sometimes arm64 builds fails, I think it is good chance to introduce
regular builds
and fix them.

also, few small improvements.

cheers,
Ilya Shipicin
From e13a8d0e0e324c49f6a34d74b889a1c3fae6b6d9 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sun, 19 Jan 2020 12:14:02 +0500
Subject: [PATCH] BUILD: CI: introduce ARM64 builds

also several small changes:

openssl-1.0.2 upgraded to 1.0.2u
ppc64le upgraded to "bionic" (it was tricky part, linux-ppc64le is xenial,
while arch: ppc64le is bionic).
additional wait introduced for build ssl.
---
 .travis.yml | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index e69f44166..9046e6dcd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -20,7 +20,11 @@ env:
 
 addons:
   apt:
-packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev ]
+update: true
+packages: [ liblua5.3-dev, libsystemd-dev, libpcre2-dev, socat ]
+  homebrew:
+update: true
+packages: [ socat ]
 
 cache:
   directories:
@@ -29,11 +33,18 @@ cache:
 
 matrix:
   include:
-  - os: linux-ppc64le
+  - os: linux
+arch: ppc64le
 if: type == cron
 compiler: gcc
-env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2t LABEL="linux-ppc64le"
+env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
+  - os: linux
+arch: amd64
+if: type != cron
+compiler: clang
+env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
   - os: linux
+arch: arm64
 if: type != cron
 compiler: clang
 env: TARGET=linux-glibc OPENSSL_VERSION=1.1.1d
@@ -93,7 +104,7 @@ install:
   - git clone https://github.com/VTest/VTest.git ../vtest
   # Special flags due to: https://github.com/vtest/VTest/issues/12
   - make -C ../vtest FLAGS="-O2 -s -Wall"
-  - scripts/build-ssl.sh > build-ssl.log 2>&1 || (cat build-ssl.log && exit 1)
+  - travis_wait scripts/build-ssl.sh > build-ssl.log 2>&1 || (cat build-ssl.log && exit 1)
 
 script:
   - if [ "$CC"  = "clang" ]; then export FLAGS="$FLAGS USE_OBSOLETE_LINKER=1" DEBUG_CFLAGS="-g -fsanitize=address" LDFLAGS="-fsanitize=address"; fi
-- 
2.24.1



Re: [PATCH] BUG/MINOR: dns: Make dns_query_id_seed unsigned

2020-01-18 Thread Илья Шипицин
сб, 18 янв. 2020 г. в 15:30, Tim Düsterhus :

> Ilya,
>
> Am 18.01.20 um 10:02 schrieb Илья Шипицин:
> > how noisy is ubsan ? should we add it to CI ?
> >
>
> No idea. I created a dedicated test executable and copied over the
> implementation to avoid needing to configure HAProxy to use DNS for the
> verification of my patch.
>
> In short: Simply try for yourself, please :-)
>

it's bloody murder

https://travis-ci.com/chipitsine/haproxy/jobs/277286655


>
> Best regards
> Tim Düsterhus
>


Re: freebsd ci is broken - commit 08fa16e - curl download stalls in reg-tests/compression/lua_validation.vtc

2020-01-14 Thread Илья Шипицин
PiBa, how many CPU cores are you running ?

it turned out that I run tests on very low vm, which only has 1 core. and
tests pass.
cirrus-ci as far as I remember do have many cores.

can you find single core vm ?

ср, 15 янв. 2020 г. в 00:02, PiBa-NL :

> Hi Ilya,
>
> Thanks!
>
> Op 14-1-2020 om 07:48 schreef Илья Шипицин:
>
> Hello,
>
> since
>
> https://github.com/haproxy/haproxy/commit/08fa16e397ffb1c6511b98ade2a3bfff9435e521
>
> freebsd CI is red: https://cirrus-ci.com/task/5960933184897024
>
> I'd say "it is something with CI itself",  when I run the same tests
> locally on freebsd, it is green.
>
> Sadly i do get the same problem on my test server (version info below its
> version 11.1 is a bit outdated, but hasn't failed my before...).
>
>
> PiBa ?
>
>
> thanks,
> Ilya Shipitcin
>
> Below a part of the output that the test generates for me. The first curl
> request seems to succeed, but the second one runs into a timeout..
> When compiled with the commit before 08fa16e
> <https://github.com/haproxy/haproxy/commit/08fa16e397ffb1c6511b98ade2a3bfff9435e521>
> it does not show that behaviour.. Current latest(24c928c) commit is still
> affected..
>
>  top  shell_out|  % Total% Received % Xferd  Average Speed
> TimeTime Time  Current
>  top  shell_out| Dload  Upload
> Total   SpentLeft  Speed
>  top  shell_out|\r  0 00 00 0  0  0
> --:--:-- --:--:-- --:--:-- 0\r100  418k0  418k0 0
> 1908k  0 --:--:-- --:--:-- --:--:-- 1908k
>  top  shell_out|  % Total% Received % Xferd  Average Speed
> TimeTime Time  Current
>  top  shell_out| Dload  Upload
> Total   SpentLeft  Speed
>  top  shell_out|\r  0 00 00 0  0  0
> --:--:-- --:--:-- --:--:-- 0\r100  141k0  141k0 0
> 284k  0 --:--:-- --:--:-- --:--:--  284k\r100  343k0  343k0
> 0   156k  0 --:--:--  0:00:02 --:--:--  156k\r100  343k0  343k
> 0 0   105k  0 --:--:--  0:00:03 --:--:--  105k\r100  343k0
> 343k0 0  81274  0 --:--:--  0:00:04 --:--:-- 81274\r100
> 343k0  343k0 0  65228  0 --:--:--  0:00:05 --:--:--
> 65240\r100  343k0  343k0 0  54481  0 --:--:--  0:00:06
> --:--:-- 34743\r100  343k0  343k0 0  46768  0 --:--:--
> 0:00:07 --:--:-- 0\r100  343k0  343k0 0  40968  0
> --:--:--  0:00:08 --:--:-- 0\r100  343k0  343k0 0
> 36452  0 --:--:--  0:00:09 --:--:-- 0\r100  343k0  343k
> 0 0  32830  0 --:--:--  0:00:10 --:--:-- 0\r100  343k0
> 343k0 0  29865  0 --:--:--  0:00:11 --:--:-- 0\r100
> 343k0  343k0 0  27395  0 --:--:--  0:00:12 --:--:--
> 0\r100  343k0  343k0 0  25297  0 --:--:--  0:00:13
> --:--:-- 0\r100  343k0  343k0 0  23500  0 --:--:--
> 0:00:14 --:--:-- 0\r100  343k0  343k0 0  23431  0
> --:--:--  0:00:15 --:--:-- 0
>  top  shell_out|curl: (28) Operation timed out after 15002
> milliseconds with 351514 bytes received
>  top  shell_out|Expecting checksum 4d9c62aa5370b8d5f84f17ec2e78f483
>  top  shell_out|Received checksum: da2d120aedfd693eeba9cf1e578897a8
>  top  shell_status = 0x0001
>  top  shell_exit not as expected: got 0x0001 wanted 0x
> *top  RESETTING after
> ./work/haproxy-08fa16e/reg-tests/compression/lua_validation.vtc
>
> Should i update to a newer FreeBSD version, or is it likely unrelated, and
> in need of some developer attention.. Do you (Willy or anyone), need more
> information from my side? Or is there a patch i can try to validate?
>
> Regards,
> PiBa-NL (Pieter)
>
>
> Yes im running a somewhat outdated OS here:
>   FreeBSD freebsd11 11.1-RELEASE FreeBSD 11.1-RELEASE #0 r321309: Fri Jul
> 21 02:08:28 UTC 2017
> r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64
>
> Version used:
>   haproxy -vv
> HA-Proxy version 2.2-dev0-08fa16e 2020/01/08 - https://haproxy.org/
> Status: development branch - not safe for use in production.
> Known bugs: https://github.com/haproxy/haproxy/issues?q=is:issue+is:open
> Build options :
>   TARGET  = freebsd
>   CPU = generic
>   CC  = cc
>   CFLAGS  = -pipe -g -fstack-protector -fno-strict-aliasing
> -fno-strict-aliasing -Wdeclaration-after-statement -fwrapv
> -fno-strict-overflow -Wno-null-dereference -Wno-unused-label
> -Wno-unused-parameter -Wno-sign-compare -Wno-ignored-qualifiers
> -Wno-unused-command-line-argument -Wno-missing-field-initializers
> -Wno-address-of-packed-member -

Re: freebsd ci is broken - commit 08fa16e - curl download stalls in reg-tests/compression/lua_validation.vtc

2020-01-14 Thread Илья Шипицин
ср, 15 янв. 2020 г. в 04:18, PiBa-NL :

> Hi Ilya, Willy,
>
> Op 14-1-2020 om 21:40 schreef Илья Шипицин:
> > PiBa, how many CPU cores are you running ?
> >
> > it turned out that I run tests on very low vm, which only has 1 core.
> > and tests pass.
> > cirrus-ci as far as I remember do have many cores.
> I was running with 16 cores..
> >
> > can you find single core vm ?
>
> Well, i reconfigured the VM to have 1 core, but same issue seems to show
> up, though not on every time the test is run, and actually a bit less
> often..
> Below some additional testresults with different kqueue / vCPU settings..
>

I run tests on MS Azure B1s vm, it is cheapest size. Single "shared" core.
Tests always pass.
looks like we've found race condition :)


>
>
> *VM with 1 vCPU*
>
> Running: ./vtest/VTest-master/vtest -Dno-htx=no -l -k -b 50M -t 5 -n 20
> ./work/haproxy-08fa16e/reg-tests/compression/lua_validation.vtc
>Results in: 4 tests failed, 0 tests skipped, 16 tests passed
>
> Adding "nokqueue" in the vtc file i get:
>8 tests failed, 0 tests skipped, 12 tests passed
>4 tests failed, 0 tests skipped, 16 tests passed
>
> So its a bit random, but the 'nokqueue' directive does not seem to
> affect results much..
>
>
> *With 16 vCPU*
> Without nokqueue: 16 tests failed, 0 tests skipped, 4 tests passed
> With nokqueue (using poll): 17 tests failed, 0 tests skipped, 3 tests
> passed
>
> The failure rate seems certainly higher with many cores..
>
>
> * Using commit 0eae632 it works OK*
> Just to be sure i re-tested on 16 cores with 2.2-dev0-0eae632 but that
> does nicely pass: 0 tests failed, 0 tests skipped, 20 tests passed
>
> Regards,
> PiBa-NL (Pieter)
>
>


Re: [PATCH] improving ssl defines

2020-01-21 Thread Илья Шипицин
ср, 22 янв. 2020 г. в 11:24, Willy Tarreau :

> Hi Ilya,
>
> On Sat, Jan 18, 2020 at 06:47:48PM +0500,  ??? wrote:
> > Hello,
> >
> > let us switch to constants check, not versions.
> >
> > cheers,
> > Ilya Shipitcin
>
> > From a8b68e746bb71c4fee65a05bea8287ad970c979c Mon Sep 17 00:00:00 2001
> > From: Ilya Shipitsin 
> > Date: Sat, 18 Jan 2020 18:42:45 +0500
> > Subject: [PATCH] BUG/MINOR: ssl: fix build on openssl-1.1.0
> >
> > while working on issue #429, I built various openssl versions,
> > let us improve ssl defines, switch to features, not versions
>
> What is the build issue you've encountered exactly (and/or in what
> specific environment), and should this build fix be backported, and if
> so, how far ?
>
>
I think it should not be backported.

I've taken openssl branches (master, openssl_1_1_1 and openssl_1_1_0) and
built haproxy against various commits (not tagged as releases).
sometimes build fail, I ended with changing "ifdef" from
version-dependendent (which is not accurate) to feature dependent.

I can find particular commits, but I think it is not important since
released versions of openssl build just fine (I'll provide detailed info if
build will ever fail on openssl release).



> Thanks,
> Willy
>


Re: [PATCH] introduce ARM64 travis-ci builds

2020-01-22 Thread Илья Шипицин
ср, 22 янв. 2020 г. в 10:49, Willy Tarreau :

> On Sun, Jan 19, 2020 at 12:18:00PM +0500,  ??? wrote:
> > hello,
> >
> > sometimes arm64 builds fails, I think it is good chance to introduce
> > regular builds
> > and fix them.
> >
> > also, few small improvements.
>


occasionely, I introduced "socat" for linux builds. god knows, I only
wanted socat on osx.

now we have daily failing openssl-1.0.2:

https://travis-ci.com/haproxy/haproxy/jobs/278244081

should I temporarily mark it as "allowed failure" on travis ? (it will be
fixed in issue #429)




>
> Merged, thanks Ilya. Next time, please be stricter and split your
> additions and your small improvements as it can help track down issues
> or do partial reverts.
>
> Willy
>


[PATCH] temporarily mark openssl-1.0.2 builds as allowed failure

2020-01-22 Thread Илья Шипицин
Hello,

this is a follow up patch for the recent travis-ci improvement.

Cheers,
Ilya Shipitcin
From f13c8cd8c28d376c914cee24ac9b7e09a7203473 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 22 Jan 2020 15:23:51 +0500
Subject: [PATCH] BUILD: CI: temporarily mark openssl-1.0.2 as allowed failure

Until #429 is resolved, let us ignore openssl-1.0.2 failures
---
 .travis.yml | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 9046e6dcd..a0d9502b4 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -99,6 +99,12 @@ matrix:
 before_script:
   - git clone http://git.1wt.eu/git/libslz.git/
   - cd libslz && make && make PREFIX=${HOME}/opt install && cd ..
+  allow_failures:
+  - os: linux
+arch: ppc64le
+if: type == cron
+compiler: gcc
+env: TARGET=linux-glibc OPENSSL_VERSION=1.0.2u
 
 install:
   - git clone https://github.com/VTest/VTest.git ../vtest
-- 
2.24.1



[PATCH[ re-enable cygwin CI builds (Github Actions)

2020-01-22 Thread Илья Шипицин
Hello,

I spent lots of time trying to get cygwin on travis working.
no luck.

let us move it to Github Actions.
as far as I tried no extra step are required for enabling, just proper file
in

.github/workflows

(attached in patch).


Cheers,
Ilya Shipitcin
From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 23 Jan 2020 02:33:38 +0500
Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions

builds on travis-ci fail because of
https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359
unfortunately, documentation does not help.

so, let us move builds to Github Actions.

also, remove deprecated "sudo" directive from .travis.yml
---
 .github/workflows/windows-latest.yml | 20 
 .travis.yml  |  7 ---
 2 files changed, 20 insertions(+), 7 deletions(-)
 create mode 100644 .github/workflows/windows-latest.yml

diff --git a/.github/workflows/windows-latest.yml b/.github/workflows/windows-latest.yml
new file mode 100644
index 0..27b16eec6
--- /dev/null
+++ b/.github/workflows/windows-latest.yml
@@ -0,0 +1,20 @@
+# build status appears on https://github.com/haproxy/haproxy/actions
+
+name: windows-latest
+
+on: [push]
+
+jobs:
+  cygwin:
+
+runs-on: windows-latest
+
+steps:
+- uses: actions/checkout@v1
+- name: install prerequisites
+  run: choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
+- name: fake step
+  run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd'
+- name: build
+  run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
+
diff --git a/.travis.yml b/.travis.yml
index a0d9502b4..bf4b82aa9 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,6 +1,5 @@
 # build status appears on https://travis-ci.com/haproxy/haproxy
 
-sudo: required
 dist: bionic
 
 language: c
@@ -86,12 +85,6 @@ matrix:
 if: type != cron
 compiler: clang
 env: TARGET=osx FLAGS="USE_OPENSSL=1" OPENSSL_VERSION=1.1.1d
-#  - os: windows
-#if: type == cron
-#install:
-#  - choco install bash make libssl-devel cygwin-devel gcc-core libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
-#script:
-#  - C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1 USE_THREAD=1 && ./haproxy -vv'
   - os: linux
 if: type == cron
 compiler: clang
-- 
2.24.1



Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

2020-01-22 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 02:59, Tim Düsterhus :

> Ilya,
>
> Am 22.01.20 um 22:41 schrieb Илья Шипицин:
> > let us move it to Github Actions.
> > as far as I tried no extra step are required for enabling, just proper
> file
> > in
> >
> > .github/workflows
>
> Yes, that's my understanding of GitHub actions as well. However I
> dislike having three types of CI (Travis, Cirrus and GitHub Actions).
> Can Travis be replaced with GitHub Actions for our use case? I guess
> Cirrus can't, because FreeBSD?
>


both travis and github actions do offer 4 parallel builds, while cirrus and
app veyor offer 1 parallel build.

travis-ci offers ARM64, ppc64le and s390x (not available on github actions).



>
> To the patch:
>
> > From 372c547a94ff02fa04ca052a87863161d3e85b37 Mon Sep 17 00:00:00 2001
> > From: Ilya Shipitsin 
> > Date: Thu, 23 Jan 2020 02:33:38 +0500
> > Subject: [PATCH] BUILD: CI: move cygwin builds to Github Actions
> >
> > builds on travis-ci fail because of
> >
> https://travis-ci.community/t/cygwin-issue-cygheap-base-mismatch-detected/5359
> > unfortunately, documentation does not help.
> >
> > so, let us move builds to Github Actions.
> >
> > also, remove deprecated "sudo" directive from .travis.yml
>
> Could you please create separate patches for several unrelated changes?
> If there's an "also" in the commit message then that's an indication
> that the patch should be split.
>
> Single-line patches are perfectly acceptable. I send series containing
> multiple single-line patches to the list on a regular basis (as distinct
> emails even, because I use git-send-email).
>
> > +steps:
> > +- uses: actions/checkout@v1
> > +- name: install prerequisites
> > +  run: choco install bash make libssl-devel cygwin-devel gcc-core
> libgcc1 binutils lua-devel libpcre-devel zlib-devel --source cygwin
> > +- name: fake step
>
> Give a proper name to that step. "Show pwd" is fine.
>


there should be no such step.
however, without that step cygwin fails for no visible reason.

I'll remove this step after I'll find it out.


>
> > +  run: C:\\tools\\cygwin\\bin\\bash -lc 'pwd'
> > +- name: build
> > +  run: C:\\tools\\cygwin\\bin\\bash -lc 'cd $OLDPWD && make
> TARGET=cygwin USE_ZLIB=1 USE_PCRE=1 USE_PCRE_JIT=1 USE_LUA=1 USE_OPENSSL=1
> USE_THREAD=1 && ./haproxy -vv'
>
> 1. Does GitHub Actions support variables for the build flags, similar to
> Travis? That would make things more readable.
>


yes.


>
> 2. Split the ./haproxy -vv into a separate step, if that's possible.
>


sure, it's possible

>
> Best regards
> Tim Düsterhus
>


Re: [PATCH[ re-enable cygwin CI builds (Github Actions)

2020-01-22 Thread Илья Шипицин
чт, 23 янв. 2020 г. в 03:10, Tim Düsterhus :

> Ilya,
>
> Am 22.01.20 um 23:04 schrieb Илья Шипицин:
> >> Yes, that's my understanding of GitHub actions as well. However I
> >> dislike having three types of CI (Travis, Cirrus and GitHub Actions).
> >> Can Travis be replaced with GitHub Actions for our use case? I guess
> >> Cirrus can't, because FreeBSD?
> >
> > both travis and github actions do offer 4 parallel builds, while cirrus
> and
> > app veyor offer 1 parallel build.
>
> Parallel Builds just improve test speed. I don't consider that an
> important selling point for us. The development process is fairly
> asynchronous anyway and the important thing is that there are results
> for more obscure configurations, not that there results within 1 minute.
> However ...
>
> > travis-ci offers ARM64, ppc64le and s390x (not available on github
> actions).
>
> ... that's a good argument to keep Travis-CI. Too bad, I like the GitHub
> Actions integration better.
>
>
me too :)

travis is not comfortable for choosing custom images (for example, if you
wish to build on Fedora or Arch).



> >>> +- name: fake step
> >>
> >> Give a proper name to that step. "Show pwd" is fine.
> >>
> >
> >
> > there should be no such step.
> > however, without that step cygwin fails for no visible reason.
>
> Then it's even more important to give a good name (or comment).
> Otherwise you might risk that someone removes it accidentally!
>
> >> 2. Split the ./haproxy -vv into a separate step, if that's possible.
> >>
> > sure, it's possible
> >
>
> Perfect. I wasn't sure whether the environment was somehow cleaned up in
> between the steps.
>
> Best regards
> Tim Düsterhus
>


Re: ARM(64) builds

2020-01-16 Thread Илья Шипицин
привет!

пт, 17 янв. 2020 г. в 11:42, Martin Grigorov :

> Привет Илья,
>
> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин 
> wrote:
>
>>
>>
>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov :
>>
>>> Hi Илья,
>>>
>>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
>>> wrote:
>>>
>>>>
>>>> Hello, Martin!
>>>>
>>>> btw, just curious, how is Apache Foundation (or you personally)
>>>> interested in all that ?
>>>> please do not blame me, I really like to know.
>>>>
>>>
>> ok, so you work in some company that is interested in haproxy on ARM64.
>> you do not want to tell it's name, at least is it legal ? is it related
>> to some government ?
>> if "no" and "no", I guess most people won't ask any more questions :)
>>
>
> It is legal and I do not work for a government of any country!
>
>
>>
>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
>> contribute to it.
>> Please do not consider me as an "official representative".
>>
>>
>> I'm interested in testing haproxy on ARM64, I planned to do so. I can
>> priorierize it according to your interest to it.
>> and yes, I can accept hardware donation (even considering I'm not part of
>> Haproxy Inc).
>>
>> also, from my point of view, what would be really useful in your case is
>> testing not just official reg-tests, but also
>> your configs. reg-tests cover only partially. If you enable clang asan in
>> your own workload there's a chance to catch
>> something interesting (or, to make sure your own workload is ok). I can
>> help with that as well.
>>
>
> Thanks for the offer!
> I've discussed it with my managers.
> Our offer is to donate a VM that could be used as an official CI agent for
> the HAProxy project long term.
>

I'd split this into short term and long term approach.

if you need to start any time soon, I'd focus on your own workload testing
first. I would build stand which emulates
your production workload, compile haproxy using clang address sanitizer and
give it a try (functional testing, load testing, ...)

I can help with that.

As for long term solution, currently haproxy simply cannot attach any
dedicated build agent to their CI. travis-ci does not allow
attaching dedicated agents. And haproxy team is very conservative in adding
new CI servers.

I think, I will add arm64 (most probably openssl-1.1.1 only for now) soon.
Also, I'm going to investigate your libressl failures.

so, dedicated vm definetly will help in troubleshooting issues, for manual
builds. It would save bunch of time. I do not mind if you will
add my ssh to that vm.


also, I requested access to Linaro.


>
> You can use Linaro for short term testing though.
> https://www.linaro.cloud/
> Here you can request free VM for short periods:
> https://servicedesk.linaro.org/servicedesk/customer/portal/19/create/265
> P.S. Linaro is not my employer!
>
>
> Regards,
> Martin
>
>
>>
>>>
>>>>
>>> @apache.org is just one of my several emails. And it is the default one
>>> in my email client.
>>> ASF is not related anyhow to my participation here.
>>> If I used my work email then it might look like some kind of
>>> advertisement. I'd like to avoid that!
>>> Next time I will use my @gmail.com one, as more neutral. Actually I've
>>> used the GMail one when registering to this mailing list, so probably the
>>> post from @apache has been moderated. I'll be more careful next time!
>>>
>>> Thanks, Илья!
>>>
>>> Regards,
>>> Martin
>>>
>>>
>>>>
>>>> чт, 16 янв. 2020 г. в 12:32, Martin Grigorov :
>>>>
>>>>> Hello HAProxy developers,
>>>>>
>>>>> At work we are going to use more and more ARM64 based servers and
>>>>> HAProxy is one of the main products we rely on.
>>>>> So I went checking whether HAProxy project has a CI environment for
>>>>> testing on ARM architecture.
>>>>>
>>>>
>>>> we are looking towards
>>>> https://docs.travis-ci.com/user/multi-cpu-architectures
>>>>
>>>>
>>>>> I've found this recent discussion:
>>>>> https://www.mail-archive.com/haproxy@formilux.org/msg35302.html  (I
>>>>> didn't find a way how to continue on the same mail thread, so I'm starting
>>>>> a new one. Apologies for that!).
>>>>>
>>>>
>>>> I played with

Re: ARM(64) builds

2020-01-17 Thread Илья Шипицин
пт, 17 янв. 2020 г. в 19:33, Martin Grigorov :

>
>
> On Fri, Jan 17, 2020 at 4:13 PM Martin Grigorov 
> wrote:
>
>> Hi all,
>>
>> Today's build consistently fails on my ARM64 VM:
>>
>> ## Starting vtest ##
>> Testing with haproxy version: 2.2-dev0-70c5b0-123
>> #top  TEST reg-tests/mcli/mcli_start_progs.vtc FAILED (3.004) exit=2
>> 1 tests failed, 0 tests skipped, 34 tests passed
>> ## Gathering results ##
>> ## Test case: reg-tests/mcli/mcli_start_progs.vtc ##
>> ## test results in:
>> "/tmp/haregtests-2020-01-17_14-01-45.SGkYcJ/vtc.12807.6adfff44"
>>  h1   haproxy h1 PID file check failed:
>>  h1   Bad exit status: 0x0100 exit 0x1 signal 0 core 0
>> Makefile:964: recipe for target 'reg-tests' failed
>> make: *** [reg-tests] Error 1
>>
>
> git bisect blaims this commit:
>
> 25b569302167e71b32e569a2366027e8e320e80a is the first bad commit
> commit 25b569302167e71b32e569a2366027e8e320e80a
> Author: William Lallemand 
> Date:   Tue Jan 14 15:38:43 2020 +0100
>
> REGTEST: mcli/mcli_start_progs: start 2 programs
>
> This regtest tests the issue #446 by starting 2 programs and checking
> if
> they exist in the "show proc" of the master CLI.
>
> Should be backported as far as 2.0.
>
>
> https://travis-ci.com/haproxy/haproxy is green
> https://cirrus-ci.com/github/haproxy/haproxy is green
> and what is even more interesting is that
> https://travis-ci.org/martin-g/haproxy/builds (my fork with enabled ARM64
> testing on TravisCI) also just passed (after few failures due to timing
> issues (I guess)
>


timing out might happen because of
https://github.com/haproxy/haproxy/commit/ac8147446c7a3d1aa607042bc782095b03bc8dc4
your fork is 16 commits behind current master. try to rebase to master


>
>
>> Regards,
>> Martin
>>
>> On Fri, Jan 17, 2020 at 9:22 AM Илья Шипицин 
>> wrote:
>>
>>> привет!
>>>
>>> пт, 17 янв. 2020 г. в 11:42, Martin Grigorov >> >:
>>>
>>>> Привет Илья,
>>>>
>>>> On Thu, Jan 16, 2020 at 10:37 AM Илья Шипицин 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> чт, 16 янв. 2020 г. в 13:26, Martin Grigorov <
>>>>> martin.grigo...@gmail.com>:
>>>>>
>>>>>> Hi Илья,
>>>>>>
>>>>>> On Thu, Jan 16, 2020 at 10:19 AM Илья Шипицин 
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hello, Martin!
>>>>>>>
>>>>>>> btw, just curious, how is Apache Foundation (or you personally)
>>>>>>> interested in all that ?
>>>>>>> please do not blame me, I really like to know.
>>>>>>>
>>>>>>
>>>>> ok, so you work in some company that is interested in haproxy on
>>>>> ARM64.
>>>>> you do not want to tell it's name, at least is it legal ? is it
>>>>> related to some government ?
>>>>> if "no" and "no", I guess most people won't ask any more questions :)
>>>>>
>>>>
>>>> It is legal and I do not work for a government of any country!
>>>>
>>>>
>>>>>
>>>>> personally, I do not work at Haproxy Inc, I use haproxy, sometimes I
>>>>> contribute to it.
>>>>> Please do not consider me as an "official representative".
>>>>>
>>>>>
>>>>> I'm interested in testing haproxy on ARM64, I planned to do so. I can
>>>>> priorierize it according to your interest to it.
>>>>> and yes, I can accept hardware donation (even considering I'm not part
>>>>> of Haproxy Inc).
>>>>>
>>>>> also, from my point of view, what would be really useful in your case
>>>>> is testing not just official reg-tests, but also
>>>>> your configs. reg-tests cover only partially. If you enable clang asan
>>>>> in your own workload there's a chance to catch
>>>>> something interesting (or, to make sure your own workload is ok). I
>>>>> can help with that as well.
>>>>>
>>>>
>>>> Thanks for the offer!
>>>> I've discussed it with my managers.
>>>> Our offer is to donate a VM that could be used as an official CI agen

[PATCH] adapt regtest for travis-ci

2020-01-08 Thread Илья Шипицин
Hello,

for some reason "echo -e" does not work at least in travis-ci.
as discussed in #423 it is good to use "printf" instead.

Cheers,
Ilya Shipitcin
From e0ae24b754b6f660b0d942df25ed1e0bfe189967 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 8 Jan 2020 22:56:30 +0500
Subject: [PATCH] REGTEST: set_ssl_cert.vtc: replace "echo" with "printf"

"echo -e" for some reason does not work on travis-ci, so let us switch
to "printf"

Fixes: #423
---
 reg-tests/ssl/set_ssl_cert.vtc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/reg-tests/ssl/set_ssl_cert.vtc b/reg-tests/ssl/set_ssl_cert.vtc
index 3d6ff0c98..11b2d154b 100644
--- a/reg-tests/ssl/set_ssl_cert.vtc
+++ b/reg-tests/ssl/set_ssl_cert.vtc
@@ -42,7 +42,7 @@ shell {
 }
 
 shell {
-   echo -e "set ssl cert ${testdir}/common.pem <<\n$(cat ${testdir}/ecdsa.pem)\n" | socat "${tmpdir}/h1/stats" -
+   printf "set ssl cert ${testdir}/common.pem <<\n$(cat ${testdir}/ecdsa.pem)\n\n" | socat "${tmpdir}/h1/stats" -
echo "commit ssl cert ${testdir}/common.pem" | socat "${tmpdir}/h1/stats" -
 }
 
-- 
2.24.1



freebsd ci is broken

2020-01-13 Thread Илья Шипицин
Hello,

since
https://github.com/haproxy/haproxy/commit/08fa16e397ffb1c6511b98ade2a3bfff9435e521

freebsd CI is red: https://cirrus-ci.com/task/5960933184897024

I'd say "it is something with CI itself",  when I run the same tests
locally on freebsd, it is green.

PiBa ?


thanks,
Ilya Shipitcin


learning SSL stuff

2020-01-01 Thread Илья Шипицин
Hello,

while working on #429,
I noticed strange things that I do not understand
for example

if (ecdhe == NULL) {
(void)SSL_CTX_set_ecdh_auto(ctx, 1);
return cfgerr;
}

why do we need to call SSL_CTX_set_ecdh_auto right before returning error ?


Cheers,
Ilya Shipitsin


[PATCH] modernize cirrus-ci

2019-12-26 Thread Илья Шипицин
Hello,

small refinement of cirrus-ci magic.

Cheers,
Ilya Shipicin
From bc6989debcd3b2e4a45f18d2a58e65fa4ab2f555 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 26 Dec 2019 13:37:36 +0500
Subject: [PATCH] BUILD: CI: modernize cirrus-ci

use freebsd-12.1 instead of freebsd-12.0,
add freebsd-11.3 to build matrix,
install socat in order to run modern reg-tests
---
 .cirrus.yml | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index e4278c2ce..c13e1d8d4 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,11 +1,12 @@
 FreeBSD_task:
   freebsd_instance:
-image: freebsd-12-0-release-amd64
+matrix:
+  image_family: freebsd-12-1
+  image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
-  env:
-ASSUME_ALWAYS_YES: TRUE # required for unattended "pkg" invocation
+  install_script: 
+- pkg install -y openssl111 git gmake lua53 socat
   script:
-- pkg install openssl111 git gmake lua53
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
 - gmake CC=clang V=1 TARGET=freebsd USE_ZLIB=1 USE_PCRE=1 USE_OPENSSL=1 USE_LUA=1 LUA_INC=/usr/local/include/lua53 LUA_LIB=/usr/local/lib LUA_LIB_NAME=lua-5.3
@@ -18,7 +19,7 @@ centos_6_task:
 image: centos:centos6
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   script:
-- yum install -q -y gcc git openssl-devel pcre-devel epel-release
+- yum install -q -y gcc git openssl-devel pcre-devel epel-release socat
 - yum install -q -y python34
 - git clone https://github.com/VTest/VTest.git ../vtest
 # Special flags due to: https://github.com/vtest/VTest/issues/12
-- 
2.24.1



[PATCH] cirrus-ci: choose proper openssl package name

2020-01-07 Thread Илья Шипицин
hello,


freebsd-11.3 names package "openssl"
freebsd-12.1 names package "openssl111"

I added switch for that

Cheers,
Ilya Shipitcin
From 614d7497641f0a2459c5f0b584fc84345ba5aeee Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 7 Jan 2020 23:41:24 +0500
Subject: [PATCH] BUILD: cirrus-ci: choose proper openssl package name

freebsd-11.3 and 12.1 comes with different openssl naming
let us add proper switch to cirrus-ci script
---
 .cirrus.yml | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index d14678111..4143e687b 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -5,7 +5,13 @@ FreeBSD_task:
   image_family: freebsd-11-3-snap
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- pkg install -y openssl111 git gmake lua53 socat
+- case `uname -r` in
+11.3*)
+  export SSL=openssl;;
+12.1*)
+  export SSL=openssl111;;
+  esac
+- pkg install -y $SSL git gmake lua53 socat
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- 
2.24.1



Re: TLS tickets prone to MITM attacks (was: [PR] Docs tls tickets)

2020-03-09 Thread Илья Шипицин
пн, 9 мар. 2020 г. в 23:21, Björn Jacke :

> On 2020-03-09 at 17:44 +0100 Lukas Tribus sent off:
> > Perhaps we can relax the wording a bit here and describe the actual
> > technical issue along with some recommendations. Apache for example
> > documents [1]:
>
> I think the wording from the patch is still quite relaxed :). One of the
> best
> summaries describing the session ticket flaws, which I recommend is this:
> https://blog.filippo.io/we-need-to-talk-about-session-tickets/
>
> I would disable session tickets by default in haproxy. Given that most
> clients support TLS 1.3 already this change would not even slow down many
> clients.
>

TLS tickets really require more love :)

actually, there are two bad choices here

1) to specify TLS ticket key
2) not to specify

if you specify, your security team will tell you that "it is not secure".
if you do not specify, keys are generated on startup and it lead to huge
CPU spike on app reload (if you apply new config, app is reloaded and keys
are generated from scratch)


>
> Björn
>
>


Re: TLS tickets prone to MITM attacks (was: [PR] Docs tls tickets)

2020-03-09 Thread Илья Шипицин
пн, 9 мар. 2020 г. в 23:59, Willy Tarreau :

> On Mon, Mar 09, 2020 at 07:18:23PM +0100, Björn Jacke wrote:
> > On 2020-03-09 at 17:44 +0100 Lukas Tribus sent off:
> > > Perhaps we can relax the wording a bit here and describe the actual
> > > technical issue along with some recommendations. Apache for example
> > > documents [1]:
> >
> > I think the wording from the patch is still quite relaxed :). One of the
> best
> > summaries describing the session ticket flaws, which I recommend is this:
> > https://blog.filippo.io/we-need-to-talk-about-session-tickets/
>
> So as explained in the article above, all of this is only a problem if keys
> are *never* rotated. New keys are used every time the process is reloaded,
> and those using distributed deployments update them via the "set ssl
> tls-key"
> command on the CLI which purposely supports old and new key in order to
> maintain a distributed state synchronized yet up to date.
>


maybe it worth separate discussion...

there was some discussion on nginx mailing list regarding BoringSSL and TLS
tickets

(sorry for link to discussion in russian)
https://forum.nginx.org/read.php?21,286846

in short, BoringSSL in 2017 has changed its behaviour related to TLS
tickets:

https://boringssl.googlesource.com/boringssl/+/72912d2

in short, it is ok if you specify TLS ticket key in file, and it simply
does not work (i.e. works only for master process) if you do not specify.
i.e. each process
has its own TLS ticket key. and user has no way to choose web worker
process.

seems, it might be related to any multi process web server, not only nginx.

are there some tests in haproxy to cover such boringssl behaviour ?



>
> > I would disable session tickets by default in haproxy. Given that most
> > clients support TLS 1.3 already this change would not even slow down many
> > clients.
>
> The problem is not as much the client slowing down as killing the servers
> by the excess of handshakes caused by the minority of remaining clients,
> because with this the service easily becomes totally unavailable for
> everyone depending on the population of clients. Not everyone has a lot
> of up-to-date browsers in their clients, and a number of deployments
> actually never see a browser at all.
>
> Willy
>
>


Re: TLS tickets prone to MITM attacks (was: [PR] Docs tls tickets)

2020-03-11 Thread Илья Шипицин
ср, 11 мар. 2020 г. в 04:09, Björn Jacke :

> On 09.03.20 20:37, Lukas Tribus wrote:
> >> I think the wording from the patch is still quite relaxed :). One of
> the best
> >> summaries describing the session ticket flaws, which I recommend is
> this:
> >> https://blog.filippo.io/we-need-to-talk-about-session-tickets/
> > Nothing about this is a MITM attack. The point in the article is that
> > when the ticket-key is compromised, captured traffic can be passively
> > decrypted (which is what broken PFS means, as explained by the Apache
> > docs).
>
> take also this article, which clearly states that session tickets are
> vulnerable to replay attacks (which are a kind of MITM):
> https://eprint.iacr.org/2019/228.pdf



major players of a big picture are 0RTT and session tickets.
indeed, if you wish to fight replay attack, you cannot use 0RTT (also, you
are supposed to maintain keys rotation).

as for keys rotation, for unfamiliar people it is not clear why haproxy
itself does not provide such rotation.
at least, it should be better documented.


>
>
> That paper also says in clear words:
>
> "The standard techniques to achieve [session resumption] are Session
> Caches or, alternatively, Session Tickets. The former provides forward
> security and resistance against replay attacks, but requires a large
> amount of server-side storage. The latter requires negligible storage,
> but provides no forward security and is known to be vulnerable to replay
> attacks"
>
> I have to admit that I was also not aware that haproxy does not rotate
> keys by itself.
>
>
I guess most people do not realise that.


> As writtenhere before I also think that session tickets should really be
> disabled by default and the documentation should point out the risks and
> possible attacks. imho simple tls session caching should be enough for
> "older"  TLS>=1.2 clients for security reasons and address the
> performance concerns. A secure setting should be a default at this
> point. If users prefer more (but less secure) performance improvement
> features, which even require additional work to make it less insecure
> (key rotation), than this should have to be enabled manually only I think.
>
> Key rotation, if enabled, should ideally not be a matter of manual
> action by the administrator but key rotation should happen automatically
> after a reasonable amount of time. A note in the documentation on the
> no-tls-tickets about this is just not enough, especially if session
> tickets stay enabled by default.
>

key rotation when happens trigger a lot of client renegotiation. and it is
observable on
haproxy reload, there are huge CPU spikes. I think most people do not
realise why those spikes happen.


>
> Björn
>
>


Re: TLS tickets prone to MITM attacks (was: [PR] Docs tls tickets)

2020-03-10 Thread Илья Шипицин
вт, 10 мар. 2020 г. в 05:37, Lukas Tribus :

> Hello,
>
>
> On Mon, 9 Mar 2020 at 20:39, Илья Шипицин  wrote:
> >> I would disable session tickets by default in haproxy. Given that most
> >> clients support TLS 1.3 already this change would not even slow down
> many
> >> clients.
> >
> >
> > TLS tickets really require more love :)
> >
> > actually, there are two bad choices here
> >
> > 1) to specify TLS ticket key
> > 2) not to specify
> >
> > if you specify, your security team will tell you that "it is not secure".
> > if you do not specify, keys are generated on startup and it lead to huge
> CPU spike on app reload (if you apply new config, app is reloaded and keys
> are generated from scratch)
>
> You rotate ticket keys with "tls-ticket-keys" and reload or you can
> even add new ticket keys online via the "set ssl tls-key" unix socket
> command. But you have to take care of it if you want ticket key
> rotation.
>

it is clear for educated people.
but documentation is written for non educated, right :) ?



>
> A performant, scalable and fully secure TLS setup is a non-trivial
> configuration to make. Probably not possible without compromises in a
> production environment with real world clients, but that is just how
> it is. Regarding security aspects the actual attack surface has to be
> considered. This attack requires the intruder to read the TLS ticket
> key from memory, this kills the concept of Forward Secrecy, but it
> does not kill anything else (that would not already be compromised
> anyway when an attacker can read from memory).
>
>
> Lukas
>


[PATCH[ special purpose weekly CI (spellcheck)

2020-03-10 Thread Илья Шипицин
Hello,

I implemented spell check CI.
it is a bit noisy, however, I hope we will polish it soon.

Cheers,
Ilya Shipitcin
From b58cc1f207649deea6d1053d62d311501e775c5e Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 10 Mar 2020 12:10:26 +0500
Subject: [PATCH] BUILD: add spellcheck github action

action is self consistent. it is scheduled to run weekly
---
 .github/workflows/codespell.yml | 17 +
 1 file changed, 17 insertions(+)
 create mode 100644 .github/workflows/codespell.yml

diff --git a/.github/workflows/codespell.yml b/.github/workflows/codespell.yml
new file mode 100644
index 0..32d5898f7
--- /dev/null
+++ b/.github/workflows/codespell.yml
@@ -0,0 +1,17 @@
+name: codespell
+
+on: 
+  schedule:
+- cron: "0 0 * * 2" 
+
+jobs:
+  codespell:
+
+runs-on: ubuntu-latest
+
+steps:
+- uses: actions/checkout@v2
+- name: install prerequisites
+  run: sudo pip install codespell
+- name: check
+  run: codespell -c -q 2 --ignore-words-list ist,hist,wan,ca,cas --skip CHANGELOG
-- 
2.24.1



[PATCH] cleanup: typo fixes in code comments

2020-03-10 Thread Илья Шипицин
Hello,

another part of typo cleanup.

Cheers,
Ilya Shipitcin
From 4384a8ad97c3b4f24b17f58e411351ac51a7cff9 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 10 Mar 2020 12:06:11 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

---
 include/common/defaults.h |  6 ++--
 include/common/h1.h   |  2 +-
 include/common/hpack-tbl.h|  2 +-
 include/common/http.h |  2 +-
 include/common/htx.h  | 12 +++
 include/common/ist.h  |  2 +-
 include/common/regex.h|  4 +--
 include/common/standard.h | 16 -
 include/common/xref.h |  2 +-
 include/proto/connection.h|  2 +-
 include/proto/fd.h|  2 +-
 include/proto/log.h   |  4 +--
 include/proto/pattern.h   |  2 +-
 include/proto/proxy.h |  2 +-
 include/proto/server.h|  2 +-
 include/proto/sink.h  |  2 +-
 include/proto/task.h  | 10 +++---
 include/types/filters.h   |  6 ++--
 include/types/pattern.h   |  6 ++--
 include/types/protocol.h  |  2 +-
 reg-tests/checks/4be_1srv_health_checks.vtc   |  2 +-
 reg-tests/http-errorfiles/errors/500-1.http   |  2 +-
 reg-tests/http-errorfiles/errors/500.http |  2 +-
 .../http-errorfiles/http_deny_errors.vtc  |  2 +-
 reg-tests/mailers/healthcheckmail.lua |  2 +-
 reg-tests/spoe/wrong_init.vtc |  2 +-
 src/ssl_sock.c| 34 +--
 27 files changed, 67 insertions(+), 67 deletions(-)

diff --git a/include/common/defaults.h b/include/common/defaults.h
index e86c9bce1..43d287d76 100644
--- a/include/common/defaults.h
+++ b/include/common/defaults.h
@@ -111,7 +111,7 @@
 #define MAX_SESS_STKCTR 3
 #endif
 
-// max # of extra stick-table data types that can be registred at runtime
+// max # of extra stick-table data types that can be registered at runtime
 #ifndef STKTABLE_EXTRA_DATA_TYPES
 #define STKTABLE_EXTRA_DATA_TYPES 0
 #endif
@@ -153,14 +153,14 @@
 #define RUNQUEUE_DEPTH 200
 #endif
 
-// cookie delimitor in "prefix" mode. This character is inserted between the
+// cookie delimiter in "prefix" mode. This character is inserted between the
 // persistence cookie and the original value. The '~' is allowed by RFC6265,
 // and should not be too common in server names.
 #ifndef COOKIE_DELIM
 #define COOKIE_DELIM'~'
 #endif
 
-// this delimitor is used between a server's name and a last visit date in
+// this delimiter is used between a server's name and a last visit date in
 // cookies exchanged with the client.
 #ifndef COOKIE_DELIM_DATE
 #define COOKIE_DELIM_DATE   '|'
diff --git a/include/common/h1.h b/include/common/h1.h
index 5ed26fa8f..fc28b4808 100644
--- a/include/common/h1.h
+++ b/include/common/h1.h
@@ -196,7 +196,7 @@ static inline const char *h1m_state_str(enum h1m_state msg_state)
  * returns the number of bytes parsed on success, so the caller can set msg_state
  * to HTTP_MSG_CHUNK_SIZE. If not enough data are available, the function does not
  * change anything and returns zero. Otherwise it returns a negative value
- * indicating the error positionn relative to . Note: this function is
+ * indicating the error position relative to . Note: this function is
  * designed to parse wrapped CRLF at the end of the buffer.
  */
 static inline int h1_skip_chunk_crlf(const struct buffer *buf, int start, int stop)
diff --git a/include/common/hpack-tbl.h b/include/common/hpack-tbl.h
index ca3f2aa96..bb1ac06e1 100644
--- a/include/common/hpack-tbl.h
+++ b/include/common/hpack-tbl.h
@@ -52,7 +52,7 @@
  * possible to maximize the delay before a collision of DTEs and data. In order
  * to always insert from the right, we need to keep a reference to the latest
  * inserted element and look before it. The last inserted cell's address defines
- * the lowest konwn address still in use, unless the area wraps in which case
+ * the lowest known address still in use, unless the area wraps in which case
  * the available space lies between the end of the tail and the beginning of the
  * head.
  *
diff --git a/include/common/http.h b/include/common/http.h
index d3519bc11..6f083d435 100644
--- a/include/common/http.h
+++ b/include/common/http.h
@@ -27,7 +27,7 @@
 #include 
 
 /*
- * some macros mainly used when parsing header fileds.
+ * some macros mainly used when parsing header fields.
  * from RFC7230:
  *   CTL = 
  *   SEP = one of the 17 defined separators or SP or HT
diff --git a/include/common/htx.h b/include/common/htx.h
index 9818a3f4d..7bd12b407 100644
--- a/include/common/htx.h
+++ b/include/common/htx.h
@@ -37,7 +37,7 @@
  * 

Re: Tests timeout on my ARM64 test VM

2020-03-11 Thread Илья Шипицин
I will a look during next weekend

BTW, I've managed to get Linaro VM :)

On Wed, Mar 11, 2020, 5:40 PM Martin Grigorov 
wrote:

> Hi,
>
> On Mon, Mar 9, 2020 at 10:22 PM Martin Grigorov 
> wrote:
>
>> Hi,
>>
>> I am not sure what have changed on my test ARM64 VM but the reg tests
>> started timing out.
>> Everything is fine on my dev machine (x86_64) and at Travis (
>> https://travis-ci.com/haproxy/haproxy).
>> I don't think it is ARM64 related. Most probably some OS setting or
>> something.
>> I've rebooted the system just to make sure it is not some busy port or
>> opened file descriptor but
>> it still fails the same way.
>>
>> Does someone see in the attached logs what could be the problem?
>>
>
> Anyone can help me here ?
>
> Martin
>
>
>> Thank you!
>>
>> Martin
>>
>


[PATCH] 3rd round of documentation typo fixes

2020-03-06 Thread Илья Шипицин
Hello,

I attached a patch based on documentation spelcheck.

Cheers,
Ilya Shipitcin
From ad38d6c0819df867524338a55ac51b3ed8e510be Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Fri, 6 Mar 2020 23:22:22 +0500
Subject: [PATCH] DOC: assorted typo fixes in the documentation

This is the third round of cleanups in various docs
---
 BRANCHES|  2 +-
 doc/design-thoughts/entities-v2.txt |  2 +-
 doc/design-thoughts/http2.txt   |  4 +--
 doc/internals/buffer-api.txt|  2 +-
 doc/internals/entities-v2.txt   |  2 +-
 doc/internals/entities.txt  | 16 +--
 doc/internals/filters.txt   | 16 +--
 doc/internals/htx-api.txt   | 10 +++
 doc/internals/notes-layers.txt  |  6 ++--
 doc/lua-api/index.rst   | 44 ++---
 doc/lua.txt |  4 +--
 11 files changed, 54 insertions(+), 54 deletions(-)

diff --git a/BRANCHES b/BRANCHES
index 521c0ee1f..6cb275c94 100644
--- a/BRANCHES
+++ b/BRANCHES
@@ -191,7 +191,7 @@ Thus the release cycle from 1.8 to 2.2 should look like this:
  `---+---+---+-+---+---+---+--> 1.8 LTS
 
 In short the non-LTS odd releases can be seen as technological previews of the
-next feature release, and will be terminated much ealier. The plan is to barely
+next feature release, and will be terminated much earlier. The plan is to barely
 let them overlap with the next non-LTS release, allowing advanced users to
 always have the choice between the last two major releases.
 
diff --git a/doc/design-thoughts/entities-v2.txt b/doc/design-thoughts/entities-v2.txt
index 905888e22..91c4fa97e 100644
--- a/doc/design-thoughts/entities-v2.txt
+++ b/doc/design-thoughts/entities-v2.txt
@@ -119,7 +119,7 @@ Identified handshake handlers for incoming connections :
   - HH_ACCEPT_PROXY (waits for PROXY line and parses it)
   - HH_TCP_RULES (processes TCP rules)
   - HH_SSL_HS (starts SSL handshake)
-  - HH_ACCEPT_SESSION (instanciates a session)
+  - HH_ACCEPT_SESSION (instantiates a session)
 
 Identified handshake handlers for outgoing connections :
   - HH_SEND_PROXY (tries to build and send the PROXY line)
diff --git a/doc/design-thoughts/http2.txt b/doc/design-thoughts/http2.txt
index 20a5c54c4..c21ac108a 100644
--- a/doc/design-thoughts/http2.txt
+++ b/doc/design-thoughts/http2.txt
@@ -97,7 +97,7 @@
   before the connection timeout so that an unused connection is verified before
   being killed. Abnormal requests must be dealt with using RST_STREAM.
 
-- ALPN : ALPN must be observed onthe client side, and transmitted to the server
+- ALPN : ALPN must be observed on the client side, and transmitted to the server
   side.
 
 - proxy protocol : proxy protocol makes little to no sense in a multiplexed
@@ -134,7 +134,7 @@
   to H2 behind. This can cause some trouble when passing H2 requests to H1
   proxies, because there's no way to know if the request should contain scheme
   and authority in H1 or not based on the H2 request. Thus a "proxy" option
-  will have to be explicitly mentionned on HTTP/1 server lines. One of the
+  will have to be explicitly mentioned on HTTP/1 server lines. One of the
   problem that it creates is that it's not longer possible to pass H/1 requests
   to H/1 proxies without an explicit configuration. Maybe a table of the
   various combinations is needed.
diff --git a/doc/internals/buffer-api.txt b/doc/internals/buffer-api.txt
index 6d192c19d..14a1ac77f 100644
--- a/doc/internals/buffer-api.txt
+++ b/doc/internals/buffer-api.txt
@@ -630,7 +630,7 @@ but fail.
<--> <-->
 oi
 
-There is this correspondance between old and new fields (some will involve a
+There is this correspondence between old and new fields (some will involve a
 knowledge of a channel when the output byte count is required) :
 
  Old| New
diff --git a/doc/internals/entities-v2.txt b/doc/internals/entities-v2.txt
index 38d633d20..86782c34d 100644
--- a/doc/internals/entities-v2.txt
+++ b/doc/internals/entities-v2.txt
@@ -116,7 +116,7 @@ Identified handshake handlers for incoming connections :
   - HH_ACCEPT_PROXY (waits for PROXY line and parses it)
   - HH_TCP_RULES (processes TCP rules)
   - HH_SSL_HS (starts SSL handshake)
-  - HH_ACCEPT_SESSION (instanciates a session)
+  - HH_ACCEPT_SESSION (instantiates a session)
 
 Identified handshake handlers for outgoing connections :
   - HH_SEND_PROXY (tries to build and send the PROXY line)
diff --git a/doc/internals/entities.txt b/doc/internals/entities.txt
index d384395f1..cdde82e34 100644
--- a/doc/internals/entities.txt
+++ b/doc/internals/entities.txt
@@ -9,7 +9,7 @@ Listener
 
 A listener is the entity which is part of a frontend and which accepts
 connections. There are as many listeners as there are ip:port couples.
-There is at least one listener instanciated for each "bind" entry, and
+There 

[PATCH] fourth round of typo fixes

2020-03-14 Thread Илья Шипицин
Hello,

more documentation fixes.

Ilya Shipitcin
From de4a671e8530b36fea78749455c1dd868c986522 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Sat, 14 Mar 2020 17:47:28 +0500
Subject: [PATCH] DOC: assorted typo fixes in the documentation

This is the fourth round of cleanups in various docs
---
 contrib/modsecurity/README |  8 
 contrib/prometheus-exporter/README |  8 
 contrib/spoa_server/README |  2 +-
 doc/internals/initcalls.txt| 10 +-
 4 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/contrib/modsecurity/README b/contrib/modsecurity/README
index 8031389db..8e74016fc 100644
--- a/contrib/modsecurity/README
+++ b/contrib/modsecurity/README
@@ -1,7 +1,7 @@
 ModSecurity for HAProxy
 ---
 
-This is a third party deamon which speaks SPOE. It gives requests send by HAProxy
+This is a third party daemon which speaks SPOE. It gives requests send by HAProxy
 to ModSecurity and returns the verdict.
 
   Compilation
@@ -24,8 +24,8 @@ the Apache dependencies are installed on the system.
 	cp standalone/*.h $PWD/INSTALL/include
 	cp apache2/*.h $PWD/INSTALL/include
 
-Note that this compilation method works, but is a litle bit rustic. I can't
-deal with Lua, I supposed that is a dependecies problem on my computer.
+Note that this compilation method works, but is a little bit rustic. I can't
+deal with Lua, I supposed that is a dependencies problem on my computer.
 
   Start the service
 -
@@ -113,7 +113,7 @@ Modsecurity bugs:
-rc = apr_global_mutex_create(>auditlog_lock, NULL, APR_LOCK_DEFAULT, mp);
+rc = apr_global_mutex_create(>auditlog_lock, NULL, APR_LOCK_PROC_PTHREAD, mp);
 
-* Configuration file loaded with wilcard (eg. Include rules/*.conf), are loaded
+* Configuration file loaded with wildcard (eg. Include rules/*.conf), are loaded
   in reverse alphabetical order. You can found a patch below. The ModSecurity
   team ignored this patch.
 
diff --git a/contrib/prometheus-exporter/README b/contrib/prometheus-exporter/README
index a63102028..1c5a99241 100644
--- a/contrib/prometheus-exporter/README
+++ b/contrib/prometheus-exporter/README
@@ -4,14 +4,14 @@ PROMEX: A Prometheus exporter for HAProxy
 Prometheus is a monitoring and alerting system. More and more people use it to
 monitor their environment (this is written February 2019). It collects metrics
 from monitored targets by scraping metrics HTTP endpoints on these targets. For
-HAProxy, The Prometheus team offically supports an exporter written in Go
+HAProxy, The Prometheus team officially supports an exporter written in Go
 (https://github.com/prometheus/haproxy_exporter). But it requires an extra
 software to deploy and monitor. PROMEX, on its side, is a built-in Prometheus
 exporter for HAProxy. It was developed as a service and is directly available in
 HAProxy, like the stats applet.
 
 However, PROMEX is not built by default with HAProxy. It is provided as an extra
-component for everyone want to use it. So you need to explicity build HAProxy
+component for everyone want to use it. So you need to explicitly build HAProxy
 with the PROMEX service, using the Makefile variable "EXTRA_OBJS". For instance:
 
 > make TARGET=linux-glibc EXTRA_OBJS="contrib/prometheus-exporter/service-prometheus.o"
@@ -46,7 +46,7 @@ applet, all metrics are not grouped by service (proxy, listener or server). With
 PROMEX, all lines for a given metric are provided as one single group. So
 instead of collecting all metrics for a proxy before moving to the next one, we
 must loop on all proxies for each metric. Same for the servers. Thus, it will
-spend much more ressources to produce the Prometheus metrics than the CSV export
+spend much more resources to produce the Prometheus metrics than the CSV export
 through the stats page. To give a comparison order, quick benchmarks shown that
 a PROMEX dump is 5x slower and 20x more verbose than a CSV export.
 
@@ -99,7 +99,7 @@ Exported metrics
 | haproxy_process_pool_used_bytes| Total amount of memory used in pools (in bytes).  |
 | haproxy_process_pool_failures_total| Total number of failed pool allocations.  |
 | haproxy_process_max_fds| Maximum number of open file descriptors; 0=unset. |
-| haproxy_process_max_sockets| Maximum numer of open sockets.|
+| haproxy_process_max_sockets| Maximum number of open sockets.   |
 | haproxy_process_max_connections| Maximum number of concurrent connections. |
 | haproxy_process_hard_max_connections   | Initial Maximum number of concurrent connections. |
 | haproxy_process_current_connections| 

Re: [PATCH] switch to clang-9 in Linux/travis-ci builds

2020-03-16 Thread Илья Шипицин
пн, 16 мар. 2020 г. в 13:40, Willy Tarreau :

> On Mon, Mar 16, 2020 at 12:51:20PM +0500,  ??? wrote:
> > From 5d5891166389dc03ab4fb63ca9edaa35feca8fcc Mon Sep 17 00:00:00 2001
> > From: Ilya Shipitsin 
> > Date: Mon, 16 Mar 2020 12:49:34 +0500
> > Subject: [PATCH] CI: switch BoringSSL back to clang-7
> >
> > it turned out that BoringSSL is not prepared for clang-9
> > https://bugs.chromium.org/p/boringssl/issues/detail?id=323
> >
> > until that issue is resolved, let us roll back to clang-7
> (...)
> > -env: TARGET=linux-glibc BORINGSSL=yes CC=clang-9
> > +env: TARGET=linux-glibc BORINGSSL=yes CC=clang
>
> I already reverted that part however I switched back to not defining CC
> as before the change. Is it OK ?
>

sure


>
> Willy
>


Re: [PATCH] switch to clang-9 in Linux/travis-ci builds

2020-03-16 Thread Илья Шипицин
пн, 16 мар. 2020 г. в 14:54, Willy Tarreau :

> On Mon, Mar 16, 2020 at 10:49:26AM +0100, Tim Düsterhus wrote:
> > Ilya,
> >
> > Am 16.03.20 um 07:52 schrieb  ???:
> > > we use clang because of its address sanitizer. I found gcc asan more
> noisy
> > > and less usable.
> >
> > I believe you broke ASAN with clang-9 anyway, because the Travis
> > configuration checks for 'clang':
> >
> https://github.com/haproxy/haproxy/blob/67b095e797a156ae27b7b52f6ccf57171717dd16/.travis.yml#L108
> >
> > It probably needs to read `if [ "$CC"  = "clang*" ]` (unless I got my
> > bash syntax wrong).
>
> Indeed. However it should be:
>
>  if [ "${CC%-*}" = "clang" ]
>

it did not indicate any error. so it looked ok :)


>
> I'd like to focus on what's still broken (i.e. the abns seamless reload
> test)
> before adding more noise. It ought to be fixed but it is still random, it
> even failed once on amd64. I definitely need to be able to reproduce it, as
> I was certain the two recent bugs fixed were the only ones responsible for
> this.
>

sure


>
> Willy
>


Re: HAproxy Error

2020-04-15 Thread Илья Шипицин
ср, 15 апр. 2020 г. в 16:41, :

> Hi Team,
>
>
>
> We are in the process of using newer HAproxy version.
>
> Below is the scenario explained where we are stuck.
>
>
>
> · In RHEL 8.1 version, installed the latest version of the
> application.
>
> haproxy-1.8.15-6.el8_1.1.x86_64
>
> # yum info haproxy
>
> Red Hat Update Infrastructure 3 Client Configuration Server
> 8
> 12 kB/s | 2.1 kB 00:00
>
> Red Hat Enterprise Linux 8 for x86_64 - AppStream from RHUI
> (RPMs)
>   24 kB/s | 2.8 kB 00:00
>
> Red Hat Enterprise Linux 8 for x86_64 - BaseOS from RHUI
> (RPMs)
> 21 kB/s | 2.4 kB 00:00
>
> Installed Packages
>
> Name : haproxy
>
> Version  : 1.8.15
>
> Release  : 6.el8_1.1
>
> Architecture : x86_64
>
> Size : 4.4 M
>
> Source   : haproxy-1.8.15-6.el8_1.1.src.rpm
>
> Repository   : @System
>
> From repo: rhel-8-appstream-rhui-rpms
>
> Summary  : HAProxy reverse proxy for high availability environments
>
> URL  : http://www.haproxy.org/
>
> License  : GPLv2+
>
> Description  : HAProxy is a TCP/HTTP reverse proxy which is particularly
> suited for high
>
>  : availability environments. Indeed, it can:
>
>  :  - route HTTP requests depending on statically assigned
> cookies
>
>  :  - spread load among several servers while assuring server
> persistence
>
>  :through the use of HTTP cookies
>
>  :  - switch to backup servers in the event a main one fails
>
>  :  - accept connections to special ports dedicated to service
> monitoring
>
>  :  - stop accepting connections without breaking existing ones
>
>  :  - add, modify, and delete HTTP headers in both directions
>
>  :  - block requests matching particular patterns
>
>  :  - report detailed status to authenticated users from a URI
>
>  :intercepted from the application
>
>
>
> · So after this configuration file is updated with our
> configurations. We need to use multiple certificates (SNI) hence used bind
> option to verify the certificates under the folder. But receiving below
> error please help us on priority.
>
> Attached configuration file for reference.
>
> Below is the error we are receiving.
>
>
>
> # haproxy -f /etc/haproxy/haproxy.cfg -c
>
> [ALERT] 105/113215 (5684) : parsing [/etc/haproxy/haproxy.cfg:33] : 'bind
> *:443' unknown keyword 'ssl'. Registered keywords :
>
> [STAT] level 
>
> [STAT] expose-fd 
>
> [STAT] severity-output 
>
> [ TCP] defer-accept
>
> [ TCP] interface 
>
> [ TCP] mss 
>
> [ TCP] tcp-ut 
>
> [ TCP] tfo
>
> [ TCP] transparent
>
> [ TCP] v4v6
>
> [ TCP] v6only
>
> [ TCP] namespace 
>
> [ ALL] accept-netscaler-cip 
>
> [ ALL] accept-proxy
>
> [ ALL] backlog 
>
> [ ALL] id 
>
> [ ALL] maxconn 
>
> [ ALL] name 
>
> [ ALL] nice 
>
> [ ALL] process 
>
> [ ALL] proto 
>
> [UNIX] gid 
>
> [UNIX] group 
>
> [UNIX] mode 
>
> [UNIX] uid 
>
> [UNIX] user 
>
> [ALERT] 105/113215 (5684) : parsing [/etc/haproxy/haproxy.cfg:36] : error
> detected while parsing an 'http-request set-header' condition : unknown
> fetch method 'ssl_fc' in ACL expression 'ssl_fc'.
>
> [ALERT] 105/113215 (5684) : Error(s) found in configuration file :
> /etc/haproxy/haproxy.cfg
>
> Please check the error and configuration and let me know what needs to
> done to fix the issue.
>
>
>
>
>
>
>
> Thanks,
>
> Bindushree D B
>
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. If you are not the intended recipient(s), please reply to the
> sender and destroy all copies of the original message. Any unauthorized
> review, use, disclosure, dissemination, forwarding, printing or copying of
>

hello.
should we destroy this message ?



> this email, and/or any action taken in reliance on the contents of this
> e-mail is strictly prohibited and may be unlawful. Where permitted by
> applicable law, this e-mail and other e-mail communications sent to and
> from Cognizant e-mail addresses may be monitored. This e-mail and any files
> transmitted with it are for the sole use of the intended recipient(s) and
> may contain confidential and privileged information. If you are not the
> intended recipient(s), please reply to the sender and destroy all copies of
> the original message. Any unauthorized review, use, disclosure,
> dissemination, forwarding, printing or copying of this email, and/or any
> action taken in reliance on the contents of this e-mail is strictly
> prohibited and may be unlawful. Where permitted by applicable law, this
> e-mail and other e-mail communications sent to and from Cognizant e-mail
> addresses may be monitored.
>


[PATCH] CI: special purpose build, testing compatibility against "no-deprecated" openssl

2020-04-16 Thread Илья Шипицин
Hello,

I added weekly build for detection incompatibilities against
"no-deprecated" openssl.

(well, I first thought to add those option to travis, but it became
over-engineered from my point of view)

Lukas, if you have suggestions how to add to travis, I can try.

Cheers,
Ilya Shipitsin
From ccb2e27d4566581196a139ccacdaecac097be2c1 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 16 Apr 2020 23:10:45 +0500
Subject: [PATCH] CI: run weekly OpenSSL "no-deprecated" builds

OpenWRT uses such OpenSSL builds (those builds are smaller)

some details might be found at ML: https://www.mail-archive.com/haproxy@formilux.org/msg35759.html
   GH: https://github.com/haproxy/haproxy/issues/367
---
 .github/workflows/openssl-nodeprecated.yml | 33 ++
 1 file changed, 33 insertions(+)
 create mode 100644 .github/workflows/openssl-nodeprecated.yml

diff --git a/.github/workflows/openssl-nodeprecated.yml b/.github/workflows/openssl-nodeprecated.yml
new file mode 100644
index 0..758ccbcc9
--- /dev/null
+++ b/.github/workflows/openssl-nodeprecated.yml
@@ -0,0 +1,33 @@
+#
+# special purpose CI: test against OpenSSL built in "no-deprecated" mode
+# let us run those builds weekly
+#
+# for example, OpenWRT uses such OpenSSL builds (those builds are smaller)
+#
+#
+# some details might be found at NL: https://www.mail-archive.com/haproxy@formilux.org/msg35759.html
+#GH: https://github.com/haproxy/haproxy/issues/367
+
+name: openssl no-deprecated
+
+on:
+  schedule:
+- cron: "0 0 * * 4"
+
+jobs:
+  test:
+
+runs-on: ubuntu-latest
+
+steps:
+- uses: actions/checkout@v1
+- name: prepare VTest
+  run: |
+git clone https://github.com/VTest/VTest.git ../vtest
+make -C ../vtest FLAGS="-O2 -s -Wall"
+- name: build haproxy
+  run: |
+make DEFINE="-DOPENSSL_API_COMPAT=0x1010L -DOPENSSL_NO_DEPRECATED" -j3 CC=gcc ERR=1 TARGET=linux-glibc USE_OPENSSL=1
+- name: run reg-tests
+  run: |
+make reg-tests VTEST_PROGRAM=../vtest/vtest
-- 
2.25.2



[PATCH] 8th iteration of typo fixes

2020-04-16 Thread Илья Шипицин
Hello,

next typo fixes.


Ilya Shipitcin
From 40a7e81223c783836d6e7d3726ad3db2f9ca418e Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Thu, 16 Apr 2020 23:51:34 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

This is 8th iteration of typo fixes
---
 include/proto/fd.h |  2 +-
 include/types/connection.h | 12 -
 include/types/fd.h |  2 +-
 include/types/listener.h   |  2 +-
 include/types/ssl_sock.h   |  2 +-
 include/types/stream.h |  2 +-
 scripts/backport   |  2 +-
 scripts/run-regtests.sh|  2 +-
 src/hlua.c | 54 +++---
 src/log.c  | 10 +++
 src/standard.c | 14 +-
 src/stats.c|  4 +--
 src/task.c |  2 +-
 13 files changed, 55 insertions(+), 55 deletions(-)

diff --git a/include/proto/fd.h b/include/proto/fd.h
index 9783f9072..884843719 100644
--- a/include/proto/fd.h
+++ b/include/proto/fd.h
@@ -310,7 +310,7 @@ static inline void fd_want_send(int fd)
 }
 
 /* Set the fd as currently running on the current thread.
- * Retuns 0 if all goes well, or -1 if we no longer own the fd, and should
+ * Returns 0 if all goes well, or -1 if we no longer own the fd, and should
  * do nothing with it.
  */
 static inline int fd_set_running(int fd)
diff --git a/include/types/connection.h b/include/types/connection.h
index 98f763836..8780590d8 100644
--- a/include/types/connection.h
+++ b/include/types/connection.h
@@ -253,7 +253,7 @@ enum {
 	CO_ER_SSL_TIMEOUT,  /* timeout during SSL handshake */
 	CO_ER_SSL_TOO_MANY, /* too many SSL connections */
 	CO_ER_SSL_NO_MEM,   /* no more memory to allocate an SSL connection */
-	CO_ER_SSL_RENEG,/* forbidden client renegociation */
+	CO_ER_SSL_RENEG,/* forbidden client renegotiation */
 	CO_ER_SSL_CA_FAIL,  /* client cert verification failed in the CA chain */
 	CO_ER_SSL_CRT_FAIL, /* client cert verification failed on the certificate */
 	CO_ER_SSL_MISMATCH, /* Server presented an SSL certificate different from the configured one */
@@ -345,7 +345,7 @@ enum mux_ctl_type {
 /* mux_ops describes the mux operations, which are to be performed at the
  * connection level after data are exchanged with the transport layer in order
  * to propagate them to streams. The  function will automatically be
- * called once the mux is instanciated by the connection's owner at the end
+ * called once the mux is instantiated by the connection's owner at the end
  * of a transport handshake, when it is about to transfer data and the data
  * layer is not ready yet.
  */
@@ -370,7 +370,7 @@ struct mux_ops {
 	void (*destroy)(void *ctx); /* Let the mux know one of its users left, so it may have to disappear */
 	void (*reset)(struct connection *conn); /* Reset the mux, because we're re-trying to connect */
 	const struct cs_info *(*get_cs_info)(struct conn_stream *cs); /* Return info on the specified conn_stream or NULL if not defined */
-	int (*ctl)(struct connection *conn, enum mux_ctl_type mux_ctl, void *arg); /* Provides informations about the mux */
+	int (*ctl)(struct connection *conn, enum mux_ctl_type mux_ctl, void *arg); /* Provides information about the mux */
 	int (*takeover)(struct connection *conn); /* Attempts to migrate the connection to the current thread */
 	unsigned int flags;   /* some flags characterizing the mux's capabilities (MX_FL_*) */
 	char name[8]; /* mux layer name, zero-terminated */
@@ -426,14 +426,14 @@ struct conn_stream {
 
 /*
  * This structure describes the info related to a conn_stream known by the mux
- * only but usefull for the upper layer.
+ * only but useful for the upper layer.
  * For now, only some dates and durations are reported. This structure will
  * envolved. But for now, only the bare minimum is referenced.
  */
 struct cs_info {
 	struct timeval create_date;  /* Creation date of the conn_stream in user date */
 	struct timeval tv_create;/* Creation date of the conn_stream in internal date (monotonic) */
-	long t_handshake;/* hanshake duration, -1 if never occurs */
+	long t_handshake;/* handshake duration, -1 if never occurs */
 	long t_idle; /* idle duration, -1 if never occurs */
 };
 
@@ -452,7 +452,7 @@ struct connection {
 	unsigned int flags;   /* CO_FL_* */
 	const struct protocol *ctrl;  /* operations at the socket layer */
 	const struct xprt_ops *xprt;  /* operations at the transport layer */
-	const struct mux_ops  *mux;   /* mux layer opreations. Must be set before xprt->init() */
+	const struct mux_ops  *mux;   /* mux layer operations. Must be set before xprt->init() */
 	void *xprt_ctx;   /* general purpose pointer, initialized to NULL */
 	void *ctx;/* highest level context (usually the mux), initialized to NULL */
 	void *owner;  /* pointer 

Re: HAProxy and musl (was: Re: HAproxy Error)

2020-04-16 Thread Илья Шипицин
чт, 16 апр. 2020 г. в 16:26, Willy Tarreau :

> On Thu, Apr 16, 2020 at 12:29:42PM +0200, Tim Düsterhus wrote:
> > > FWIW musl seems to work OK here when building for linux-glibc-legacy.
> >
> > Yes. HAProxy linked against Musl is smoke tested as part of the Docker
> > Official Images program, because the Alpine-based Docker images use Musl
> > as their libc. In fact you can even use TARGET=linux-glibc +
> USE_BACKTRACE=.
>
> By the way, I initially thought I was the only one building with musl
> for my EdgeRouter-x that I'm using as a distcc load balancer for the
> build farm at work. But if there are other users, we'd rather add
>

private buildfarm ? what do you use ? just curious

a linux-musl target, as the split between OS and library was precisely
> made for this purpose!
>
> Anyone objects against something like this (+ the appropriate entries
> in other places and doc) ?
>

looks good


>
>
> diff --git a/Makefile b/Makefile
> index d5841a5..a3dad36 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -341,6 +341,18 @@ ifeq ($(TARGET),linux-glibc-legacy)
>  USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP
> USE_GETADDRINFO)
>  endif
>
> +# For linux >= 2.6.28 and musl
> +ifeq ($(TARGET),linux-musl)
> +  set_target_defaults = $(call default_opts, \
> +USE_POLL USE_TPROXY USE_LIBCRYPT USE_DL USE_RT USE_CRYPT_H
> USE_NETFILTER  \
> +USE_CPU_AFFINITY USE_THREAD USE_EPOLL USE_FUTEX USE_LINUX_TPROXY
> \
> +USE_ACCEPT4 USE_LINUX_SPLICE USE_PRCTL USE_THREAD_DUMP USE_NS
> USE_TFO \
> +USE_GETADDRINFO)
> +ifneq ($(shell echo __arm__/__aarch64__ | $(CC) -E -xc - | grep
> '^[^\#]'),__arm__/__aarch64__)
> +  TARGET_LDFLAGS=-latomic
> +endif
> +endif
> +
>  # Solaris 8 and above
>  ifeq ($(TARGET),solaris)
># We also enable getaddrinfo() which works since solaris 8.
>
> Willy
>


[PATCH] CI: fix openssl download script

2020-04-07 Thread Илья Шипицин
Hello,

previous patch fixed builds for openssl-1.1.1f
however, links for download latest and previous releases are now different,
so I adopted script to handle older releases too.


Cheers,
Ilya Shipitcin
From c0023f9a7328980e7313c0f7ebfb6d355dab0ea9 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Tue, 7 Apr 2020 23:35:49 +0500
Subject: [PATCH] CI: adopt openssl download script to download all versions

with recent change, OpenSSL download URL was changed in
incompatiable way. i.e. only the most recent openssl version
might be downloaded using previous script.

older versions are available under different URLs. as we need
several openssl versions, let us adopt script accordingly.

bug was caught after travis-ci cache was purged for some reason.
---
 scripts/build-ssl.sh | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/scripts/build-ssl.sh b/scripts/build-ssl.sh
index 60b876c5d..e9d46bb76 100755
--- a/scripts/build-ssl.sh
+++ b/scripts/build-ssl.sh
@@ -3,8 +3,17 @@ set -eux
 
 download_openssl () {
 if [ ! -f "download-cache/openssl-${OPENSSL_VERSION}.tar.gz" ]; then
+
+#
+# OpenSSL has different links for latest and previous releases
+# since we want to download several versions, let us try to treat
+# current version as latest, if it fails, follow with previous
+# 
+
+	wget -P download-cache/ \
+	"https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz; || \
 wget -P download-cache/ \
-"https://www.openssl.org/source/openssl-${OPENSSL_VERSION}.tar.gz;
+"https://www.openssl.org/source/old/${OPENSSL_VERSION%[a-z]}/openssl-${OPENSSL_VERSION}.tar.gz;
 fi
 }
 
-- 
2.25.1



[PATCH] CI: rename "openssl111" to "openssl" after it is renamed in FreeBSD

2020-04-07 Thread Илья Шипицин
Hello,

this is required to get freebsd builds green again.

Cheers,
Ilya Shipitcin
From 0a50786e6b255f990af5f2c8bcc37c0d463a23fd Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 8 Apr 2020 00:29:26 +0500
Subject: [PATCH] CI: cirrus-ci: rename openssl package after it is renamed in
 FreeBSD

for the reason yet to be determined FreeBSD has renamed openssl111
to openssl. let us rename as well
---
 .cirrus.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 982773467..2d26952c0 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -4,7 +4,7 @@ FreeBSD_task:
   image_family: freebsd-12-1
   only_if: $CIRRUS_BRANCH =~ 'master|next'
   install_script:
-- pkg update -f && pkg upgrade -y && pkg install -y openssl111 git gmake lua53 socat
+- pkg update -f && pkg upgrade -y && pkg install -y openssl git gmake lua53 socat
   script:
 - git clone https://github.com/VTest/VTest.git ../vtest
 - make -C ../vtest
-- 
2.25.1



[PATCH] weekly typo fixes

2020-04-07 Thread Илья Шипицин
Hello,

7th iteration of typo fixes.

Cheers,
Ilya Shipitcin
From 51fa043677ee1783a2c3c92b37ff48ac7b21b647 Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Wed, 8 Apr 2020 01:07:56 +0500
Subject: [PATCH] CLEANUP: assorted typo fixes in the code and comments

This is 7th iteration of typo fixes
---
 ebtree/ebistree.c|  2 +-
 ebtree/ebistree.h|  4 ++--
 ebtree/ebsttree.c|  2 +-
 ebtree/ebsttree.h|  4 ++--
 include/types/acl.h  |  2 +-
 include/types/applet.h   |  4 ++--
 include/types/compression.h  |  2 +-
 include/types/dns.h  |  4 ++--
 include/types/hlua.h |  2 +-
 include/types/map.h  |  2 +-
 include/types/server.h   |  4 ++--
 include/types/spoe.h |  8 
 src/checks.c |  2 +-
 src/cli.c|  6 +++---
 src/hlua.c   | 32 
 src/http_htx.c   |  4 ++--
 src/stick_table.c|  2 +-
 tests/test-acl-args.cfg  |  2 +-
 tests/test-sample-fetch-args.cfg |  2 +-
 19 files changed, 45 insertions(+), 45 deletions(-)

diff --git a/ebtree/ebistree.c b/ebtree/ebistree.c
index 098ccfb43..6ad5e95f4 100644
--- a/ebtree/ebistree.c
+++ b/ebtree/ebistree.c
@@ -23,7 +23,7 @@
 #include "ebistree.h"
 
 /* Find the first occurrence of a zero-terminated string  in the tree .
- * It's the caller's reponsibility to use this function only on trees which
+ * It's the caller's responsibility to use this function only on trees which
  * only contain zero-terminated strings. If none can be found, return NULL.
  */
 struct ebpt_node *ebis_lookup(struct eb_root *root, const char *x)
diff --git a/ebtree/ebistree.h b/ebtree/ebistree.h
index 0c995483f..a438fa147 100644
--- a/ebtree/ebistree.h
+++ b/ebtree/ebistree.h
@@ -39,7 +39,7 @@ struct ebpt_node *ebis_lookup(struct eb_root *root, const char *x);
 struct ebpt_node *ebis_insert(struct eb_root *root, struct ebpt_node *new);
 
 /* Find the first occurrence of a length  string  in the tree .
- * It's the caller's reponsibility to use this function only on trees which
+ * It's the caller's responsibility to use this function only on trees which
  * only contain zero-terminated strings, and that no null character is present
  * in string  in the first  chars. If none can be found, return NULL.
  */
@@ -55,7 +55,7 @@ ebis_lookup_len(struct eb_root *root, const char *x, unsigned int len)
 }
 
 /* Find the first occurrence of a zero-terminated string  in the tree .
- * It's the caller's reponsibility to use this function only on trees which
+ * It's the caller's responsibility to use this function only on trees which
  * only contain zero-terminated strings. If none can be found, return NULL.
  */
 static forceinline struct ebpt_node *__ebis_lookup(struct eb_root *root, const void *x)
diff --git a/ebtree/ebsttree.c b/ebtree/ebsttree.c
index e44796ca6..12469cc1f 100644
--- a/ebtree/ebsttree.c
+++ b/ebtree/ebsttree.c
@@ -23,7 +23,7 @@
 #include "ebsttree.h"
 
 /* Find the first occurrence of a zero-terminated string  in the tree .
- * It's the caller's reponsibility to use this function only on trees which
+ * It's the caller's responsibility to use this function only on trees which
  * only contain zero-terminated strings. If none can be found, return NULL.
  */
 struct ebmb_node *ebst_lookup(struct eb_root *root, const char *x)
diff --git a/ebtree/ebsttree.h b/ebtree/ebsttree.h
index d1949c788..db2267bf1 100644
--- a/ebtree/ebsttree.h
+++ b/ebtree/ebsttree.h
@@ -33,7 +33,7 @@ struct ebmb_node *ebst_lookup(struct eb_root *root, const char *x);
 struct ebmb_node *ebst_insert(struct eb_root *root, struct ebmb_node *new);
 
 /* Find the first occurrence of a length  string  in the tree .
- * It's the caller's reponsibility to use this function only on trees which
+ * It's the caller's responsibility to use this function only on trees which
  * only contain zero-terminated strings, and that no null character is present
  * in string  in the first  chars. If none can be found, return NULL.
  */
@@ -49,7 +49,7 @@ ebst_lookup_len(struct eb_root *root, const char *x, unsigned int len)
 }
 
 /* Find the first occurrence of a zero-terminated string  in the tree .
- * It's the caller's reponsibility to use this function only on trees which
+ * It's the caller's responsibility to use this function only on trees which
  * only contain zero-terminated strings. If none can be found, return NULL.
  */
 static forceinline struct ebmb_node *__ebst_lookup(struct eb_root *root, const void *x)
diff --git a/include/types/acl.h b/include/types/acl.h
index 3d4428434..04318ea04 100644
--- a/include/types/acl.h
+++ b/include/types/acl.h
@@ -42,7 +42,7 @@
  *   - FAIL : no mattern may ever match
  *
  * We assign values 0, 1 and 3 to FAIL, MISS and PASS respectively, so that we
- * can make use of standard arithmetics for the truth tables 

Re: [PATCH] CI: special purpose build, testing compatibility against "no-deprecated" openssl

2020-04-21 Thread Илья Шипицин
nice, I finished all CI stuff :)

I'll focus in copr / rpm next

вт, 21 апр. 2020 г. в 13:29, William Lallemand :

> On Mon, Apr 20, 2020 at 07:12:41PM +0500, Илья Шипицин wrote:
> > Lukas, Willy ?
> >
> > чт, 16 апр. 2020 г. в 23:16, Илья Шипицин :
> >
> > > Hello,
> > >
> > > I added weekly build for detection incompatibilities against
> > > "no-deprecated" openssl.
> > >
> > > (well, I first thought to add those option to travis, but it became
> > > over-engineered from my point of view)
> > >
> > > Lukas, if you have suggestions how to add to travis, I can try.
> > >
> > > Cheers,
> > > Ilya Shipitsin
> > >
>
> Thanks Ilya, I merged it.
>
> --
> William Lallemand
>


Re: [ANNOUNCE] haproxy-2.1.4

2020-04-21 Thread Илья Шипицин
another option would be to enlist project at HackerOne and wait while Guido
Vranken will fuzz it :)

he already fuzzed dozens of projects, including openssl, openvpn, ...

https://guidovranken.com/

вт, 21 апр. 2020 г. в 18:21, Tim Düsterhus :

> Willy,
>
> Am 02.04.20 um 15:03 schrieb Willy Tarreau:
> > The main driver for this release is that it contains a fix for a serious
> > vulnerability that was responsibly reported last week by Felix Wilhelm
> > from Google Project Zero, affecting the HPACK decoder used for HTTP/2.
> > CVE-2020-11100 was assigned to this issue.
> >
> > There is no configuration-based workaround for 2.1 and above.
> >
> > This vulnerability makes it possible under certain circumstances to write
> > to a wide range of memory locations within the process' heap, with the
> > limitation that the attacker doesn't control the absolute address, so the
> > most likely result and by a far margin will be a process crash, but it is
> > not possible to completely rule out the faint possibility of a remote
> code
> > execution, at least in a lab-controlled environment. Felix was kind
> enough
> > to agree to delay the publication of his findings to the 20th of this
> month
> > in order to leave enough time to haproxy users to apply updates. But
> please
> > do not wait, as it is not very difficult to figure how to exploit the bug
> > based on the fix. Distros were notified and will also have fixes
> available
> > very shortly.
> >
>
> The write-up is available now:
> https://bugs.chromium.org/p/project-zero/issues/detail?id=2023
>
> It has a "Methodology-Fuzzing" label, so after CVE-2018-14645 and
> CVE-2018-20615 this is the 3rd CVE within H2 found using fuzzing that
> I'm aware of. It probably won't be the last. Can we please allocate some
> resources on making HAProxy more fuzzer friendly after 2.2 is out?
>
> I would also be interested in how Felix Wilhelm performed the fuzzing,
> do you happen to have details about that?
>
> Best regards
> Tim Düsterhus
>
>


Re: [ANNOUNCE] haproxy-2.1.4

2020-04-21 Thread Илья Шипицин
вт, 21 апр. 2020 г. в 18:21, Tim Düsterhus :

> Willy,
>
> Am 02.04.20 um 15:03 schrieb Willy Tarreau:
> > The main driver for this release is that it contains a fix for a serious
> > vulnerability that was responsibly reported last week by Felix Wilhelm
> > from Google Project Zero, affecting the HPACK decoder used for HTTP/2.
> > CVE-2020-11100 was assigned to this issue.
> >
> > There is no configuration-based workaround for 2.1 and above.
> >
> > This vulnerability makes it possible under certain circumstances to write
> > to a wide range of memory locations within the process' heap, with the
> > limitation that the attacker doesn't control the absolute address, so the
> > most likely result and by a far margin will be a process crash, but it is
> > not possible to completely rule out the faint possibility of a remote
> code
> > execution, at least in a lab-controlled environment. Felix was kind
> enough
> > to agree to delay the publication of his findings to the 20th of this
> month
> > in order to leave enough time to haproxy users to apply updates. But
> please
> > do not wait, as it is not very difficult to figure how to exploit the bug
> > based on the fix. Distros were notified and will also have fixes
> available
> > very shortly.
> >
>
> The write-up is available now:
> https://bugs.chromium.org/p/project-zero/issues/detail?id=2023
>
> It has a "Methodology-Fuzzing" label, so after CVE-2018-14645 and
> CVE-2018-20615 this is the 3rd CVE within H2 found using fuzzing that
> I'm aware of. It probably won't be the last. Can we please allocate some
> resources on making HAProxy more fuzzer friendly after 2.2 is out?
>
> I would also be interested in how Felix Wilhelm performed the fuzzing,
> do you happen to have details about that?
>

h2spec is very close to fuzzing. so, we just fire numerous requests and see
what's going on.

ok, couple of things missing - core dump catch and address sanitizing. not
hard to add.

the question is "how to generate h2 fuzzing workload"


>
> Best regards
> Tim Düsterhus
>
>


Re: [ANNOUNCE] haproxy-2.1.4

2020-04-21 Thread Илья Шипицин
вт, 21 апр. 2020 г. в 19:13, Tim Düsterhus :

> Ilya,
>
> Am 21.04.20 um 15:47 schrieb Илья Шипицин:
> >> The write-up is available now:
> >> https://bugs.chromium.org/p/project-zero/issues/detail?id=2023
> >>
> >> It has a "Methodology-Fuzzing" label, so after CVE-2018-14645 and
> >> CVE-2018-20615 this is the 3rd CVE within H2 found using fuzzing that
> >> I'm aware of. It probably won't be the last. Can we please allocate some
> >> resources on making HAProxy more fuzzer friendly after 2.2 is out?
> >>
> >> I would also be interested in how Felix Wilhelm performed the fuzzing,
> >> do you happen to have details about that?
> >>
> >
> > h2spec is very close to fuzzing. so, we just fire numerous requests and
> see
> > what's going on.
> >
> > ok, couple of things missing - core dump catch and address sanitizing.
> not
> > hard to add.
> >
> > the question is "how to generate h2 fuzzing workload"
> >
>
> The two CVEs I mentioned were bugs *I* found using afl-fuzz. The biggest
> hurdle back when I attempted fuzzing was not getting an appropriate
> workload (I've just created a few basic requests using nghttp), but
> instead getting the requests into HAProxy in a way so that afl is able
> to detect branches that change based on input changes. This branch
> detection is *the* main selling point of afl. Just sending random
> garbage is not going to turn up interesting stuff, if anything.
>


I really beleive that people who can perform fuzzing are smarter than me.
But I hope
to be able to run fuzzing some day :)

what are "branches" ? are them git branches ? do you have any setup
step-by-step
described how those CVE were detected ?


>
> For CVE-2018-14645 this worked well, because I could use the standalone
> hpack decoder. For CVE-2018-20615 I worked with preeny/desock and saw
> that issues with branches being non-deterministic (I assume slight
> timing issues or packets being cut differently or something like that).
>
> Best regards
> Tim Düsterhus
>


Re: [PATCH] CI: special purpose build, testing compatibility against "no-deprecated" openssl

2020-04-20 Thread Илья Шипицин
Lukas, Willy ?

чт, 16 апр. 2020 г. в 23:16, Илья Шипицин :

> Hello,
>
> I added weekly build for detection incompatibilities against
> "no-deprecated" openssl.
>
> (well, I first thought to add those option to travis, but it became
> over-engineered from my point of view)
>
> Lukas, if you have suggestions how to add to travis, I can try.
>
> Cheers,
> Ilya Shipitsin
>


Re: HAproxy Error

2020-04-16 Thread Илья Шипицин
чт, 16 апр. 2020 г. в 12:48, Lukas Tribus :

> Hello,
>
> On Thu, 16 Apr 2020 at 06:04,  wrote:
> >
> > Hi Team
> >
> > Let us know your availability to work on this.
>
> As Aleks already said:
>
> This haproxy executable has been build without OpenSSL support, which
> is required for your configuration.
>
> Provide the output of "which haproxy" and "haproxy -vv", I doubt you
>

also, it would be nice to see

rpm -qf `which haproxy`



> are actually running the Redhat package you indicated, more likely you
> built Haproxy manually from source on top of it.
>
>
> Lukas
>
>


Re: HAproxy Error

2020-04-16 Thread Илья Шипицин
hmm.

seems, we need some musl or picolibc in CI.
beeing glibc dependent is dangerous

чт, 16 апр. 2020 г. в 13:29, Willy Tarreau :

> Hi Lukas,
>
> On Thu, Apr 16, 2020 at 09:44:39AM +0200, Lukas Tribus wrote:
> > Provide the output of "which haproxy" and "haproxy -vv", I doubt you
> > are actually running the Redhat package you indicated, more likely you
> > built Haproxy manually from source on top of it.
>
> This just makes me think that on glibc-based systems we could use
> getauxval() to report the full path name to the executable in case
> of error. I'll see if I can come up with something usable.
>
> Willy
>
>


Re: HAProxy and musl (was: Re: HAproxy Error)

2020-04-16 Thread Илья Шипицин
yep, I thought about alpine as well.

I'm not sure how often official docker validation runs. If it runs often
enough, maybe we do not need CI.

чт, 16 апр. 2020 г. в 15:29, Tim Düsterhus :

> Willy,
>
> [removed Bindushree from Cc as we disgress from the main topic]
>
> Am 16.04.20 um 11:44 schrieb Willy Tarreau:
> >> seems, we need some musl or picolibc in CI.
> >> beeing glibc dependent is dangerous
> >
> > It's not really glibc-dependent in that it's properly enclosed in ifdefs.
> > But you'd be welcome to add musl if you find an easy way to do it. Please
> > just avoid building a toolchain and musl on the fly for this, as this
> would
> > eat a huge amount of resources on the CI infrastructure :-/
> >
> > FWIW musl seems to work OK here when building for linux-glibc-legacy.
>
> Yes. HAProxy linked against Musl is smoke tested as part of the Docker
> Official Images program, because the Alpine-based Docker images use Musl
> as their libc. In fact you can even use TARGET=linux-glibc +
> USE_BACKTRACE=.
>
> Make options are here:
>
> https://github.com/docker-library/haproxy/blob/3dd3917d3a70c230d8b192541ee08764e1da16af/2.2-rc/alpine/Dockerfile#L31-L45
>
> Basic smoke test (Reverse Proxy to example.com) is here:
>
> https://github.com/docker-library/official-images/tree/master/test/tests/haproxy-basics
>
> Best regards
> Tim Düsterhus
>


Re: [PATCH] CI: minor cleanup on SSL linking

2020-04-06 Thread Илья Шипицин
Hello,

that patch was not applied?

пт, 3 апр. 2020 г. в 17:18, Willy Tarreau :

> On Thu, Apr 02, 2020 at 11:46:58PM +0500,  ??? wrote:
> > Hello,
> >
> > this PR cleans up SSL linking.
> > it is very well aligned to "how to link to custom openssl" documentation.
>
> It's indeed cleaner, thanks!
> Willy
>


Re: s390x and HAProxy?

2020-03-13 Thread Илья Шипицин
пт, 13 мар. 2020 г. в 16:48, Aleksandar Lazic :

> Mar 13, 2020 12:11:16 PM Илья Шипицин :
>
> > initial motivation was that article
> >
> > https://docs.travis-ci.com/user/multi-cpu-architectures
>
> Thanks for the link.
>
> > travis allows to run builds on various archs, why not to test ?
>
> Full Agreeing. ;-)
>
> Would be interesting if anyone use it on Host or power.
>

there were issues related to haproxy on openwrt.
I am not surprised if somebody runs on s390x



>
> > пт, 13 мар. 2020 г. в 16:07, Aleksandar Lazic < al-hapr...@none.at >:
> >
> >
> > > Hi.
> > >
> > > I'm wondering that this target is tested.
> > >
> http://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=d726386421dcd184ca2518d17332f82e9cd79f2d
> > >
> > > Are there really user which runs HAProxy on Host? 8-O
> > > How perform HAProxy on that platform?
> > >
> > > Regards
> > > Aleks
>
>
>


Re: s390x and HAProxy?

2020-03-13 Thread Илья Шипицин
initial motivation was that article
https://docs.travis-ci.com/user/multi-cpu-architectures

travis allows to run builds on various archs, why not to test ?

пт, 13 мар. 2020 г. в 16:07, Aleksandar Lazic :

> Hi.
>
> I'm wondering that this target is tested.
>
> http://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=d726386421dcd184ca2518d17332f82e9cd79f2d
>
> Are there really user which runs HAProxy on Host? 8-O
> How perform HAProxy on that platform?
>
> Regards
> Aleks
>
>
>
>


Re: Tests timeout on my ARM64 test VM

2020-03-15 Thread Илья Шипицин
tests are easy to reproduce manually.

each test consiss of few parts

1) prepare config
2) start haproxy
3) perform queries against it


it might be something like iptables (just guess).
I would have a look inside some (failing) vtc and create config to start
haproxy manually

On Sun, Mar 15, 2020, 6:30 PM Martin Grigorov 
wrote:

> Hi Willy,
>
> On Fri, Mar 13, 2020 at 10:03 PM Willy Tarreau  wrote:
>
>> Hi Martin,
>>
>> On Fri, Mar 13, 2020 at 12:35:12PM +0200, Martin Grigorov wrote:
>> > Hi ,
>> >
>> > Suddenly today the build is again green here!
>> > I didn't make any changes to my testing setup.
>> > It must be something on the OS level but I wasn't able to figure out
>> what
>> > makes the HAProxy tests timeout in the last several days.
>>
>> We've had issues with the abns test on other platforms in the past,
>> namely s390x and ppc64le. It used to occasionally break on x86_64 as
>> well but far less frequently. It was affected by two bugs that were
>> solved yesterday after a few days of investigation and testing. We've
>> seen yet another failure again on ppc64 while it was expected not to
>> fail, so I'd be careful before claiming victory. However the abns
>> test is extremely time sensitive and uses short delays around 15ms to
>> try to trigger the issue, and in a VM it is possible to see this
>> happen from time to time due to noisy neighbors. That's why I'm
>> staying extremely prudent on the verdict. The PPC64 machine I tested
>> on is provided by Minicloud and is a VM running on a real CPU, so it's
>> much less affected by timing issues. I've run the test several hundreds
>> of times in a row and couldn't make it fail anymore.
>>
>> So don't worry too much if it appeared and disappeared. The change
>> that emphasized it was the increase in default maxconn (304e17eb8),
>> apparently just due to a slightly longer startup time! And the ones
>> expected to have fixed it are between bdb00c5d and 4b3f27b included.
>>
>> Note that I didn't manage to make it fail on arm64 (real machine,
>> SolidRun's Macchiatobin).
>>
>> Hoping this clarifies the situation.
>>
>
> Thank you for this explanation!
>
> The problem here was that the tests were failing even with older commits.
> I've tried to git bisect the problem but no matter how far back in Git
> history I went the problem was still there. But the logs from the tests
> runs from just few days back were OK, no errors & timeouts.
> Also the ARM64 tests on TravisCI were OK. And Travis's ARM64 nodes are
> less powerful than mine VM.
> Those are the reasons I believe that the problem was at my VM.
> I just needed help with reading HAProxy's tests error logs. I didn't know
> how to approach debugging them.
>
> Regards,
> Martin
>
>
>>
>> Regards,
>> Willy
>>
>


Re: [PATCH[ special purpose weekly CI (spellcheck)

2020-03-13 Thread Илья Шипицин
Willy, can we apply this?

I scheduled it on tuesday every week.

вт, 10 мар. 2020 г. в 12:12, Илья Шипицин :

> Hello,
>
> I implemented spell check CI.
> it is a bit noisy, however, I hope we will polish it soon.
>
> Cheers,
> Ilya Shipitcin
>


"warning" on reg-tests

2020-03-13 Thread Илья Шипицин
Hello,

there's always noise like that


***  h1   debug|[WARNING] 072/163523 (6090) : config : log format
ignored for proxy 'testme' since it has no log address.
***  h1   debug|[WARNING] 072/163523 (6090) : config : log format
ignored for frontend 'test_abns' since it has no log address.


any special meaning to have logs misconfigured for reg-tests ? or we
can reduce that noise ?

Cheers,

Ilya Shipitcin


Re: [PATCH] switch to clang-9 in Linux/travis-ci builds

2020-03-16 Thread Илья Шипицин
пн, 16 мар. 2020 г. в 11:35, Willy Tarreau :

> On Sun, Mar 15, 2020 at 10:54:56PM +0500,  ??? wrote:
> > ??, 14 ???. 2020 ?. ? 14:23, Willy Tarreau :
> >
> > > Hi Ilya,
> > >
> > > On Fri, Jan 24, 2020 at 11:46:45AM +0500,  ??? wrote:
> > > > Hello,
> > > >
> > > > let us use clang-9 instead of default clang-7 for linux builds.
> > >
> > > It seems I missed this one. Now applied carefully, we'll see. If it
> > > causes new failures, we'll adjust accordingly.
> > >
> >
> > BoringSSL is not happy
> > https://travis-ci.com/github/haproxy/haproxy/jobs/298267505
> >
> > I'll have a look
>
> It's complaining about this:
>
>   error: unknown warning option '-Wno-free-nonheap-object'; did you mean
> '-Wno-sequence-point'? [-Werror,-Wunknown-warning-option]
>

https://bugs.chromium.org/p/boringssl/issues/detail?id=323


>
> Thus it's pretty clear that boringssl uses hard-coded gcc options and
> is not even meant to be built with clang. We should probably roll back
> to gcc for this one. I can do it if you want.
>

we use clang because of its address sanitizer. I found gcc asan more noisy
and less usable.


anyway, we can switch back to clang-7 or gcc, until boringssl will fix that.


>
> Willy
>


Re: [PATCH] switch to clang-9 in Linux/travis-ci builds

2020-03-16 Thread Илья Шипицин
пн, 16 мар. 2020 г. в 12:09, Willy Tarreau :

> On Mon, Mar 16, 2020 at 11:52:27AM +0500,  ??? wrote:
> > ??, 16 ???. 2020 ?. ? 11:35, Willy Tarreau :
> >
> > > On Sun, Mar 15, 2020 at 10:54:56PM +0500,  ??? wrote:
> > > > ??, 14 ???. 2020 ?. ? 14:23, Willy Tarreau :
> > > >
> > > > > Hi Ilya,
> > > > >
> > > > > On Fri, Jan 24, 2020 at 11:46:45AM +0500,  ??? wrote:
> > > > > > Hello,
> > > > > >
> > > > > > let us use clang-9 instead of default clang-7 for linux builds.
> > > > >
> > > > > It seems I missed this one. Now applied carefully, we'll see. If it
> > > > > causes new failures, we'll adjust accordingly.
> > > > >
> > > >
> > > > BoringSSL is not happy
> > > > https://travis-ci.com/github/haproxy/haproxy/jobs/298267505
> > > >
> > > > I'll have a look
> > >
> > > It's complaining about this:
> > >
> > >   error: unknown warning option '-Wno-free-nonheap-object'; did you
> mean
> > > '-Wno-sequence-point'? [-Werror,-Wunknown-warning-option]
> > >
> >
> > https://bugs.chromium.org/p/boringssl/issues/detail?id=323
>
> Great, thank you.
>
> > > Thus it's pretty clear that boringssl uses hard-coded gcc options and
> > > is not even meant to be built with clang. We should probably roll back
> > > to gcc for this one. I can do it if you want.
> > >
> >
> > we use clang because of its address sanitizer. I found gcc asan more
> noisy
> > and less usable.
> >
> >
> > anyway, we can switch back to clang-7 or gcc, until boringssl will fix
> that.
>
> OK, I'll check which entry it is and revert the relevant part.
>
> Willy
>
From 5d5891166389dc03ab4fb63ca9edaa35feca8fcc Mon Sep 17 00:00:00 2001
From: Ilya Shipitsin 
Date: Mon, 16 Mar 2020 12:49:34 +0500
Subject: [PATCH] CI: switch BoringSSL back to clang-7

it turned out that BoringSSL is not prepared for clang-9
https://bugs.chromium.org/p/boringssl/issues/detail?id=323

until that issue is resolved, let us roll back to clang-7
---
 .travis.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.travis.yml b/.travis.yml
index 96f16ed42..05f4a3f1e 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -81,7 +81,7 @@ matrix:
   - os: linux
 if: type == cron
 compiler: clang
-env: TARGET=linux-glibc BORINGSSL=yes CC=clang-9
+env: TARGET=linux-glibc BORINGSSL=yes CC=clang
   - os: linux
 if: type != cron
 compiler: clang
-- 
2.24.1



<    1   2   3   4   5   6   7   8   9   10   >