[PATCH] BUILD: common: Add __ha_cas_dw fallback for single threaded builds
__ha_cas_dw() is used in fd_rm_from_fd_list() and when built without
USE_THREADS=1 the linker fails to find __ha_cas_dw(). Add a definition
of __ha_cas_dw() for the #ifndef USE_THREADS case.
Signed-off-by: Chris Packham
---
include/common/hathreads.h | 5 +
1 file changed, 5 insertions(+)
diff --git a/include/common/hathreads.h b/include/common/hathreads.h
index cae6eabe..516a66d0 100644
--- a/include/common/hathreads.h
+++ b/include/common/hathreads.h
@@ -140,6 +140,11 @@ static inline void __ha_barrier_full(void)
{
}
+static inline int __ha_cas_dw(void *target, void *compare, void *set)
+{
+ return HA_ATOMIC_CAS(target, compare, set);
+}
+
static inline void thread_harmless_now()
{
}
--
2.21.0
Link error building haproxy-1.9.7
Hi, I'm encountering the following linker error when building haproxy-1.9.7 make CC=arm-softfloat-linux-gnueabi USE_OPENSSL=1 ... LD haproxy /usr/bin/../lib/gcc/arm-softfloat-linux-gnueabi/8.3.0/../../../../arm-softfloat-linux-gnueabi/bin/ld: src/fd.o: in function `fd_rm_from_fd_list': haproxy-1.9.7/src/fd.c:267: undefined reference to `__ha_cas_dw' collect2: error: ld returned 1 exit status Makefile:994: recipe for target 'haproxy' failed make: *** [haproxy] Error 1 Eyeballing the code I think it's because USE_THREAD is not defined and __ha_cas_dw is only defined when USE_THREAD is defined
Re: [PATCH v2 1/2] MINOR: systemd: Use the variables from /etc/default/haproxy
❦ 8 mai 2019 16:23 +02, Tim Düsterhus : >> I think, instead of After=rsyslog.service, it should be >> After=syslog.service, then any logger daemon could be used that has >> Alias=syslog.service. >> >> https://www.freedesktop.org/wiki/Software/systemd/syslog/ >> > > The HAProxy provided unit file does not include a dependency at all. > That's a Debian specific patch to include the dependency. I think we didn't know about the alias stuff. We'll fix that on our side on next upload. -- Make it right before you make it faster. - The Elements of Programming Style (Kernighan & Plauger)
unsubscribe
RE: cygwin compilation error
Ilya Shipitsin wrote on Wed, 08 May 2019 03:22:20 -0700: >I guess the same would be with BoringSSL. It exposes OPENSSL_VERSION_NUMBER >and it is not openssl itself. > >yes, we can add warning. I wouldn't bother even trying to add support for BoringSSL - they themselves discourage people from doing so in their mission statement: "Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability. Programs ship their own copies of BoringSSL when they use it and we update everything as needed when deciding to make API changes. This allows us to mostly avoid compromises in the name of compatibility. It works for us, but it may not work for you." Regards, Bob
[PATCH] BUILD: add BoringSSL to travis-ci build matrix
From: Ilya Shipitsin
---
.travis.yml | 3 +++
scripts/build-ssl.sh | 19 +++
2 files changed, 22 insertions(+)
diff --git a/.travis.yml b/.travis.yml
index f9a13586..c00725d8 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -44,6 +44,9 @@ matrix:
- os: linux
compiler: gcc
env: TARGET=linux2628 LIBRESSL_VERSION=2.7.5
+ - os: linux
+compiler: gcc
+env: TARGET=linux2628 BORINGSSL=yes
- os: linux
compiler: clang
env: TARGET=linux2628 FLAGS=
diff --git a/scripts/build-ssl.sh b/scripts/build-ssl.sh
index a8863ed3..68312c63 100755
--- a/scripts/build-ssl.sh
+++ b/scripts/build-ssl.sh
@@ -66,4 +66,23 @@ if [ ! -z ${OPENSSL_VERSION+x} ]; then
build_openssl
fi
+if [ ! -z ${BORINGSSL+x} ]; then
+ (
+ git clone --depth=1 https://boringssl.googlesource.com/boringssl
+ cd boringssl
+ mkdir build
+ cd build
+ cmake -DCMAKE_BUILD_TYPE=release -DBUILD_SHARED_LIBS=1 ..
+ make
+
+ rm -rf ${SSL_LIB} || exit 0
+ rm -rf ${SSL_INC} || exit 0
+
+ mkdir -p ${SSL_LIB}
+ cp crypto/libcrypto.so ssl/libssl.so ${SSL_LIB}
+
+ mkdir -p ${SSL_INC}
+ mv ../include/* ${SSL_INC}
+ )
+fi
--
2.20.1
CI question related to openssl matrix
Hello, does haproxy have some issues when it is built using openssl-1.1.0 and running with openssl-1.1.1, for example ? should we consider such situations in travis-ci openssl matrix ? thanks! Ilya Shipitsin
Re: [1.9 HEAD] HAProxy using 100% CPU
Ok, thanks! śr., 8.05.2019, 16:50 użytkownik Willy Tarreau napisał: > On Wed, May 08, 2019 at 04:48:55PM +0200, Maciej Zdeb wrote: > > I'll gladly test Olivier patch after backporting. :) > > Thank you Maciej. I still have other stuff to deal with before going back > to the pending 1.9 backports, and I'll merge it. > > Willy >
Re: [1.9 HEAD] HAProxy using 100% CPU
I'll gladly test Olivier patch after backporting. :) śr., 8.05.2019, 15:29 użytkownik Willy Tarreau napisał: > On Wed, May 08, 2019 at 03:03:23PM +0200, Olivier Houchard wrote: > > > > I can't seem to remember :) > > > > > > Given the number of bugs we've dealt with in the last few weeks, you're > > > forgiven :-) > > > > > > > I'm afraid I'm getting old :/ > > Ah! you see how it feels! > > > > > I think that patch is safe to backport anyway, but I wouldn't swear > it solves > > > > Maciej's issue. > > > > > > OK. I'll take it then so that we can study the situation under better > > > conditions if the problem happens again. It may be possible that we've > > > also overlooked other situations where this infinite loop could happen > > > with SUB_CALL_UNSUBSCRIBE and that were addressed by your patch. > > > > > > > Yes it is certainly possible it helps, I just can affirm it with > confidence. > > This is why I prefer to take it. The amount of changes that went into 1.9 > was significant enough to get such hard-to-debug issues. As long as fixes > don't harm, I prefer to take them than having to fight one full week with > a bug just to discover that it was already fixed upstream. > > Thanks, > Willy >
Re: [PATCH v2 1/2] MINOR: systemd: Use the variables from /etc/default/haproxy
Veiko, Am 08.05.19 um 15:44 schrieb Veiko Kukk: > I think, instead of After=rsyslog.service, it should be > After=syslog.service, then any logger daemon could be used that has > Alias=syslog.service. > > https://www.freedesktop.org/wiki/Software/systemd/syslog/ > The HAProxy provided unit file does not include a dependency at all. That's a Debian specific patch to include the dependency. Best regards Tim Düsterhus
Re: [PATCH v2 1/2] MINOR: systemd: Use the variables from /etc/default/haproxy
On 2019-05-06 11:00, Tim Duesterhus wrote: From: Apollon Oikonomopoulos This will allow seamless upgrades from the sysvinit system while respecting any changes the users may have made. It will also make local configuration easier than overriding the systemd unit file. Note by Tim: This GPL-2 licensed patch was taken from the Debian project at [1]. It was slightly modified to cleanly apply, because HAProxy's default unit file does not include rsyslog.service as an 'After' dependency. Also the subject line was modified to include the proper subsystem and severity. I think, instead of After=rsyslog.service, it should be After=syslog.service, then any logger daemon could be used that has Alias=syslog.service. https://www.freedesktop.org/wiki/Software/systemd/syslog/ Regards, Veiko
Re: [1.9 HEAD] HAProxy using 100% CPU
On Wed, May 08, 2019 at 03:03:23PM +0200, Olivier Houchard wrote: > > > I can't seem to remember :) > > > > Given the number of bugs we've dealt with in the last few weeks, you're > > forgiven :-) > > > > I'm afraid I'm getting old :/ Ah! you see how it feels! > > > I think that patch is safe to backport anyway, but I wouldn't swear it > > > solves > > > Maciej's issue. > > > > OK. I'll take it then so that we can study the situation under better > > conditions if the problem happens again. It may be possible that we've > > also overlooked other situations where this infinite loop could happen > > with SUB_CALL_UNSUBSCRIBE and that were addressed by your patch. > > > > Yes it is certainly possible it helps, I just can affirm it with confidence. This is why I prefer to take it. The amount of changes that went into 1.9 was significant enough to get such hard-to-debug issues. As long as fixes don't harm, I prefer to take them than having to fight one full week with a bug just to discover that it was already fixed upstream. Thanks, Willy
Re: [1.9 HEAD] HAProxy using 100% CPU
On Wed, May 08, 2019 at 02:30:07PM +0200, Willy Tarreau wrote:
> On Wed, May 08, 2019 at 01:56:05PM +0200, Olivier Houchard wrote:
> > > > One of processes stuck in infinite loop, admin socket is not responsive
> > > > so
> > > > I've got information only from gdb:
> > > >
> > > > 0x00484ab8 in h2_process_mux (h2c=0x2e8ff30) at
> > > > src/mux_h2.c:2589
> > > > 2589if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
> > > > (gdb) n
> > > (...)
> > >
> > > CCing Olivier. Olivier, I'm wondering if this is not directly related to
> > > what you addressed with this fix merged in 2.0 but not backported :
> > >
> > > 998410a ("BUG/MEDIUM: h2: Revamp the way send subscriptions works.")
> > >
> > > From what I'm seeing there's no error, the stream is in the sending list,
> > > there's no blocking flag, well, everything looks OK, but we're looping
> > > on SUB_CALL_UNSUBSCRIBE which apprently should not if I understand it
> > > right. Do you think we should backport this patch ?
> > >
> > > Remaining of the trace below for reference.
> > >
> > > THanks,
> > > Willy
> > >
> >
> > I can't seem to remember :)
>
> Given the number of bugs we've dealt with in the last few weeks, you're
> forgiven :-)
>
I'm afraid I'm getting old :/
> > I remember a bug involving the h2s being added to the send_list twice, and
> > thus leading to an infinite loop when parsing the send_list, but it was
> > fixed by 9a0f559676c4d309edbe42ba33197e7dd8935f1c, and backported.
> > For me, this one was mostly to make me happier about the code, and stop
> > abusing SUB_CALL_UNSUBSCRIBE, and if I didn't mention it should be
> > backported
> > it's probably I considered the code in 1.9 was OK (or I just forgot), but
> > it's still flagged as "BUG" too (but that could be me having "BUG/MEDIUM"
> > hardcoded in my finger).
> > I think that patch is safe to backport anyway, but I wouldn't swear it
> > solves
> > Maciej's issue.
>
> OK. I'll take it then so that we can study the situation under better
> conditions if the problem happens again. It may be possible that we've
> also overlooked other situations where this infinite loop could happen
> with SUB_CALL_UNSUBSCRIBE and that were addressed by your patch.
>
Yes it is certainly possible it helps, I just can affirm it with confidence.
Regards,
Olivier
Re: [1.9 HEAD] HAProxy using 100% CPU
On Wed, May 08, 2019 at 01:56:05PM +0200, Olivier Houchard wrote:
> > > One of processes stuck in infinite loop, admin socket is not responsive so
> > > I've got information only from gdb:
> > >
> > > 0x00484ab8 in h2_process_mux (h2c=0x2e8ff30) at src/mux_h2.c:2589
> > > 2589if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
> > > (gdb) n
> > (...)
> >
> > CCing Olivier. Olivier, I'm wondering if this is not directly related to
> > what you addressed with this fix merged in 2.0 but not backported :
> >
> > 998410a ("BUG/MEDIUM: h2: Revamp the way send subscriptions works.")
> >
> > From what I'm seeing there's no error, the stream is in the sending list,
> > there's no blocking flag, well, everything looks OK, but we're looping
> > on SUB_CALL_UNSUBSCRIBE which apprently should not if I understand it
> > right. Do you think we should backport this patch ?
> >
> > Remaining of the trace below for reference.
> >
> > THanks,
> > Willy
> >
>
> I can't seem to remember :)
Given the number of bugs we've dealt with in the last few weeks, you're
forgiven :-)
> I remember a bug involving the h2s being added to the send_list twice, and
> thus leading to an infinite loop when parsing the send_list, but it was
> fixed by 9a0f559676c4d309edbe42ba33197e7dd8935f1c, and backported.
> For me, this one was mostly to make me happier about the code, and stop
> abusing SUB_CALL_UNSUBSCRIBE, and if I didn't mention it should be backported
> it's probably I considered the code in 1.9 was OK (or I just forgot), but
> it's still flagged as "BUG" too (but that could be me having "BUG/MEDIUM"
> hardcoded in my finger).
> I think that patch is safe to backport anyway, but I wouldn't swear it solves
> Maciej's issue.
OK. I'll take it then so that we can study the situation under better
conditions if the problem happens again. It may be possible that we've
also overlooked other situations where this infinite loop could happen
with SUB_CALL_UNSUBSCRIBE and that were addressed by your patch.
Thanks,
Willy
Re: [1.9 HEAD] HAProxy using 100% CPU
Hi,
On Wed, May 08, 2019 at 07:11:27AM +0200, Willy Tarreau wrote:
> Hi Maciej,
>
> On Tue, May 07, 2019 at 07:08:47PM +0200, Maciej Zdeb wrote:
> > Hi,
> >
> > I've got another bug with 100% CPU on HAProxy process, it is built from
> > HEAD of 1.9 branch.
> >
> > One of processes stuck in infinite loop, admin socket is not responsive so
> > I've got information only from gdb:
> >
> > 0x00484ab8 in h2_process_mux (h2c=0x2e8ff30) at src/mux_h2.c:2589
> > 2589if (h2s->send_wait->events & SUB_CALL_UNSUBSCRIBE)
> > (gdb) n
> (...)
>
> CCing Olivier. Olivier, I'm wondering if this is not directly related to
> what you addressed with this fix merged in 2.0 but not backported :
>
> 998410a ("BUG/MEDIUM: h2: Revamp the way send subscriptions works.")
>
> From what I'm seeing there's no error, the stream is in the sending list,
> there's no blocking flag, well, everything looks OK, but we're looping
> on SUB_CALL_UNSUBSCRIBE which apprently should not if I understand it
> right. Do you think we should backport this patch ?
>
> Remaining of the trace below for reference.
>
> THanks,
> Willy
>
I can't seem to remember :)
I remember a bug involving the h2s being added to the send_list twice, and
thus leading to an infinite loop when parsing the send_list, but it was
fixed by 9a0f559676c4d309edbe42ba33197e7dd8935f1c, and backported.
For me, this one was mostly to make me happier about the code, and stop
abusing SUB_CALL_UNSUBSCRIBE, and if I didn't mention it should be backported
it's probably I considered the code in 1.9 was OK (or I just forgot), but
it's still flagged as "BUG" too (but that could be me having "BUG/MEDIUM"
hardcoded in my finger).
I think that patch is safe to backport anyway, but I wouldn't swear it solves
Maciej's issue.
Regards,
Olivier
Re: cygwin compilation error
ср, 8 мая 2019 г. в 14:50, Willy Tarreau : > On Wed, May 08, 2019 at 02:06:31PM +0500, ??? wrote: > > > Ilya, could you please instead change the test like this and test > again : > > > > > > -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) > > > +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x1010100fL) > > > > > > > LibreSSL defines is > > > > #define OPENSSL_VERSION_NUMBER0x2000L > > > > it is bigger then any released OpenSSL (yet, for openssl master it is > 3.0.0) > > So this behaviour from them make them complete bastards and will > constantly break each and every program trying to build with it :-( > > This stupidity really makes me want to completely remove support for > libressl. > I guess the same would be with BoringSSL. It exposes OPENSSL_VERSION_NUMBER and it is not openssl itself. yes, we can add warning. > > I don't know when they forked nor what is the latest version they are > *really* compatible with, but what we should probably do is to change > their marketing version to a real version in the compat.h file to do > something like this : > > #if defined(LIBRESSL_VERSION_NUMBER) > #undef OPENSSL_VERSION_NUMBER > #define OPENSSL_VERSION_NUMBER 0x10something > #endif > > This way we won't have to guard ourselves against these lies each and > every time we add something to deal with some openssl-specific issues > or features. Ideas welcome, of course. If for any reason we can't do > something like the above, we should at least add a big fat warning when > building with it to explicitly mention that it uses fake version numbers > overlapping with openssl versions and may trigger API incompatibility > issues that might result in runtime problems, so that users are warned. > > Willy >
Re: cygwin compilation error
On Wed, May 08, 2019 at 02:06:31PM +0500, ??? wrote: > > Ilya, could you please instead change the test like this and test again : > > > > -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) > > +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x1010100fL) > > > > LibreSSL defines is > > #define OPENSSL_VERSION_NUMBER0x2000L > > it is bigger then any released OpenSSL (yet, for openssl master it is 3.0.0) So this behaviour from them make them complete bastards and will constantly break each and every program trying to build with it :-( This stupidity really makes me want to completely remove support for libressl. I don't know when they forked nor what is the latest version they are *really* compatible with, but what we should probably do is to change their marketing version to a real version in the compat.h file to do something like this : #if defined(LIBRESSL_VERSION_NUMBER) #undef OPENSSL_VERSION_NUMBER #define OPENSSL_VERSION_NUMBER 0x10something #endif This way we won't have to guard ourselves against these lies each and every time we add something to deal with some openssl-specific issues or features. Ideas welcome, of course. If for any reason we can't do something like the above, we should at least add a big fat warning when building with it to explicitly mention that it uses fake version numbers overlapping with openssl versions and may trigger API incompatibility issues that might result in runtime problems, so that users are warned. Willy
Re: cygwin compilation error
ср, 8 мая 2019 г. в 13:55, Willy Tarreau : > On Wed, May 08, 2019 at 01:13:56PM +0500, ??? wrote: > > > libressl ? My understanding of this thing is that this problem is not > > > easy to detect by accident and causes a mess for people who reload > often. > > > If libressl is affected by this we probably need to find a different > > > fix. And if it's not affected, at least the tested version(s) must be > > > mentioned in the commit message so that we can reconsider or refine > this > > > choice later if/when the problem appears with a subsequent version. > > > CCing William and Emeric who worked on addressing this issue for > OpenSSL. > > > > > > > I planned to have a look at it actually. The idea is to write some reg > test > > which will reload and watch for open FDs. > > not sure whether it is easy or not > > But before writing reg tests, it's important not to revert part of a patch > without knowing if it brings the issue back. Otherwise you end up with a > patch merged into a branch, making users believe their bug is fixed since > the patch is there, while in fact it was later silently reverted as a > "build fix". > > > the idea behind quick patch is "if you use LibreSSL you are on your own > and > > you have been warned" > > (yes, we did our best to make it work with LibreSSL, but it is still a > > niche solution, not very well tested) > > Some of the users here do rely on it. However, seeing that you had to > turn off this test makes me think that LibreSSL pretends to be openssl > 1.1.1 but is not compatible with it. I suspect that instead the OpenSSL > test version is wrong in the original patch. It seems to be testing for > 1.1.1-dev instead of testing for 1.1.1-release. So probably that this > RAND_* function appears late in the development process and that libressl > only complies with an early 1.1.1-dev version. > > Surprisingly I'm seeing that *all* of our tests for 1.1.1 are wrong. I > suspect that one was either wrong or deliberate initially and that it > got copy-pasted everywhere :-( > > Ilya, could you please instead change the test like this and test again : > > -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) > +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x1010100fL) > LibreSSL defines is #define OPENSSL_VERSION_NUMBER0x2000L it is bigger then any released OpenSSL (yet, for openssl master it is 3.0.0) > > Thanks, > Willy >
Re: cygwin compilation error
On Wed, May 08, 2019 at 01:13:56PM +0500, ??? wrote: > > libressl ? My understanding of this thing is that this problem is not > > easy to detect by accident and causes a mess for people who reload often. > > If libressl is affected by this we probably need to find a different > > fix. And if it's not affected, at least the tested version(s) must be > > mentioned in the commit message so that we can reconsider or refine this > > choice later if/when the problem appears with a subsequent version. > > CCing William and Emeric who worked on addressing this issue for OpenSSL. > > > > I planned to have a look at it actually. The idea is to write some reg test > which will reload and watch for open FDs. > not sure whether it is easy or not But before writing reg tests, it's important not to revert part of a patch without knowing if it brings the issue back. Otherwise you end up with a patch merged into a branch, making users believe their bug is fixed since the patch is there, while in fact it was later silently reverted as a "build fix". > the idea behind quick patch is "if you use LibreSSL you are on your own and > you have been warned" > (yes, we did our best to make it work with LibreSSL, but it is still a > niche solution, not very well tested) Some of the users here do rely on it. However, seeing that you had to turn off this test makes me think that LibreSSL pretends to be openssl 1.1.1 but is not compatible with it. I suspect that instead the OpenSSL test version is wrong in the original patch. It seems to be testing for 1.1.1-dev instead of testing for 1.1.1-release. So probably that this RAND_* function appears late in the development process and that libressl only complies with an early 1.1.1-dev version. Surprisingly I'm seeing that *all* of our tests for 1.1.1 are wrong. I suspect that one was either wrong or deliberate initially and that it got copy-pasted everywhere :-( Ilya, could you please instead change the test like this and test again : -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x1010100fL) Thanks, Willy
Re: cygwin compilation error
Frederic, can you help with some "haproxy reg test getting started" doc ? ср, 8 мая 2019 г. в 13:13, Илья Шипицин : > > > ср, 8 мая 2019 г. в 13:03, Willy Tarreau : > >> Hi Ilya, >> >> On Wed, May 08, 2019 at 11:34:57AM +0500, ??? wrote: >> > From ad9961e92c692430272c9088a49759c889dac6f1 Mon Sep 17 00:00:00 2001 >> > From: Ilya Shipitsin >> > Date: Wed, 8 May 2019 11:32:02 +0500 >> > Subject: [PATCH] BUILD: do not use "RAND_keep_random_devices_open" when >> > building against LibreSSL >> > >> > --- >> > src/haproxy.c | 2 +- >> > 1 file changed, 1 insertion(+), 1 deletion(-) >> > >> > diff --git a/src/haproxy.c b/src/haproxy.c >> > index 4c371254..c8a8aaf0 100644 >> > --- a/src/haproxy.c >> > +++ b/src/haproxy.c >> > @@ -590,7 +590,7 @@ void mworker_reload() >> > ptdf->fct(); >> > if (fdtab) >> > deinit_pollers(); >> > -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) >> > +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) && >> !defined LIBRESSL_VERSION_NUMBER) >> >> A parenthesis is missing here, please be careful to always try to build >> the code with submitted patches. >> >> > if (global.ssl_used_frontend || global.ssl_used_backend) >> > /* close random device FDs */ >> > RAND_keep_random_devices_open(0); >> >> Did you verify if this has an impact on FD leaks upon reloads when using >> > > I did a mess two times :) > > > >> libressl ? My understanding of this thing is that this problem is not >> easy to detect by accident and causes a mess for people who reload often. >> If libressl is affected by this we probably need to find a different >> fix. And if it's not affected, at least the tested version(s) must be >> mentioned in the commit message so that we can reconsider or refine this >> choice later if/when the problem appears with a subsequent version. >> CCing William and Emeric who worked on addressing this issue for OpenSSL. >> > > I planned to have a look at it actually. The idea is to write some reg > test which will reload and watch for open FDs. > not sure whether it is easy or not > > > the idea behind quick patch is "if you use LibreSSL you are on your own > and you have been warned" > (yes, we did our best to make it work with LibreSSL, but it is still a > niche solution, not very well tested) > > >> Thanks, >> Willy >> >
Re: cygwin compilation error
ср, 8 мая 2019 г. в 13:03, Willy Tarreau : > Hi Ilya, > > On Wed, May 08, 2019 at 11:34:57AM +0500, ??? wrote: > > From ad9961e92c692430272c9088a49759c889dac6f1 Mon Sep 17 00:00:00 2001 > > From: Ilya Shipitsin > > Date: Wed, 8 May 2019 11:32:02 +0500 > > Subject: [PATCH] BUILD: do not use "RAND_keep_random_devices_open" when > > building against LibreSSL > > > > --- > > src/haproxy.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/src/haproxy.c b/src/haproxy.c > > index 4c371254..c8a8aaf0 100644 > > --- a/src/haproxy.c > > +++ b/src/haproxy.c > > @@ -590,7 +590,7 @@ void mworker_reload() > > ptdf->fct(); > > if (fdtab) > > deinit_pollers(); > > -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) > > +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) && > !defined LIBRESSL_VERSION_NUMBER) > > A parenthesis is missing here, please be careful to always try to build > the code with submitted patches. > > > if (global.ssl_used_frontend || global.ssl_used_backend) > > /* close random device FDs */ > > RAND_keep_random_devices_open(0); > > Did you verify if this has an impact on FD leaks upon reloads when using > I did a mess two times :) > libressl ? My understanding of this thing is that this problem is not > easy to detect by accident and causes a mess for people who reload often. > If libressl is affected by this we probably need to find a different > fix. And if it's not affected, at least the tested version(s) must be > mentioned in the commit message so that we can reconsider or refine this > choice later if/when the problem appears with a subsequent version. > CCing William and Emeric who worked on addressing this issue for OpenSSL. > I planned to have a look at it actually. The idea is to write some reg test which will reload and watch for open FDs. not sure whether it is easy or not the idea behind quick patch is "if you use LibreSSL you are on your own and you have been warned" (yes, we did our best to make it work with LibreSSL, but it is still a niche solution, not very well tested) > Thanks, > Willy >
Re: haproxy-1.9 sanitizers finding
I would like to run sanitizers before new 1.9 release is out On Wed, May 8, 2019, 10:54 AM Willy Tarreau wrote: > Hi Ilya, > > On Tue, May 07, 2019 at 11:47:54AM +0500, ??? wrote: > > Hello, > > > > when running regtests against 1.9 branch there are findings (not seen in > > master branch) > > > > *** h10.0 > > debug|= > > *** h10.0 debug|==16493==ERROR: AddressSanitizer: > heap-use-after-free > > on address 0x61903c95 at pc 0x006ca207 bp 0x7ffd92124b60 sp > > 0x7ffd92124b50 > > *** h10.0 debug|WRITE of size 1 at 0x61903c95 thread T0 > > *** h10.0 debug|#0 0x6ca206 in update_log_hdr src/log.c:1260 > > *** h10.0 debug|#1 0x6ca206 in __send_log src/log.c:1445 > > *** h10.0 debug|#2 0x6ca48a in send_log src/log.c:1323 > (...) > > OK these are the same that you reported on master which is fixed there > and not backported yet. It should eventually get backported ;-) > > Thanks, > Willy >
Re: cygwin compilation error
Hi Ilya, On Wed, May 08, 2019 at 11:34:57AM +0500, ??? wrote: > From ad9961e92c692430272c9088a49759c889dac6f1 Mon Sep 17 00:00:00 2001 > From: Ilya Shipitsin > Date: Wed, 8 May 2019 11:32:02 +0500 > Subject: [PATCH] BUILD: do not use "RAND_keep_random_devices_open" when > building against LibreSSL > > --- > src/haproxy.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/haproxy.c b/src/haproxy.c > index 4c371254..c8a8aaf0 100644 > --- a/src/haproxy.c > +++ b/src/haproxy.c > @@ -590,7 +590,7 @@ void mworker_reload() > ptdf->fct(); > if (fdtab) > deinit_pollers(); > -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) > +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) && > !defined LIBRESSL_VERSION_NUMBER) A parenthesis is missing here, please be careful to always try to build the code with submitted patches. > if (global.ssl_used_frontend || global.ssl_used_backend) > /* close random device FDs */ > RAND_keep_random_devices_open(0); Did you verify if this has an impact on FD leaks upon reloads when using libressl ? My understanding of this thing is that this problem is not easy to detect by accident and causes a mess for people who reload often. If libressl is affected by this we probably need to find a different fix. And if it's not affected, at least the tested version(s) must be mentioned in the commit message so that we can reconsider or refine this choice later if/when the problem appears with a subsequent version. CCing William and Emeric who worked on addressing this issue for OpenSSL. Thanks, Willy
Re: cygwin compilation error
I messed up with commit message. One more try ср, 8 мая 2019 г. в 11:33, Илья Шипицин : > small fix > > ср, 8 мая 2019 г. в 11:12, Willy Tarreau : > >> On Wed, May 08, 2019 at 11:09:04AM +0500, ??? wrote: >> > ??, 8 ??? 2019 ?. ? 11:06, Willy Tarreau : >> > >> > > On Wed, May 08, 2019 at 10:59:20AM +0500, ??? wrote: >> > > > travis-ci supports windows builds. >> > > >> > > cool! >> > > >> > >> > my current roadmap is >> > >> > 1) patch fixes SSL variants (already sent to list). without it we are >> NOT >> > building LibreSSL at all (i.e. we use default openssl-1.0.2 for all >> builds) >> >> Pushed just now. >> >> > 2) BoringSSL >> > >> > 3) update gcc, clang, enable sanitizers >> > >> > 4) cygwin >> >> OK, sounds good. >> >> Thanks, >> Willy >> > From ad9961e92c692430272c9088a49759c889dac6f1 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Wed, 8 May 2019 11:32:02 +0500 Subject: [PATCH] BUILD: do not use "RAND_keep_random_devices_open" when building against LibreSSL --- src/haproxy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/haproxy.c b/src/haproxy.c index 4c371254..c8a8aaf0 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -590,7 +590,7 @@ void mworker_reload() ptdf->fct(); if (fdtab) deinit_pollers(); -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) && !defined LIBRESSL_VERSION_NUMBER) if (global.ssl_used_frontend || global.ssl_used_backend) /* close random device FDs */ RAND_keep_random_devices_open(0); -- 2.20.1
Re: cygwin compilation error
small fix ср, 8 мая 2019 г. в 11:12, Willy Tarreau : > On Wed, May 08, 2019 at 11:09:04AM +0500, ??? wrote: > > ??, 8 ??? 2019 ?. ? 11:06, Willy Tarreau : > > > > > On Wed, May 08, 2019 at 10:59:20AM +0500, ??? wrote: > > > > travis-ci supports windows builds. > > > > > > cool! > > > > > > > my current roadmap is > > > > 1) patch fixes SSL variants (already sent to list). without it we are NOT > > building LibreSSL at all (i.e. we use default openssl-1.0.2 for all > builds) > > Pushed just now. > > > 2) BoringSSL > > > > 3) update gcc, clang, enable sanitizers > > > > 4) cygwin > > OK, sounds good. > > Thanks, > Willy > From ad9961e92c692430272c9088a49759c889dac6f1 Mon Sep 17 00:00:00 2001 From: Ilya Shipitsin Date: Wed, 8 May 2019 11:32:02 +0500 Subject: [PATCH] BUILD: do not use && !defined LIBRESSL_VERSION_NUMBER) when building against LibreSSL --- src/haproxy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/haproxy.c b/src/haproxy.c index 4c371254..c8a8aaf0 100644 --- a/src/haproxy.c +++ b/src/haproxy.c @@ -590,7 +590,7 @@ void mworker_reload() ptdf->fct(); if (fdtab) deinit_pollers(); -#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) +#if defined(USE_OPENSSL) && (OPENSSL_VERSION_NUMBER >= 0x10101000L) && !defined LIBRESSL_VERSION_NUMBER) if (global.ssl_used_frontend || global.ssl_used_backend) /* close random device FDs */ RAND_keep_random_devices_open(0); -- 2.20.1
Re: [PATCH 1/1] BUILD: travis-ci bugfixes and improvements
On Tue, May 07, 2019 at 01:42:43AM +0500, chipits...@gmail.com wrote: > From: Ilya Shipitsin > > Call missing scripts/build-ssl.sh (which actually builds SSL variants) > Enable OpenSSL, LibreSSL builds caching, it saves a bunch of time > LibreSSL builds are not allowed to fail anymore > Add openssl to osx builds Merged, thanks! Willy
Re: cygwin compilation error
On Wed, May 08, 2019 at 11:09:04AM +0500, ??? wrote: > ??, 8 ??? 2019 ?. ? 11:06, Willy Tarreau : > > > On Wed, May 08, 2019 at 10:59:20AM +0500, ??? wrote: > > > travis-ci supports windows builds. > > > > cool! > > > > my current roadmap is > > 1) patch fixes SSL variants (already sent to list). without it we are NOT > building LibreSSL at all (i.e. we use default openssl-1.0.2 for all builds) Pushed just now. > 2) BoringSSL > > 3) update gcc, clang, enable sanitizers > > 4) cygwin OK, sounds good. Thanks, Willy
Re: cygwin compilation error
ср, 8 мая 2019 г. в 11:06, Willy Tarreau : > On Wed, May 08, 2019 at 10:59:20AM +0500, ??? wrote: > > travis-ci supports windows builds. > > cool! > my current roadmap is 1) patch fixes SSL variants (already sent to list). without it we are NOT building LibreSSL at all (i.e. we use default openssl-1.0.2 for all builds) 2) BoringSSL 3) update gcc, clang, enable sanitizers 4) cygwin > > > I will add such build a bit later (after > > we settle with current travis-ci fixes) > > ...and this cygwin build issue :-) > > Willy >
Re: cygwin compilation error
On Wed, May 08, 2019 at 10:59:20AM +0500, ??? wrote: > travis-ci supports windows builds. cool! > I will add such build a bit later (after > we settle with current travis-ci fixes) ...and this cygwin build issue :-) Willy
Re: haproxy 2.0 docker images
Hi Aleks, On Mon, May 06, 2019 at 08:17:23AM +0200, Aleksandar Lazic wrote: > > The outputs below raises some questions to me. > > > > * Should in the OPTIONS output also be the EXTRA_OBJS ? That's a good question. I was hesitating but given that the goal is to be able to easily rebuild a similar executable, maybe we should add it indeed. > > * Should PCRE2 be used instead of PCRE ? No opinion :-) > > * Should PRIVATE_CACHE be used in the default build? No, because this one disables inter-process sharing of SSL sessions. > > * Should SLZ be used in the default build? It's just a matter of choice. I personally always build with it for prod servers because it saves a huge amount of memory and some CPU, but it also adds one extra dependency. I'd say that if it doesn't require extra efforts it's worth it. If it adds some packaging burden you can simply drop it and fall back to zlib. > > * Make NS sense in a container image? I don't think so indeed, though it doesn't cost much to keep it, at least so that you use the same build options everywhere. > > * Can DEVICEATLAS 51DEGREES WURFL be used together? > > - From technically point of view >From a technical point of view I don't see any obvious incompatibility. However doing automated builds from all 3 of these might not always be trivial as it will require that you can include these respective libraries, some of which may only be downloaded after registering on their site. Please don't ship an executable built with the dummy libs since it will be useless and misleading (it's only useful for full- featured builds). > > - From license point of view You have to carefully check. I believe at least one of them mentions patents so this can even make the resulting executable look dangerous for some users and make them stay away from your images. Anyway as usual with anything related to licensing, the best advice I could give you is to ask a lawyer :-/ This alone might be a valid reason for not wasting too much time down this road. Cheers, Willy
Re: cygwin compilation error
travis-ci supports windows builds. I will add such build a bit later (after we settle with current travis-ci fixes) ср, 8 мая 2019 г. в 10:52, Willy Tarreau : > Hi, > > On Mon, May 06, 2019 at 12:54:47PM +0300, Gil Bahat wrote: > > Hi, > > > > is cygwin still supported anymore? > > Well, we never know :-) I mean, we're always open to fixes to make it > work as long as they don't impact other platforms. > > > the target seems to be present in the > > Makefiles and I'd love to be able to use it. I'm running into what seems > to > > be a workable linker error: > > > > $ make TARGET=cygwin > > LD haproxy > > src/http_act.o:http_act.c:(.rdata+0x340): multiple definition of > > `.weak.ist_uc.' > > src/ev_poll.o:ev_poll.c:(.rdata+0x20): first defined here > > Aie that's really bad, it means the linker doesn't support weak symbols :-( > Weak symbols are very handy as they are able to be included and linked in > only once if they are used, and not linked if unused. The info I'm finding > on the net suggest that symbols must be resolved at link time, which is the > case here. So maybe it's just a matter of definition. > > I can suggest a few things to try in include/common/ist.h : > > - replace "__weak__" with "weak" just in case it's different there > (I don't even know why I marked it "__weak__", probably just by > mimetism with "__attribute__" and because it worked > > - add "#pragma weak ist_lc" and "#pragma weak ist_uc" in ist.h, > before the definitions > > - add "extern const unsigned char ist_lc[256];" and > "extern const unsigned char ist_uc[256];" before the definitions > > In case one of them is enough to work, we can merge them. > > Thanks, > Willy > >
