Re: [RFC PATCH] BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2

[Willy Tarreau]

On Mon, Jul 01, 2019 at 10:55:41PM +0200, Lukas Tribus wrote:
> Hello Willy,
> 
> On Mon, 1 Jul 2019 at 22:34, Willy Tarreau  wrote:
> >
> > On Mon, Jul 01, 2019 at 10:32:29PM +0200, Lukas Tribus wrote:
> > > Commit 54832b97 ("BUILD: enable several LibreSSL hacks, including")
> > > changed empty handshake detection in OpenSSL <= 1.0.2 and LibreSSL,
> > > from accessing packet_length directly (not available in LibreSSL) to
> > > calling SSL_state() instead.
> > (...)
> >
> > Thanks a lot Lukas. Just out of curiosity, do you have any idea of a
> > concrete user-visible issue this bug can cause ? It would help bisecting
> > issues later. I don't know in what case an empty handshake may happen.
> 
> The investigation was initiated by the following discourse thread:
> https://discourse.haproxy.org/t/haproxy-2-0-ssl-handshake-failure/3954
> 
> It's about an Amazon ELB load-balancer and it's TCP-only health check,
> which does not initiate a SSL handshake after connecting (and in this
> case, sending the proxy protocol). Haproxy 1.9 recognized the missing
> handshake and provides an appropriate log, 2.0 does not and logs a
> handshake failure/error. There is still a disparity between what the
> discourse report is saying (no logs at all for the ELB health checks
> in 1.9), and what I can actually reproduce (empty handshake detection
> causing a "Connection closed during SSL handshake" log message) - but
> I'm gonna ignore that as I don't have any clue why this is.
> 
> So the difference is what we log in this situation; we should not log
> an error or a failure when the SSL handshake didn't even begin.

Ah OK, this makes sense, thanks for the explanation!

Willy

Re: The case for changing the documentation syntax

[Lukas Tribus]

Hello Nick,


On Mon, 1 Jul 2019 at 17:02, Nick Ramirez  wrote:
>
> Hello all,
>
> I'd like to propose something radical, but that will greatly help us in terms 
> of documentation. (And documentation is very important when it comes to 
> people choosing whether to use a piece of software, as I am sure you agree!)
>
> First, the problem: Our documentation at 
> https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt is 
> written using a sort of home-grown syntax that uses various conventions for 
> indicating sections, keywords, etc.
>
> However, parsing this home-grown documentation is difficult. For example, I 
> contribute to the HAProxy Syntax Support for Atom project 
> (https://github.com/abulimov/atom-language-haproxy). This is a python program 
> that must parse the HAProxy configuration.txt file and find the keywords by 
> first finding specific section titles, then looking for lines that don't have 
> spaces in front of them. That's how we find the keywords in each section. It 
> must be updated when new versions of HAProxy are released because new 
> sections are added and the section numbers may change, and some sections are 
> not reliably using the home-grown syntax. In short, parsing configuration.txt 
> is difficult, error-prone and requires regular maintenance because its syntax 
> is:
>
> * Not a standard
> * Not used consistently throughout the document
> * Not easily parsed by existing tools (home-grown tools must be created and 
> maintained)
>
> You may wonder, why do we need to parse configuration.txt? The reasons are:
>
> * A text file without any styling is difficult to read, so we want to add 
> styling (e.g. convert it to HTML with CSS or offer a PDF download)
> * We want search functionality of the document and an auto-generated table of 
> contents
> * We want to write haproxy.cfg files and have them displayed in 
> syntax-highlighted color when using Github Gist or any modern text editor 
> (Atom, VS Code, Sublime Text, etc.). For that, we must currently parse 
> configuration.txt to learn the keywords (as in the atom-language-haproxy 
> project mentioned). For example, we use Github Gist, with the 
> atom-language-haproxy project, to display HAProxy configuration snippets in 
> color on the haproxy.com/blog. It would be easier to maintain this if we 
> could parse configuration.text more easily.


Actually since 7 years we do 2 of the 3 things you mention here;
documentation.txt and others are parsed automatically (in python) and
generate a verify nice HTML site, searchable and indexed with table of
contents, etc:

https://www.mail-archive.com/haproxy@formilux.org/msg07040.html
https://github.com/cbonte/haproxy-dconv
https://cbonte.github.io/haproxy-dconv/


We use this extensively and are able to point people to specific
sections or keywords of the documentation. When the documentation is
inconsistent and breaks the tool, we (or more specifically Cyril)
fixes it. I don't see any 2.0 specific changes in haproxy-dconv, and
I'm not sure if a structured text would fix all the issues you have
with the atom project.

I'm not saying we should maintain configuration.txt as it currently
is, but to me the status-quo does not feel as dire as you suggested.


haproxy-dconv also mentions:

> The purpose of this project is to ultimately convert the HAProxy 
> documentation into a more generic format (example : ReStructuredText) that 
> would allow for an easier spreading of various output files (.pdf, .html, 
> .epub, etc).

So it seems like there is common ground. I'm CCing Cyril who has
invested a lot of time for this already.


I think I agree that we would benefit from moving towards a
standardized, structured text.


Regarding markdown vs reStructuredText vs asciidoc, I don't have a lot
of experience with either of those, but if we go down this road I feel
like we should pick a format that is here to stay and is standardized,
and for me, that is reStructuredText. Markdown is probably the worst
possible choice and I know first hand how the lack of standardization
negatively affects it's interoperability (specifically a site had a JS
based preview that looked different than when the server-side code
parsed it after the submission ... so I have a strong negative opinion
about Markdown).

Readthedocs supports reStructuredText (and discourages but supports
markdown), however asciidoc is not supported. Not that we need to use
readthedocs, but it's something to keep in mind.



cheers,
lukas

Re: [RFC PATCH] BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2

[Илья Шипицин]

вт, 2 июл. 2019 г. в 01:34, Willy Tarreau :

> On Mon, Jul 01, 2019 at 10:32:29PM +0200, Lukas Tribus wrote:
> > Commit 54832b97 ("BUILD: enable several LibreSSL hacks, including")
> > changed empty handshake detection in OpenSSL <= 1.0.2 and LibreSSL,
> > from accessing packet_length directly (not available in LibreSSL) to
> > calling SSL_state() instead.
> (...)
>
> Thanks a lot Lukas. Just out of curiosity, do you have any idea of a
> concrete user-visible issue this bug can cause ? It would help bisecting
> issues later. I don't know in what case an empty handshake may happen.
>

nmap scan ?


>
> Cheers,
> Willy
>

Re: [RFC PATCH] BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2

[Lukas Tribus]

Hello Willy,

On Mon, 1 Jul 2019 at 22:34, Willy Tarreau  wrote:
>
> On Mon, Jul 01, 2019 at 10:32:29PM +0200, Lukas Tribus wrote:
> > Commit 54832b97 ("BUILD: enable several LibreSSL hacks, including")
> > changed empty handshake detection in OpenSSL <= 1.0.2 and LibreSSL,
> > from accessing packet_length directly (not available in LibreSSL) to
> > calling SSL_state() instead.
> (...)
>
> Thanks a lot Lukas. Just out of curiosity, do you have any idea of a
> concrete user-visible issue this bug can cause ? It would help bisecting
> issues later. I don't know in what case an empty handshake may happen.

The investigation was initiated by the following discourse thread:
https://discourse.haproxy.org/t/haproxy-2-0-ssl-handshake-failure/3954

It's about an Amazon ELB load-balancer and it's TCP-only health check,
which does not initiate a SSL handshake after connecting (and in this
case, sending the proxy protocol). Haproxy 1.9 recognized the missing
handshake and provides an appropriate log, 2.0 does not and logs a
handshake failure/error. There is still a disparity between what the
discourse report is saying (no logs at all for the ELB health checks
in 1.9), and what I can actually reproduce (empty handshake detection
causing a "Connection closed during SSL handshake" log message) - but
I'm gonna ignore that as I don't have any clue why this is.

So the difference is what we log in this situation; we should not log
an error or a failure when the SSL handshake didn't even begin.


Lukas

Re: Re[2]: The case for changing the documentation syntax

[Hugues Alary]

Adding my 2 cents here: I write documentation a lot and would like to
mention the Asciidoc format, and more specifically asciidoctor (
https://asciidoctor.org/). Asciidoc is a _very_ powerful syntax yet
extremely simple to use.

Here's a link to their cheat sheet to give you a quick idea of the syntax:
https://asciidoctor.org/docs/asciidoc-syntax-quick-reference/
And a more in depth manual:
https://asciidoctor.org/docs/user-manual/#introduction-to-asciidoctor

On Mon, Jul 1, 2019 at 1:51 PM Nick Ramirez  wrote:

> Yes, either reStructuredText or Markdown would be okay. They both have a
> very intuitive syntax, so newcomers would pick it up and become
> productive with it quickly. It is quite easy to learn either one.
>
>
>
> -- Original Message --
> From: "Aleksandar Lazic" 
> To: "Nick Ramirez" ; "haproxy@formilux.org"
> 
> Sent: 7/1/2019 12:05:15 PM
> Subject: Re: The case for changing the documentation syntax
>
> >Hi Nick.
> >
> >Am 01.07.2019 um 17:01 schrieb Nick Ramirez:
> >>  Hello all,
> >>
> >>  I'd like to propose something radical, but that will greatly help us
> in terms of
> >>  documentation. (And documentation is very important when it comes to
> people
> >>  choosing whether to use a piece of software, as I am sure you agree!)
> >
> >Full Ack. This discussion comes up from time to time and I agree with you
> that a
> >more mainstream format would be nice.
> >
> >>  First, the problem: Our documentation
> >>  at
> https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt is
> >>  written using a sort of home-grown syntax that uses various
> conventions for
> >>  indicating sections, keywords, etc.
> >>
> >>  However, parsing this home-grown documentation is difficult. For
> example, I
> >>  contribute to the HAProxy Syntax Support for Atom project
> >>  (https://github.com/abulimov/atom-language-haproxy). This is a python
> program
> >>  that must parse the HAProxy configuration.txt file and find the
> keywords by
> >>  first finding specific section titles, then looking for lines that
> don't have
> >>  spaces in front of them. That's how we find the keywords in each
> section. It
> >>  must be updated when new versions of HAProxy are released because new
> sections
> >>  are added and the section numbers may change, and some sections are
> not reliably
> >>  using the home-grown syntax. In short, parsing configuration.txt is
> difficult,
> >>  error-prone and requires regular maintenance because its syntax is:
> >>
> >>  * Not a standard
> >>  * Not used consistently throughout the document
> >>  * Not easily parsed by existing tools (home-grown tools must be
> created and
> >>  maintained)
> >>
> >>  You may wonder, why do we need to parse configuration.txt? The reasons
> are:
> >>
> >>  * A text file without any styling is difficult to read, so we want to
> add
> >>  styling (e.g. convert it to HTML with CSS or offer a PDF download)
> >>  * We want search functionality of the document and an auto-generated
> table of
> >>  contents
> >>  * We want to write haproxy.cfg files and have them displayed in
> >>  syntax-highlighted color when using Github Gist or any modern text
> editor (Atom,
> >>  VS Code, Sublime Text, etc.). For that, we must currently parse
> >>  configuration.txt to learn the keywords (as in the
> atom-language-haproxy project
> >>  mentioned). For example, we use Github Gist, with the
> atom-language-haproxy
> >>  project, to display HAProxy configuration snippets in color on the
> >>  haproxy.com/blog. It would be easier to maintain this if we could
> parse
> >>  configuration.text more easily.
> >>
> >>  The solution I am proposing:
> >>
> >>  Rather than using a home-grown, difficult to parse,
> not-consistently-used
> >>  grammar. We should use a standard. We should use
> >>  reStructuredText: http://docutils.sourceforge.net/rst.html
> >>
> >>  The reStructuredText syntax gives us the following benefits:
> >>
> >>  * It is well documented
> >>  * Tools exist to parse this and convert it to other formats (such as
> HTML)
> >>  * Tools exist that will "error check" the document to ensure that the
> correct
> >>  syntax is used throughout configuration.txt (which would become
> configuration.rst)
> >>  * Tools such as Jekyll can easily parse reStructuredText and build
> >>  sophisticated, beautiful webpages that feature search functionality,
> >>  table-of-contents, images, graphs, links, etc. We could really start
> to make the
> >>  documentation shine!
> >>  * We won't have to worry about updating special tools because
> reStructuredText
> >>  syntax will allow us to reliably parse it forever
> >>  * reStructuredText is still easily human-readable using a terminal,
> plain-text
> >>  editor, etc.
> >>
> >>  I and others are fully willing to make the conversion to
> reStructuredText, too.
> >>  What do you all think?
> >
> >I would prefer Markdown with pandoc as I don't like the rst format, but
> I'm fine
> >with what the community and contributes 

[RFC PATCH] BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2

[Lukas Tribus]

Commit 54832b97 ("BUILD: enable several LibreSSL hacks, including")
changed empty handshake detection in OpenSSL <= 1.0.2 and LibreSSL,
from accessing packet_length directly (not available in LibreSSL) to
calling SSL_state() instead.

However, SSL_state() appears to be fully broken in both OpenSSL and
LibreSSL.

Since there is no possibility in LibreSSL to detect an empty handshake,
let's not try (like BoringSSL) and restore this functionality for
OpenSSL 1.0.2 and older, by reverting to the previous behavior.

Should be backported to 2.0.
---

Requesting feedback from Ilya.

---
 src/ssl_sock.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/ssl_sock.c b/src/ssl_sock.c
index c9fffbe..d4b6852 100644
--- a/src/ssl_sock.c
+++ b/src/ssl_sock.c
@@ -5352,7 +5352,7 @@ static int ssl_sock_handshake(struct connection *conn, 
unsigned int flag)
if (!errno && conn->flags & CO_FL_WAIT_L4_CONN)
conn->flags &= ~CO_FL_WAIT_L4_CONN;
if (!conn->err_code) {
-#ifdef OPENSSL_IS_BORINGSSL /* BoringSSL */
+#if defined(OPENSSL_IS_BORINGSSL) || defined(LIBRESSL_VERSION_NUMBER) /* 
BoringSSL or LibreSSL */
conn->err_code = CO_ER_SSL_HANDSHAKE;
 #else
int empty_handshake;
@@ -5360,7 +5360,7 @@ static int ssl_sock_handshake(struct connection *conn, 
unsigned int flag)
OSSL_HANDSHAKE_STATE state = 
SSL_get_state((SSL *)ctx->ssl);
empty_handshake = state == 
TLS_ST_BEFORE;
 #else
-   empty_handshake = SSL_state((SSL 
*)ctx->ssl) == SSL_ST_BEFORE;
+   empty_handshake = 
!ctx->ssl->packet_length;
 #endif
if (empty_handshake) {
if (!errno) {
@@ -5433,7 +5433,7 @@ check_error:
if (!errno && conn->flags & CO_FL_WAIT_L4_CONN)
conn->flags &= ~CO_FL_WAIT_L4_CONN;
if (!conn->err_code) {
-#ifdef OPENSSL_IS_BORINGSSL  /* BoringSSL */
+#if defined(OPENSSL_IS_BORINGSSL) || defined(LIBRESSL_VERSION_NUMBER) /* 
BoringSSL or LibreSSL */
conn->err_code = CO_ER_SSL_HANDSHAKE;
 #else
int empty_handshake;
@@ -5441,7 +5441,7 @@ check_error:
OSSL_HANDSHAKE_STATE state = 
SSL_get_state(ctx->ssl);
empty_handshake = state == TLS_ST_BEFORE;
 #else
-   empty_handshake = SSL_state((SSL *)ctx->ssl) == 
SSL_ST_BEFORE;
+   empty_handshake = !ctx->ssl->packet_length;
 #endif
if (empty_handshake) {
if (!errno) {
-- 
2.7.4

Re: [RFC PATCH] BUG/MINOR: ssl: revert empty handshake detection in OpenSSL <= 1.0.2

[Willy Tarreau]

On Mon, Jul 01, 2019 at 10:32:29PM +0200, Lukas Tribus wrote:
> Commit 54832b97 ("BUILD: enable several LibreSSL hacks, including")
> changed empty handshake detection in OpenSSL <= 1.0.2 and LibreSSL,
> from accessing packet_length directly (not available in LibreSSL) to
> calling SSL_state() instead.
(...)

Thanks a lot Lukas. Just out of curiosity, do you have any idea of a
concrete user-visible issue this bug can cause ? It would help bisecting
issues later. I don't know in what case an empty handshake may happen.

Cheers,
Willy

Re: [PR] BUILD: extend travis-ci matrix

[Илья Шипицин]

wow. I am surprised how proxy protocl might be used.
can we add reg test on that ?

вт, 2 июл. 2019 г. в 01:31, Lukas Tribus :

> Hello Ilya, hello Willy,
>
>
> On Thu, 27 Jun 2019 at 14:04, Lukas Tribus  wrote:
> >
> > Hello,
> >
> > On Thu, 27 Jun 2019 at 13:19, Willy Tarreau  wrote:
> > >
> > > Hi guys,
> > >
> > > On Thu, Jun 27, 2019 at 03:55:54PM +0500,  ??? wrote:
> > > > you are right, commit messages is not my best.
> > > > sorry about that
> > > >
> > > > anyway, if you think it is wrong, you are welcome to provide a fix
> > > > (sometimes, I do wrong things, I'm ok with that)
> > >
> > > Ilya, don't take it bad, bugs always happen and the problem with bugs
> > > is to detect them then to figure how to address them.
> >
> > Indeed, bugs will always happen, it doesn't matter who the author is,
> > and that was not my critique. My issue was merely with the amount of
> > changes and the description.
> >
> > I will provide some more technical context tomorrow.
>
> SSL_state() would return an int, not an enum, so the comparison would
> have to be a bitwise AND (but even OpenSSL got it wrong in their own
> server example code).
>
> Even fixing that, it does not work; neither OpenSSL nor LibreSSL
> recognizes the empty handshake. I assume this broke a long time ago in
> OpenSSL (before the forks).
>
> An alternative, even simpler call available in both OpenSSL and
> LibreSSL would be SSL_in_before(), but that's really just a:
> # define SSL_in_before(a) (SSL_state(a)_ST_BEFORE)
>
> so it fails just like it's parent function.
>
> When I say fail, I mean the comparison is never true and we do not
> recognize the empty handshake, which I simulated with and without
> proxy protocol like this:
>
> echo -ne "PROXY TCP4 172.30.1.153 172.30.1.153 3052 3443\r\n" | nc
> 127.0.0.1 443
> echo -ne "" | nc 127.0.0.1 444
>
>
> I did not find any way in LibreSSL to make it recognize an empty
> handshake, which is why I suggest we remove LibreSSL from the equation
> just like we did with BoringSSL.
>
> I will follow-up with an RFC patch shortly and I'd ask you, Ilya, to
> provide feedback.
>
>
> I have yet to verify a possible additional issue with empty handshake
> detection in 2.0 and -dev for OpenSSL 1.1.0 that came up during
> testing, but are unrelated to this specific change.
>
>
> Regards,
> Lukas
>

Re: [PR] BUILD: extend travis-ci matrix

[Lukas Tribus]

Hello Ilya, hello Willy,


On Thu, 27 Jun 2019 at 14:04, Lukas Tribus  wrote:
>
> Hello,
>
> On Thu, 27 Jun 2019 at 13:19, Willy Tarreau  wrote:
> >
> > Hi guys,
> >
> > On Thu, Jun 27, 2019 at 03:55:54PM +0500,  ??? wrote:
> > > you are right, commit messages is not my best.
> > > sorry about that
> > >
> > > anyway, if you think it is wrong, you are welcome to provide a fix
> > > (sometimes, I do wrong things, I'm ok with that)
> >
> > Ilya, don't take it bad, bugs always happen and the problem with bugs
> > is to detect them then to figure how to address them.
>
> Indeed, bugs will always happen, it doesn't matter who the author is,
> and that was not my critique. My issue was merely with the amount of
> changes and the description.
>
> I will provide some more technical context tomorrow.

SSL_state() would return an int, not an enum, so the comparison would
have to be a bitwise AND (but even OpenSSL got it wrong in their own
server example code).

Even fixing that, it does not work; neither OpenSSL nor LibreSSL
recognizes the empty handshake. I assume this broke a long time ago in
OpenSSL (before the forks).

An alternative, even simpler call available in both OpenSSL and
LibreSSL would be SSL_in_before(), but that's really just a:
# define SSL_in_before(a) (SSL_state(a)_ST_BEFORE)

so it fails just like it's parent function.

When I say fail, I mean the comparison is never true and we do not
recognize the empty handshake, which I simulated with and without
proxy protocol like this:

echo -ne "PROXY TCP4 172.30.1.153 172.30.1.153 3052 3443\r\n" | nc 127.0.0.1 443
echo -ne "" | nc 127.0.0.1 444


I did not find any way in LibreSSL to make it recognize an empty
handshake, which is why I suggest we remove LibreSSL from the equation
just like we did with BoringSSL.

I will follow-up with an RFC patch shortly and I'd ask you, Ilya, to
provide feedback.


I have yet to verify a possible additional issue with empty handshake
detection in 2.0 and -dev for OpenSSL 1.1.0 that came up during
testing, but are unrelated to this specific change.


Regards,
Lukas

Re: Please help for a solution like secure_link

[Aleksandar Lazic]

Tim,

Mon Jul 01 21:36:11 GMT+02:00 2019 Tim Düsterhus :

> Aleks,
 >
 > Am 01.07.19 um 21:27 schrieb Aleksandar Lazic:
 > > Maybe it's also possible with spoe ?
 >
 > I never worked with SPOE before, but I believe it might be possible.
 > SPOE is painful and fragile with HAProxy 1.8, though, because you have
 > to spawn the SPOA manually. Also using SPOE is pretty heavy, because it
 > requires interprocess communication which is not required with Lua.

Yes, that's true.

> Ideally you would just upgrade to HAProxy 2.0 or rebuild to include Lua.

I would like to use 2.0.1 as I switched today to it for my nextcloud and xmpp 
server and it works. :-)

I will try to see what's possible.

Thank you very much for your time an solution.

> Best regards
 > Tim Düsterhus

Best regards
 Aleks

> > Lua requires a rebuild of haproxy , which I want to avoid.
 > >
 > > Mon Jul 01 21:18:42 GMT+02:00 2019 Tim Düsterhus :
 > >
 > >> Aleks,
 > >>
 > >> Am 01.07.19 um 21:16 schrieb Aleksandar Lazic:
 > >>>
 > >>> The concat isn't available in 1.8 any substitution?
 > >>
 > >> Ugh, yeah. Both concat and strcmp are 1.9+. I must've missed that
 > >> requirement. You can use Lua to add yourself a concat and strcmp
 > >> converter. Or you do everything in Lua if you need Lua anyway.
 > >>
 > >> Best regards
 > >> Tim Düsterhus
 > >>
 > >
 >
 >

Re: Please help for a solution like secure_link

[Tim Düsterhus]

Aleks,

Am 01.07.19 um 21:27 schrieb Aleksandar Lazic:
> Maybe it's also possible with spoe ?

I never worked with SPOE before, but I believe it might be possible.
SPOE is painful and fragile with HAProxy 1.8, though, because you have
to spawn the SPOA manually. Also using SPOE is pretty heavy, because it
requires interprocess communication which is not required with Lua.

Ideally you would just upgrade to HAProxy 2.0 or rebuild to include Lua.

Best regards
Tim Düsterhus

> Lua requires a rebuild of haproxy , which I want to avoid.
> 
> Mon Jul 01 21:18:42 GMT+02:00 2019 Tim Düsterhus :
> 
>> Aleks,
>>
>> Am 01.07.19 um 21:16 schrieb Aleksandar Lazic:
>>>
>>> The concat isn't available in 1.8 any substitution?
>>
>> Ugh, yeah. Both concat and strcmp are 1.9+. I must've missed that
>> requirement. You can use Lua to add yourself a concat and strcmp
>> converter. Or you do everything in Lua if you need Lua anyway.
>>
>> Best regards
>> Tim Düsterhus
>>
> 

Re: Please help for a solution like secure_link

[Aleksandar Lazic]

Thanks.

Maybe it's also possible with spoe ?

Lua requires a rebuild of haproxy , which I want to avoid.

Mon Jul 01 21:18:42 GMT+02:00 2019 Tim Düsterhus :

> Aleks,
>
> Am 01.07.19 um 21:16 schrieb Aleksandar Lazic:
> >
> > The concat isn't available in 1.8 any substitution?
>
> Ugh, yeah. Both concat and strcmp are 1.9+. I must've missed that
> requirement. You can use Lua to add yourself a concat and strcmp
> converter. Or you do everything in Lua if you need Lua anyway.
>
> Best regards
> Tim Düsterhus
>

Re: Please help for a solution like secure_link

[Tim Düsterhus]

Aleks,

Am 01.07.19 um 21:16 schrieb Aleksandar Lazic:
> 
> The concat isn't available in 1.8 any substitution?

Ugh, yeah. Both concat and strcmp are 1.9+. I must've missed that
requirement. You can use Lua to add yourself a concat and strcmp
converter. Or you do everything in Lua if you need Lua anyway.

Best regards
Tim Düsterhus

Re: Please help for a solution like secure_link

[Aleksandar Lazic]

The concat isn't available in 1.8 any substitution?

Mon Jul 01 17:56:56 GMT+02:00 2019 Aleksandar Lazic :

> Hi Tim.
>
> Am 01.07.2019 um 17:48 schrieb Tim Düsterhus:
> > Aleks,
> >
> > Am 01.07.19 um 16:16 schrieb Aleksandar Lazic:
> >> My Idea is to use something like this in haproxy but I'm not sure if 
> >> haproxy
> >> only or haproxy+lua is the way to go?
> >
> > If you are fine with sha1 then it's theoretically possible with HAProxy
> > only:
>
> Cool, that was fast, I will try it tommorw and keep you updated.
> I love this community.
>
> >> http-request set-var(txn.sha1) url_param(sha1)
> >> http-request set-var(txn.expires) url_param(expires)
> >> http-request set-var(txn.expected_hash) path,concat(,txn.expires,),sha1,hex
> >>
> >> acl hash_valid var(txn.expected_hash),strcmp(txn.sha1) -m int eq 0
> >> acl expired date,sub(txn.expires) ge 0
> >>
> >> http-response set-header Date %[date]
> >> http-response set-header Expires %[var(txn.expires)]
> >> http-response set-header Expired %[date,sub(txn.expires)] if expired
> >> http-response set-header Not-Expired %[date,sub(txn.expires)] if !expired
> >> http-response set-header Given-Hash %[var(txn.sha1)]
> >> http-response set-header Expected-Hash %[var(txn.expected_hash)]
> >> http-response set-header Hash-Valid true if hash_valid
> >> http-response set-header Hash-Valid false if !hash_valid
> >
> > Inserting a secret is left as an exercise to the reader. Properly using
> > the two ACLs to allow or deny requests is left as an exercise as well.
>
> Yep it's a good start, many thanks.
>
> > NOTE OF CAUTION: The code above is vulnerable to a timing attack,
> > because strcmp does not perform a constant time comparison. The 'hex'
> > converter is not constant time either. The correct way to add the secret
> > would be using HMAC which is not trivial to do (there is no ready
> > converter), if even possible.
>
> Thank you to raise this topic, I will keep it in mind.
>
> > Best regards
> > Tim Düsterhus
>
> Best regards
> Aleks
>
>

Re[2]: The case for changing the documentation syntax

[Nick Ramirez]

Yes, either reStructuredText or Markdown would be okay. They both have a 
very intuitive syntax, so newcomers would pick it up and become 
productive with it quickly. It is quite easy to learn either one.




-- Original Message --
From: "Aleksandar Lazic" 
To: "Nick Ramirez" ; "haproxy@formilux.org" 


Sent: 7/1/2019 12:05:15 PM
Subject: Re: The case for changing the documentation syntax


Hi Nick.

Am 01.07.2019 um 17:01 schrieb Nick Ramirez:

 Hello all,

 I'd like to propose something radical, but that will greatly help us in terms 
of
 documentation. (And documentation is very important when it comes to people
 choosing whether to use a piece of software, as I am sure you agree!)


Full Ack. This discussion comes up from time to time and I agree with you that a
more mainstream format would be nice.


 First, the problem: Our documentation
 at https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt is
 written using a sort of home-grown syntax that uses various conventions for
 indicating sections, keywords, etc.

 However, parsing this home-grown documentation is difficult. For example, I
 contribute to the HAProxy Syntax Support for Atom project
 (https://github.com/abulimov/atom-language-haproxy). This is a python program
 that must parse the HAProxy configuration.txt file and find the keywords by
 first finding specific section titles, then looking for lines that don't have
 spaces in front of them. That's how we find the keywords in each section. It
 must be updated when new versions of HAProxy are released because new sections
 are added and the section numbers may change, and some sections are not 
reliably
 using the home-grown syntax. In short, parsing configuration.txt is difficult,
 error-prone and requires regular maintenance because its syntax is:

 * Not a standard
 * Not used consistently throughout the document
 * Not easily parsed by existing tools (home-grown tools must be created and
 maintained)

 You may wonder, why do we need to parse configuration.txt? The reasons are:

 * A text file without any styling is difficult to read, so we want to add
 styling (e.g. convert it to HTML with CSS or offer a PDF download)
 * We want search functionality of the document and an auto-generated table of
 contents
 * We want to write haproxy.cfg files and have them displayed in
 syntax-highlighted color when using Github Gist or any modern text editor 
(Atom,
 VS Code, Sublime Text, etc.). For that, we must currently parse
 configuration.txt to learn the keywords (as in the atom-language-haproxy 
project
 mentioned). For example, we use Github Gist, with the atom-language-haproxy
 project, to display HAProxy configuration snippets in color on the
 haproxy.com/blog. It would be easier to maintain this if we could parse
 configuration.text more easily.

 The solution I am proposing:

 Rather than using a home-grown, difficult to parse, not-consistently-used
 grammar. We should use a standard. We should use
 reStructuredText: http://docutils.sourceforge.net/rst.html

 The reStructuredText syntax gives us the following benefits:

 * It is well documented
 * Tools exist to parse this and convert it to other formats (such as HTML)
 * Tools exist that will "error check" the document to ensure that the correct
 syntax is used throughout configuration.txt (which would become 
configuration.rst)
 * Tools such as Jekyll can easily parse reStructuredText and build
 sophisticated, beautiful webpages that feature search functionality,
 table-of-contents, images, graphs, links, etc. We could really start to make 
the
 documentation shine!
 * We won't have to worry about updating special tools because reStructuredText
 syntax will allow us to reliably parse it forever
 * reStructuredText is still easily human-readable using a terminal, plain-text
 editor, etc.

 I and others are fully willing to make the conversion to reStructuredText, too.
 What do you all think?


I would prefer Markdown with pandoc as I don't like the rst format, but I'm fine
with what the community and contributes decides.


 Thanks,
 Nick Ramirez


Regards
Aleks

Re: Invalid blank line in master socket output when run in non-admin level

[Daniel MacDougall]

That's great to hear, thanks very much for the fast turnaround!

On Mon, Jul 1, 2019 at 6:56 AM William Lallemand 
wrote:

>
> On Mon, Jul 01, 2019 at 11:16:21AM +0200, William Lallemand wrote:
> > Hi Daniel,
> >
> > On Sat, Jun 29, 2019 at 03:16:06PM +0200, William Lallemand wrote:
> > >
> > > I'll take a look at the code on monday to see if it's easily fixable.
> > >
> >
> > The attached patch should fix your problem.
> >
> > Regards,
> >
>
> I pushed the fix in master, 2.0 and 1.9.
>
>
> --
> William Lallemand
>

Re: The case for changing the documentation syntax

[Aleksandar Lazic]

Hi Nick.

Am 01.07.2019 um 17:01 schrieb Nick Ramirez:
> Hello all,
> 
> I'd like to propose something radical, but that will greatly help us in terms 
> of
> documentation. (And documentation is very important when it comes to people
> choosing whether to use a piece of software, as I am sure you agree!)

Full Ack. This discussion comes up from time to time and I agree with you that a
more mainstream format would be nice.

> First, the problem: Our documentation
> at https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt is
> written using a sort of home-grown syntax that uses various conventions for
> indicating sections, keywords, etc.
> 
> However, parsing this home-grown documentation is difficult. For example, I
> contribute to the HAProxy Syntax Support for Atom project
> (https://github.com/abulimov/atom-language-haproxy). This is a python program
> that must parse the HAProxy configuration.txt file and find the keywords by
> first finding specific section titles, then looking for lines that don't have
> spaces in front of them. That's how we find the keywords in each section. It
> must be updated when new versions of HAProxy are released because new sections
> are added and the section numbers may change, and some sections are not 
> reliably
> using the home-grown syntax. In short, parsing configuration.txt is difficult,
> error-prone and requires regular maintenance because its syntax is:
> 
> * Not a standard
> * Not used consistently throughout the document
> * Not easily parsed by existing tools (home-grown tools must be created and
> maintained)
> 
> You may wonder, why do we need to parse configuration.txt? The reasons are:
> 
> * A text file without any styling is difficult to read, so we want to add
> styling (e.g. convert it to HTML with CSS or offer a PDF download)
> * We want search functionality of the document and an auto-generated table of
> contents
> * We want to write haproxy.cfg files and have them displayed in
> syntax-highlighted color when using Github Gist or any modern text editor 
> (Atom,
> VS Code, Sublime Text, etc.). For that, we must currently parse
> configuration.txt to learn the keywords (as in the atom-language-haproxy 
> project
> mentioned). For example, we use Github Gist, with the atom-language-haproxy
> project, to display HAProxy configuration snippets in color on the
> haproxy.com/blog. It would be easier to maintain this if we could parse
> configuration.text more easily.
> 
> The solution I am proposing:
> 
> Rather than using a home-grown, difficult to parse, not-consistently-used
> grammar. We should use a standard. We should use
> reStructuredText: http://docutils.sourceforge.net/rst.html
>
> The reStructuredText syntax gives us the following benefits:
> 
> * It is well documented
> * Tools exist to parse this and convert it to other formats (such as HTML)
> * Tools exist that will "error check" the document to ensure that the correct
> syntax is used throughout configuration.txt (which would become 
> configuration.rst)
> * Tools such as Jekyll can easily parse reStructuredText and build
> sophisticated, beautiful webpages that feature search functionality,
> table-of-contents, images, graphs, links, etc. We could really start to make 
> the
> documentation shine!
> * We won't have to worry about updating special tools because reStructuredText
> syntax will allow us to reliably parse it forever
> * reStructuredText is still easily human-readable using a terminal, plain-text
> editor, etc.
> 
> I and others are fully willing to make the conversion to reStructuredText, 
> too.
> What do you all think?

I would prefer Markdown with pandoc as I don't like the rst format, but I'm fine
with what the community and contributes decides.

> Thanks,
> Nick Ramirez

Regards
Aleks

Re: Please help for a solution like secure_link

[Aleksandar Lazic]

Hi Tim.

Am 01.07.2019 um 17:48 schrieb Tim Düsterhus:
> Aleks,
> 
> Am 01.07.19 um 16:16 schrieb Aleksandar Lazic:
>> My Idea is to use something like this in haproxy but I'm not sure if haproxy
>> only or haproxy+lua is the way to go?
> 
> If you are fine with sha1 then it's theoretically possible with HAProxy
> only:

Cool, that was fast, I will try it tommorw and keep you updated.
I love this community.

>>  http-request set-var(txn.sha1) url_param(sha1)
>>  http-request set-var(txn.expires) url_param(expires)
>>  http-request set-var(txn.expected_hash) 
>> path,concat(,txn.expires,),sha1,hex
>>
>>  acl hash_valid var(txn.expected_hash),strcmp(txn.sha1) -m int eq 0
>>  acl expired date,sub(txn.expires) ge 0
>>
>>  http-response set-header Date  %[date]
>>  http-response set-header Expires   %[var(txn.expires)]
>>  http-response set-header Expired   %[date,sub(txn.expires)] if  
>> expired
>>  http-response set-header Not-Expired   %[date,sub(txn.expires)] if 
>> !expired
>>  http-response set-header Given-Hash%[var(txn.sha1)]
>>  http-response set-header Expected-Hash %[var(txn.expected_hash)]
>>  http-response set-header Hash-Validtrue  if  hash_valid
>>  http-response set-header Hash-Validfalse if !hash_valid
> 
> Inserting a secret is left as an exercise to the reader. Properly using
> the two ACLs to allow or deny requests is left as an exercise as well.

Yep it's a good start, many thanks.

> NOTE OF CAUTION: The code above is vulnerable to a timing attack,
> because strcmp does not perform a constant time comparison. The 'hex'
> converter is not constant time either. The correct way to add the secret
> would be using HMAC which is not trivial to do (there is no ready
> converter), if even possible.

Thank you to raise this topic, I will keep it in mind.

> Best regards
> Tim Düsterhus

Best regards
Aleks

Re: Please help for a solution like secure_link

[Tim Düsterhus]

Aleks,

Am 01.07.19 um 16:16 schrieb Aleksandar Lazic:
> My Idea is to use something like this in haproxy but I'm not sure if haproxy
> only or haproxy+lua is the way to go?

If you are fine with sha1 then it's theoretically possible with HAProxy
only:

>   http-request set-var(txn.sha1) url_param(sha1)
>   http-request set-var(txn.expires) url_param(expires)
>   http-request set-var(txn.expected_hash) 
> path,concat(,txn.expires,),sha1,hex
> 
>   acl hash_valid var(txn.expected_hash),strcmp(txn.sha1) -m int eq 0
>   acl expired date,sub(txn.expires) ge 0
> 
>   http-response set-header Date  %[date]
>   http-response set-header Expires   %[var(txn.expires)]
>   http-response set-header Expired   %[date,sub(txn.expires)] if  
> expired
>   http-response set-header Not-Expired   %[date,sub(txn.expires)] if 
> !expired>http-response set-header Given-Hash%[var(txn.sha1)]
>   http-response set-header Expected-Hash %[var(txn.expected_hash)]
>   http-response set-header Hash-Validtrue  if  hash_valid
>   http-response set-header Hash-Validfalse if !hash_valid

Inserting a secret is left as an exercise to the reader. Properly using
the two ACLs to allow or deny requests is left as an exercise as well.

NOTE OF CAUTION: The code above is vulnerable to a timing attack,
because strcmp does not perform a constant time comparison. The 'hex'
converter is not constant time either. The correct way to add the secret
would be using HMAC which is not trivial to do (there is no ready
converter), if even possible.

Best regards
Tim Düsterhus

The case for changing the documentation syntax

[Nick Ramirez]

Hello all,

I'd like to propose something radical, but that will greatly help us in 
terms of documentation. (And documentation is very important when it 
comes to people choosing whether to use a piece of software, as I am 
sure you agree!)


First, the problem: Our documentation at 
https://github.com/haproxy/haproxy/blob/master/doc/configuration.txt is 
written using a sort of home-grown syntax that uses various conventions 
for indicating sections, keywords, etc.


However, parsing this home-grown documentation is difficult. For 
example, I contribute to the HAProxy Syntax Support for Atom project 
(https://github.com/abulimov/atom-language-haproxy). This is a python 
program that must parse the HAProxy configuration.txt file and find the 
keywords by first finding specific section titles, then looking for 
lines that don't have spaces in front of them. That's how we find the 
keywords in each section. It must be updated when new versions of 
HAProxy are released because new sections are added and the section 
numbers may change, and some sections are not reliably using the 
home-grown syntax. In short, parsing configuration.txt is difficult, 
error-prone and requires regular maintenance because its syntax is:


* Not a standard
* Not used consistently throughout the document
* Not easily parsed by existing tools (home-grown tools must be created 
and maintained)


You may wonder, why do we need to parse configuration.txt? The reasons 
are:


* A text file without any styling is difficult to read, so we want to 
add styling (e.g. convert it to HTML with CSS or offer a PDF download)
* We want search functionality of the document and an auto-generated 
table of contents
* We want to write haproxy.cfg files and have them displayed in 
syntax-highlighted color when using Github Gist or any modern text 
editor (Atom, VS Code, Sublime Text, etc.). For that, we must currently 
parse configuration.txt to learn the keywords (as in the 
atom-language-haproxy project mentioned). For example, we use Github 
Gist, with the atom-language-haproxy project, to display HAProxy 
configuration snippets in color on the haproxy.com/blog. It would be 
easier to maintain this if we could parse configuration.text more 
easily.


The solution I am proposing:

Rather than using a home-grown, difficult to parse, 
not-consistently-used grammar. We should use a standard. We should use 
reStructuredText: http://docutils.sourceforge.net/rst.html



The reStructuredText syntax gives us the following benefits:

* It is well documented
* Tools exist to parse this and convert it to other formats (such as 
HTML)
* Tools exist that will "error check" the document to ensure that the 
correct syntax is used throughout configuration.txt (which would become 
configuration.rst)
* Tools such as Jekyll can easily parse reStructuredText and build 
sophisticated, beautiful webpages that feature search functionality, 
table-of-contents, images, graphs, links, etc. We could really start to 
make the documentation shine!
* We won't have to worry about updating special tools because 
reStructuredText syntax will allow us to reliably parse it forever
* reStructuredText is still easily human-readable using a terminal, 
plain-text editor, etc.


I and others are fully willing to make the conversion to 
reStructuredText, too. What do you all think?


Thanks,
Nick Ramirez

Please help for a solution like secure_link

[Aleksandar Lazic]

Hi.

I try to implement with haproxy 1.8 the following solution.

https://aws.amazon.com/fr/blogs/networking-and-content-delivery/serving-private-content-using-amazon-cloudfront-aws-lambdaedge/

https://www.nginx.com/blog/securing-urls-secure-link-module-nginx-plus/
https://nginx.org/en/docs/http/ngx_http_secure_link_module.html

In short.

The URL `https://host/secure/myfile?(...&)?md5=...=...` should be 
validated.

```
# where engima is the password.
# Make sure you keep one space between $uri and password

secure_link $arg_md5,$arg_expires;
secure_link_md5 "$secure_link_expires$uri enigma";

if ($secure_link = "") { return 403; }
if ($secure_link = "0") { return 410; }
```

It looks like similar to create a S3 download protection where the application
behind nginx/HAProxy create a MD5 URL which nginx/HAProxy needs to verify before
the client can download the file.

My Idea is to use something like this in haproxy but I'm not sure if haproxy
only or haproxy+lua is the way to go?


ENV SECRET=enigma

```

http-request set-var(sess.md5) url_param(md5)
http-request set-var(sess.expires) url_param(expires)

# is there any md5 function, I haven't seen it in the doc.
acl allow -m str
%[md5(url-without-params,sess.expires,"${SECRET}"),base64,regsub(/=/,'',g),regsub(/+/,
'-',g),regsub(/\//,'_',g)] %[sess.md5]

acl expired -m int %[date(-3600)] %[sess.expires]

http-request deny deny_status 403 if ! allow ! expired
http-request deny deny_status 410 if expired  # <= this is not possible AFAIK
http-request allow if allow

```

How difficult is it to make the
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#7.3.1-base64
compliant https://tools.ietf.org/html/rfc4648#section-5

That's the code from nginx for ngx_decode_base64url.
http://hg.nginx.org/nginx/file/tip/src/core/ngx_string.c#l1228

Any opinions and thanks for help?

Best regards
Aleks

Re: Invalid blank line in master socket output when run in non-admin level

[William Lallemand]

On Mon, Jul 01, 2019 at 11:16:21AM +0200, William Lallemand wrote:
> Hi Daniel,
> 
> On Sat, Jun 29, 2019 at 03:16:06PM +0200, William Lallemand wrote:
> > 
> > I'll take a look at the code on monday to see if it's easily fixable.
> > 
> 
> The attached patch should fix your problem.
> 
> Regards,
> 
 
I pushed the fix in master, 2.0 and 1.9.


-- 
William Lallemand

Re: 1.9 external health checks fail suddenly

[Veiko Kukk]

On 2019-07-01 10:11, Veiko Kukk wrote:

Hi

Sometimes (infrequently) all external checks hang and time out:
* Has happened with versions 1.9.4 and 1.9.8 on multiple servers with
nbproc 1 and nbthread set to (4-12) depending on server.
* Happens infrequently, last one happened after 10 days of uptime.
* External checks are written in python and write errors into their
own log file directly. When hanging happens, nothing is logged by
external check.
* Only external checks fail, common 'option httpcheck' does not fail
at the same time.


Might be useful to add that reload helps to get over, external health 
checks start working again.

Re: 1.9 external health checks fail suddenly

[Lukas Tribus]

Hello,

On Mon, 1 Jul 2019 at 12:14, Lukas Tribus  wrote:
>
> Hello Veiko,
>
>
> On Mon, 1 Jul 2019 at 12:12, Veiko Kukk  wrote:
> >
> > Hi
> >
> > Sometimes (infrequently) all external checks hang and time out:
> > * Has happened with versions 1.9.4 and 1.9.8 on multiple servers with
> > nbproc 1 and nbthread set to (4-12) depending on server.
> > * Happens infrequently, last one happened after 10 days of uptime.
> > * External checks are written in python and write errors into their own
> > log file directly. When hanging happens, nothing is logged by external
> > check.
> > * Only external checks fail, common 'option httpcheck' does not fail at
> > the same time.
>
> External checks are not thread-safe, that's a bug.
>
> Could you try the suggest patch in:
>
> https://github.com/haproxy/haproxy/issues/140#issuecomment-507119534

Sorry, I think I got confused here with something else ... the bug is
about signals being blocked, which is not your problem.



Lukas

Re: 1.9 external health checks fail suddenly

[Lukas Tribus]

Hello Veiko,


On Mon, 1 Jul 2019 at 12:12, Veiko Kukk  wrote:
>
> Hi
>
> Sometimes (infrequently) all external checks hang and time out:
> * Has happened with versions 1.9.4 and 1.9.8 on multiple servers with
> nbproc 1 and nbthread set to (4-12) depending on server.
> * Happens infrequently, last one happened after 10 days of uptime.
> * External checks are written in python and write errors into their own
> log file directly. When hanging happens, nothing is logged by external
> check.
> * Only external checks fail, common 'option httpcheck' does not fail at
> the same time.

External checks are not thread-safe, that's a bug.

Could you try the suggest patch in:

https://github.com/haproxy/haproxy/issues/140#issuecomment-507119534


Thanks,
Lukas

1.9 external health checks fail suddenly

[Veiko Kukk]

Hi

Sometimes (infrequently) all external checks hang and time out:
* Has happened with versions 1.9.4 and 1.9.8 on multiple servers with 
nbproc 1 and nbthread set to (4-12) depending on server.

* Happens infrequently, last one happened after 10 days of uptime.
* External checks are written in python and write errors into their own 
log file directly. When hanging happens, nothing is logged by external 
check.
* Only external checks fail, common 'option httpcheck' does not fail at 
the same time.


HA-Proxy version 1.9.8 2019/05/13 - https://haproxy.org/
Build options :
  TARGET  = linux2628
  CPU = generic
  CC  = gcc
  CFLAGS  = -m64 -march=x86-64 -O2 -g -fno-strict-aliasing 
-Wdeclaration-after-statement -fwrapv -Wno-unused-label 
-Wno-sign-compare -Wno-unused-parameter -Wno-old-style-declaration 
-Wno-ignored-qualifiers -Wno-clobbered -Wno-missing-field-initializers 
-Wtype-limits
  OPTIONS = USE_ZLIB=1 USE_THREAD=1 USE_OPENSSL=1 USE_LUA=1 
USE_STATIC_PCRE=1


Default settings :
  maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 
200


Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : SSLv3 TLSv1.0 TLSv1.1 TLSv1.2
Built with Lua version : Lua 5.3.5
Built with transparent proxy support using: IP_TRANSPARENT 
IPV6_TRANSPARENT IP_FREEBIND

Built with zlib version : 1.2.3
Running on zlib version : 1.2.7
Compression algorithms supported : identity("identity"), 
deflate("deflate"), raw-deflate("deflate"), gzip("gzip")

Built with PCRE version : 7.8 2008-09-05
Running on PCRE version : 7.8 2008-09-05
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Encrypted password support via crypt(3): yes
Built with multi-threading support.

Available polling systems :
  epoll : pref=300,  test result OK
   poll : pref=200,  test result OK
 select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as  cannot be specified using 'proto' 
keyword)

  h2 : mode=HTXside=FE|BE
  h2 : mode=HTTP   side=FE
: mode=HTXside=FE|BE
: mode=TCP|HTTP   side=FE|BE

Available filters :
[SPOE] spoe
[COMP] compression
[CACHE] cache
[TRACE] trace

Veiko

Re: Invalid blank line in master socket output when run in non-admin level

[William Lallemand]

Hi Daniel,

On Sat, Jun 29, 2019 at 03:16:06PM +0200, William Lallemand wrote:
> 
> I'll take a look at the code on monday to see if it's easily fixable.
> 

The attached patch should fix your problem.

Regards,

-- 
William Lallemand
>From 3dffa809d35e6380a958d657e10b9c1885836070 Mon Sep 17 00:00:00 2001
From: William Lallemand 
Date: Mon, 1 Jul 2019 10:56:15 +0200
Subject: [PATCH] BUG/MINOR: mworker/cli: don't output a \n before the response

When using a level lower than admin on the master CLI, a \n is output
before the response, this is caused by the response of the "operator" or
"user" that are sent before the actual command.

To fix this problem we introduce the flag APPCTX_CLI_ST1_NOLF which ask
a command response to not be followed by the final \n.
This patch made a special case with the command operator and user
followed by a - so they are not followed by \n.

This patch must be backported to 2.0.
---
 include/types/applet.h |  1 +
 src/cli.c  | 16 +++-
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/include/types/applet.h b/include/types/applet.h
index c9e02d173..1f3a4983a 100644
--- a/include/types/applet.h
+++ b/include/types/applet.h
@@ -50,6 +50,7 @@ struct applet {
 
 #define APPCTX_CLI_ST1_PROMPT  (1 << 0)
 #define APPCTX_CLI_ST1_PAYLOAD (1 << 1)
+#define APPCTX_CLI_ST1_NOLF(1 << 2)
 
 /* Context of a running applet. */
 struct appctx {
diff --git a/src/cli.c b/src/cli.c
index 44ddc7bfb..9a9f80f90 100644
--- a/src/cli.c
+++ b/src/cli.c
@@ -821,7 +821,7 @@ static void cli_io_handler(struct appctx *appctx)
 		prompt = "\n> ";
 }
 else {
-	if (!(appctx->st1 & APPCTX_CLI_ST1_PAYLOAD))
+	if (!(appctx->st1 & (APPCTX_CLI_ST1_PAYLOAD|APPCTX_CLI_ST1_NOLF)))
 		prompt = "\n";
 }
 
@@ -848,6 +848,8 @@ static void cli_io_handler(struct appctx *appctx)
 
 			/* switch state back to GETREQ to read next requests */
 			appctx->st0 = CLI_ST_GETREQ;
+			/* reactivate the \n at the end of the response for the next command */
+			appctx->st1 &= ~APPCTX_CLI_ST1_NOLF;
 		}
 	}
 
@@ -1442,6 +1444,10 @@ static int cli_parse_show_lvl(char **args, char *payload, struct appctx *appctx,
 /* parse and set the CLI level dynamically */
 static int cli_parse_set_lvl(char **args, char *payload, struct appctx *appctx, void *private)
 {
+	/* this will ask the applet to not output a \n after the command */
+	if (!strcmp(args[1], "-"))
+	appctx->st1 |= APPCTX_CLI_ST1_NOLF;
+
 	if (!strcmp(args[0], "operator")) {
 		if (!cli_has_level(appctx, ACCESS_LVL_OPER)) {
 			return 1;
@@ -2097,11 +2103,11 @@ int pcli_parse_request(struct stream *s, struct channel *req, char **errmsg, int
 		if (pcli_has_level(s, ACCESS_LVL_ADMIN)) {
 			goto end;
 		} else if (pcli_has_level(s, ACCESS_LVL_OPER)) {
-			ci_insert_line2(req, 0, "operator", strlen("operator"));
-			ret += strlen("operator") + 2;
+			ci_insert_line2(req, 0, "operator -", strlen("operator -"));
+			ret += strlen("operator -") + 2;
 		} else if (pcli_has_level(s, ACCESS_LVL_USER)) {
-			ci_insert_line2(req, 0, "user", strlen("user"));
-			ret += strlen("user") + 2;
+			ci_insert_line2(req, 0, "user -", strlen("user -"));
+			ret += strlen("user -") + 2;
 		}
 	}
 end:
-- 
2.21.0

Re: [BUG] memory leak with treads and stick-table/peers

[Emmanuel Hocdet]

Hi,

no more leak after "BUG/MINOR: memory: Set objects size for pools in the 
per-thread cache" 

++
Manu


> Le 5 juin 2019 à 16:13, Emmanuel Hocdet  a écrit :
> 
> 
>> Le 5 juin 2019 à 16:07, Emmanuel Hocdet > > a écrit :
>> 
>> Hi Frederic
>> 
>>> Le 5 juin 2019 à 15:44, Frederic Lecaille >> > a écrit :
>>> 
>>> On 6/5/19 3:06 PM, Emmanuel Hocdet wrote:
 Hi,
>>> 
>>> Hi Emmanuel,
>>> 
 After switched to haproxy 1.9 with threads activated, i noticed a 
 significant memory leak.
>>> 
>>> Is valgrind able to expose this memory leak?
>>> 
 With threads disable (and bind process omitted) leak disappear.
 This seems to be related to stick-table/peers with regard to the 
 (simplified) configuration.
>>> 
>>> Has this been revealed by a git bisect?
>>> 
>> 
>> Report is from production, i need to reproduce this on dev platform to debug 
>> more.
>> 
> 
> As a reminder, the leak was detected after thread enable, on 1.9.5.
>