Re: [PATCH 1/1] REGTEST: A basic test for "http-buffer-request"

[Willy Tarreau]

Thank you Fred, this is quite useful! I had to modify it a little bit
because it was failing ~60% of the time for me, due to too strict an
expectation on the timers in the logs, which had to be exactly zero
millisecond :-)

This was this one :

+   expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/srv1 0/0/0/0/0 200 .* - 
-  .* .* \"GET / HTTP/1\\.1\""

I changed it to this :

+   expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/srv1 
[0-9]*/[0-9]*/[0-9]*/[0-9]*/[0-9]* 200 .* - -  .* .* \"GET / HTTP/1\\.1\""

And that fixed it for me.

Willy

[ANNOUNCE] haproxy-1.8.16

[William Lallemand]

Hi,

HAProxy 1.8.16 was released on 2018/12/21. It added 3 new commits
after version 1.8.15.

It was not the last 1.8 of the year like it was suspected. :-)

This version fixes 2 overflows in the DNS code and a bug with the startup-logs.

Two bugs were found in the DNS code, and one was introduced by a fix in 1.8.15,
so if you're still using 1.8.14, you should directly update to 1.8.16.

Please find the usual URLs below :
   Site index   : http://www.haproxy.org/
   Discourse: http://discourse.haproxy.org/
   Sources  : http://www.haproxy.org/download/1.8/src/
   Git repository   : http://git.haproxy.org/git/haproxy-1.8.git/
   Git Web browsing : http://git.haproxy.org/?p=haproxy-1.8.git
   Changelog: http://www.haproxy.org/download/1.8/src/CHANGELOG
   Cyril's HTML doc : http://cbonte.github.io/haproxy-dconv/


---
Complete changelog :
Jérôme Magnin (1):
  BUG/MEDIUM: dns: Don't prevent reading the last byte of the payload in 
dns_validate_response()

Nikhil Agrawal (1):
  BUG/MEDIUM: dns: overflowed dns name start position causing invalid dns 
error

Willy Tarreau (1):
  BUG/MINOR: logs: leave startup-logs global and not per-thread

---

-- 
William Lallemand

Re: HA Proxy Load Balancer

[Aleksandar Lazic]

Hi Lance.

Please keep the list in the loop as there are several other persons which can
also help, thank you.

Am 21.12.2018 um 14:49 schrieb Lance Melancon:
> I hope this helps in what you are requesting. So this config works great but I
> need to redirect the server to a sub site as in myserver.net/site. We are
> looking for the exact syntax to add to the haproxy.cfg. I’m including my
> programmer that may understand your feedback better than myself. We did try
> several things referring to the documentation with no luck. Thanks!

docx with embedded Images is not a very secure nor a common format on this list,
due to this fact let me copy the content of the docx here and comment it inline
and answer below.

> Haproxy.cfg:
> global
>log /dev/log local0
>log /dev/log local1 notice
>chroot /var/lib/haproxy
>stats timeout 30s
>user haproxy
>group haproxy
>daemon
>maxconn 15000
> 
> defaults
>log global
>mode http
>option httplog
>option dontlognull
>timeout connect 5000
>timeout client 5
>timeout server 5
> 
> frontend myserver.net
>bind *:443
>mode tcp

Okay here is the problem.

As the haproxy is only used for tcp proxying not for http you will not be able
to make what you want.

https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4-mode

>maxconn 15000
>default_backend hac_cluster
> 
> backend hac_cluster
>mode tcp
>balance leastconn
>server myserver 192.1.1.1:443 check maxconn 5000
>server myserver 192.1.1.2:443 check maxconn 5000
> 
>listen statistics
>bind *:80

I would not recommend to put statistics on port 80, but that's only my opinion.

>mode http
>stats enable
>stats hide-version
>stats refresh 30s
>stats show-node
>stats auth myserver:password   
>stats admin if TRUE
>stats uri /lbstats
> 
> 
> haproxy -vv
>> ## excerpt from image
> Version 1.7.8
> No compression libs, openssl, pcre nor lua support

On which platform is this haproxy running?
Is haproxy installed from the package management or was it build from sources?

To be able to do what you want you will need to do the following steps.

* Install haproxy with openssl support

* get the certificates from the backend server and add it to the haproxy

https://www.haproxy.com/blog/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/
  - Pay attention that you copy teh certificates into the chroot dir
>chroot /var/lib/haproxy

* create a frontend acl for the path `acl my_site path_beg -i /site`

* create a use_backend line `use_backend my_site if my_site`

* create a backend with the name `my_site` with the server line like
  `server myserver myserver.net: ...`

As I mentioned before it's not a easy task to dig into this topic, therefore I
strongly recommend to give you and your programmer some time to understand how
load balancing on level 6(TLS/SSL) + 7(http) works.

Here are some links which could help to get a better picture of HAProxy and LB
in general.
http://www.haproxy.org/download/1.7/doc/intro.txt
https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration/
https://www.haproxy.com/blog/introduction-to-haproxy-acls/

In any case please post some logs, configs or anything directly in the mail body
so that the persons which reads this list via a console are able to follow it
without to open a word document.

We are glad to help as long as we can read the mails ;-)

Very best regards
Aleks


> -Original Message-
> From: Aleksandar Lazic 
> Sent: Thursday, December 20, 2018 4:21 PM
> To: Lance Melancon 
> Cc: haproxy@formilux.org
> Subject: Re: HA Proxy Load Balancer
> 
>  
> 
> CAUTION: This email originated from outside Cypress-Fairbanks ISD. Do not 
> click
> links or open attachments unless you recognize the sender and know the content
> is safe.
> 
>  
> 
>  
> 
>  
> 
> Hi Lance.
> 
>  
> 
> Am 20-12-2018 21:41, schrieb Lance Melancon:
> 
>> Thanks for the info. Unfortunately I am not a programmer by a long
> 
>> shot and syntax is a big problem for me. I tried a few things but no
> 
>> luck and I can't find any examples of a redirect.
> 
>> So do I need both the backend and acl statements?
> 
>> I'm simply trying to use mysite.net to direct to mysite.net/website.
> 
>> Any time I use a / the config fails.
> 
>  
> 
> I'm not sure if you have read and understand my last mail?
> 
> Have you time to dig into this topic as it isn't a quick shot, mostly AFAIK.
> 
>  
> 
> We need some more infos to be able to help you.
> 
>  
> 
>> haproxy -vv
> 
>> anonymized config
> 
>  
> 
> Regards
> 
> Aleks
> 
>  
> 
>> -Original Message-
> 
>> From: Aleksandar Lazic mailto:al-hapr...@none.at>>
> 
>> Sent: Thursday, December 20, 2018 2:00 PM
> 
>> To: Lance Melancon > >
> 
>> Cc: haproxy@formilux.org 
> 
>> Subject: Re: HA Proxy Load Balancer
> 
>> 
> 
>> CAUTION: 

Re: BUG/MAJOR: dns: overflowed dns name start position causing invalid dns error

[Willy Tarreau]

On Fri, Dec 21, 2018 at 04:17:20PM +0530, Nikhil Agrawal wrote:
> Great, Thanks .
> 
> May you backport the fix to 1.7 too?

Sure, that's why I added the info at the end of the commit message.

Willy

[PATCH 1/1] REGTEST: A basic test for "http-buffer-request"

[flecaille]

From: Frédéric Lécaille 

---
 reg-tests/http-messaging/h3.vtc | 78 +
 1 file changed, 78 insertions(+)
 create mode 100644 reg-tests/http-messaging/h3.vtc

diff --git a/reg-tests/http-messaging/h3.vtc 
b/reg-tests/http-messaging/h3.vtc
new file mode 100644
index ..2ce21408
--- /dev/null
+++ b/reg-tests/http-messaging/h3.vtc
@@ -0,0 +1,78 @@
+varnishtest "A test for http-request-buffer option"
+feature ignore_unknown_macro
+
+#REQUIRE_VERSION=1.6
+
+# This test checks HTTP request buffering feature.
+# We run one server s1 which can serve only one client (no -repeat argument 
here).
+# c1 client uses a malformed request which is not transferred to s1 server
+# thanks to "http-buffer-request". If this was the case, c2 client
+# could not connect to s1 server and this would lead to make this test fail.
+
+server s1 {
+   rxreq
+   expect req.bodylen == 257
+   txresp
+} -start
+
+syslog S -level info {
+   recv
+   expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 fe1/ .* 408 .* - - 
cD-- .* .* \"GET 
/this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url-this-is-a-long-url
 HTTP/1\\.1\""
+   recv
+   expect ~ "[^:\\[ ]*\\[[0-9]*\\]: .* .* fe1 be1/srv1 0/0/0/0/0 200 .* - 
-  .* .* \"GET / HTTP/1\\.1\""
+} -start
+
+haproxy h1 -conf {
+   defaults
+   mode http
+   ${no-htx} option http-use-htx
+   timeout client  100ms
+   timeout server  1s
+   timeout connect 1s
+
+   backend be1
+   server srv1 ${s1_addr}:${s1_port}
+
+   frontend fe1
+   option httplog
+   option http-buffer-request
+   log ${S_addr}:${S_port} local0 debug err
+   bind "fd@${fe1}"
+   use_backend be1
+} -start
+
+client c1 -connect ${h1_fe1_sock} {
+   send "GET"
+   send " "
+   send "/this-is-a-long-url"
+   send "-this-is-a-long-url"
+   send "-this-is-a-long-url"
+   send "-this-is-a-long-url"
+   send "-this-is-a-long-url"
+   send "-this-is-a-long-url"
+   send "-this-is-a-long-url"
+   send " HTT"
+   send "P/1.1"
+   send "\r"
+   send "\n"
+   send "Content-Length: 209\r\n\r\n"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   send "abcdefghijklmnopqrstuvwxyz"
+   rxresp
+   expect resp.status == 408
+} -run
+
+client c2 -connect ${h1_fe1_sock} {
+   txreq -bodylen 257
+   rxresp
+   expect resp.status == 200
+} -run
+
+syslog S -wait
+
-- 
2.11.0

[PATCH 0/1] A reg test for "http-buffer-request" option

[flecaille]

From: Frédéric Lécaille 

Fred.

Frédéric Lécaille (1):
  REGTEST: A basic test for "http-buffer-request"

 reg-tests/http-messaging/h3.vtc | 78 +
 1 file changed, 78 insertions(+)
 create mode 100644 reg-tests/http-messaging/h3.vtc

-- 
2.11.0

Re: BUG/MAJOR: dns: overflowed dns name start position causing invalid dns error

[Nikhil Agrawal]

Great, Thanks .

May you backport the fix to 1.7 too?

On Fri, Dec 21, 2018 at 4:07 PM Willy Tarreau  wrote:

> On Fri, Dec 21, 2018 at 03:40:53PM +0530, Nikhil Agrawal wrote:
> > Hi Willy,
> >
> > Thanks for response and correcting the mistake.
> >
> > Its OK for you to fix this.
> > marking this as MEDIUM is also fine :).
>
> OK thanks, now merged. It will be included into next 1.8 ASAP.
>
> Willy
>

Re: BUG/MAJOR: dns: overflowed dns name start position causing invalid dns error

[Willy Tarreau]

On Fri, Dec 21, 2018 at 03:40:53PM +0530, Nikhil Agrawal wrote:
> Hi Willy,
> 
> Thanks for response and correcting the mistake.
> 
> Its OK for you to fix this.
> marking this as MEDIUM is also fine :).

OK thanks, now merged. It will be included into next 1.8 ASAP.

Willy

Re: BUG/MAJOR: dns: overflowed dns name start position causing invalid dns error

[Nikhil Agrawal]

Hi Willy,

Thanks for response and correcting the mistake.

Its OK for you to fix this.
marking this as MEDIUM is also fine :).

Regards,
Nikhil Agrawal



On Fri, Dec 21, 2018 at 2:58 PM Willy Tarreau  wrote:

> Hi,
>
> On Fri, Dec 21, 2018 at 11:02:24AM +0530, Nikhil Agrawal wrote:
> > Dear Haproxy Maintainers,
> >
> > In dns_read_name() when dns name is used with name compression and start
> > position of name is greater than 255, name is read from incorrect
> position
> > ( actual position%256). This causes "Invalid dns error" and backend is
> > marked as down permanently.
> > eg: hexadecimal value at start of dns "0xc1 1b". "0xc" specifies name
> > compression being used. in this scenario only "1b" (27) is taken as the
> > start of the name but actual name starts from "11b" (283).
> >
> >
> > This is a regression scenario starting from version 1.7.0 and is present
> in
> > current version.
> >
> > Resolution:
> > Include nibble from byte used for checking compression. i.e take "11b" as
> > start position of dns name.
> >
> > i am attaching patch to fix this bug.
>
> It's indeed a bug, however after checking RFC1035, your patch is still
> bogus as it only includes 4 of the 6 bits :
>
>   https://tools.ietf.org/html/rfc1035#section-4.1.4
>
> The pointer takes the form of a two octet sequence:
>
> +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
> | 1  1|OFFSET   |
> +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
> ^ ^
> | |
> | +- your patch starts here
> +--- the standard says the length starts here
>
> If you're OK with this I can simply fix it, just let me know. Also I'd
> relabel it as medium, as it's an annoyance that requires some efforts to
> be worked around, but not a major loss of functionality.
>
> Thanks for the test case by the way ;-)
>
> Willy
>

Re: BUG/MAJOR: dns: overflowed dns name start position causing invalid dns error

[Willy Tarreau]

Hi,

On Fri, Dec 21, 2018 at 11:02:24AM +0530, Nikhil Agrawal wrote:
> Dear Haproxy Maintainers,
> 
> In dns_read_name() when dns name is used with name compression and start
> position of name is greater than 255, name is read from incorrect position
> ( actual position%256). This causes "Invalid dns error" and backend is
> marked as down permanently.
> eg: hexadecimal value at start of dns "0xc1 1b". "0xc" specifies name
> compression being used. in this scenario only "1b" (27) is taken as the
> start of the name but actual name starts from "11b" (283).
>
> 
> This is a regression scenario starting from version 1.7.0 and is present in
> current version.
> 
> Resolution:
> Include nibble from byte used for checking compression. i.e take "11b" as
> start position of dns name.
> 
> i am attaching patch to fix this bug.

It's indeed a bug, however after checking RFC1035, your patch is still
bogus as it only includes 4 of the 6 bits :

  https://tools.ietf.org/html/rfc1035#section-4.1.4

The pointer takes the form of a two octet sequence:

+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
| 1  1|OFFSET   |
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
^ ^
| |
| +- your patch starts here
+--- the standard says the length starts here

If you're OK with this I can simply fix it, just let me know. Also I'd
relabel it as medium, as it's an annoyance that requires some efforts to
be worked around, but not a major loss of functionality.

Thanks for the test case by the way ;-)

Willy

Re: HA Proxy Load Balancer

[Veiko Kukk]

On 2018-12-20 20:41, Lance Melancon wrote:

Thanks for the info. Unfortunately I am not a programmer by a long
shot and syntax is a big problem for me. I tried a few things but no
luck and I can't find any examples of a redirect.
So do I need both the backend and acl statements?
I'm simply trying to use mysite.net to direct to mysite.net/website.
Any time I use a / the config fails.


Maybe this will help you 
http://www.catb.org/esr/faqs/smart-questions.html


Veiko