Re: BUG: Lua tasks can't use client sockets after bf89ff3d

[Willy Tarreau]

Hi Pieter,

On Fri, Nov 30, 2018 at 02:42:26AM +0100, PiBa-NL wrote:
> Attached a new test which does, and does indeed fail on versions since the
> mentioned commit.

Thanks!

> Should i make a patch out of it for inclusion in git? Or can you guys do
> that once the fix is also ready.? i think it was the preferred to get
> bugfix+regtest 'linked' then ?

I prefer to have the tests ASAP, as they are helpful to test bugs. In
my opinion, tests should not be limited to known bugs, since the only
purpose they provide in this case is to help with backports of fixes.
So please send a patch for inclusion, I'll take it.

Thanks!
Willy

Re: [PATCH] REGTEST/MINOR: script: add run-regtests.sh script

[Willy Tarreau]

On Thu, Nov 29, 2018 at 10:04:29PM +0100, Frederic Lecaille wrote:
> Here is a patch for haproxy (named 0001-REGTEST*) to fix these issues.

Works like a charm, thank you Fred! Now applied.

Willy

Re: BUG: Lua tasks can't use client sockets after bf89ff3d

[PiBa-NL]

Hi Frederic, Adis,

Op 29-11-2018 om 14:53 schreef Frederic Lecaille:

Hi Adis,

On 11/29/18 10:03 AM, Adis Nezirovic wrote:

On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote:

OK thanks, I'll take a look at it once I've flushed my pending stuff on
H2+HTX :-(


Great, I had my morning coffee and visited my optometrist, so here is
a fixed test script (correctly setting Host header).

P.S.
Lua usually suffers trying to do things in tasks, I don't think this is
the first time something gets broken. Can we make reg test with Lua
script (maybe strip out LuaSocket requirement)?



Yes. There already exist LUA reg tests in reg-tests/lua directory.

Fred.

Indeed some LUA tests already exists, but it didn't check to use a 
socket from a task.


Attached a new test which does, and does indeed fail on versions since 
the mentioned commit.
Should i make a patch out of it for inclusion in git? Or can you guys do 
that once the fix is also ready.? i think it was the preferred to get 
bugfix+regtest 'linked' then ?


Regards,
PiBa-NL (Pieter)

varnishtest "Lua: txn:get_priv() scope"
feature ignore_unknown_macro

#REQUIRE_OPTIONS=LUA
#REQUIRE_VERSION=1.6

server s1 {
rxreq
txresp -bodylen 20
} -start

haproxy h1 -conf {
global
lua-load ${testdir}/b4.lua

frontend fe1
mode http
bind "fd@${fe1}"
default_backend b1

backend b1
mode http
http-request use-service lua.fakeserv

} -start

client c0 -connect ${h1_fe1_sock} {
txreq -url "/" -hdr "vtcport: ${s1_port}"
rxresp
expect resp.status == 200
} -run


server s1 -wait
local vtc_port = 0

core.register_service("fakeserv", "http", function(applet)
   vtc_port = applet.headers["vtcport"][0]
core.Info("APPLET START")
local response = "OK"
applet:add_header("Server", "haproxy/webstats")
applet:add_header("Content-Length", string.len(response))
applet:add_header("Content-Type", "text/html")
applet:start_response()
applet:send(response)
core.Info("APPLET DONE")
end)

local function cron()
-- wait for until the correct port is set through the c0 request..
while vtc_port == 0 do
core.msleep(1)
end
core.Debug('CRON port:' .. vtc_port)

   local socket = core.tcp()
   local success = socket:connect("127.0.0.1", vtc_port)
core.Info("SOCKET MADE ".. (success or "??"))
if success ~= 1 then
core.Info("CONNECT SOCKET FAILED?")
return
end
local request = "GET / HTTP/1.1\r\n\r\n"
core.Info("SENDING REQUEST")
socket:send(request)
local result = ""
repeat
core.Info("4")
local d = socket:receive("*a")
if d ~= nil then
result = result .. d
end
until d == nil or d == 0
core.Info("Received: "..result)
end

core.register_task(cron)

reg-test failure for /connection/b00000.vtc after commit 3e1f68b

[PiBa-NL]

Hi Olivier, List,

It seems one of the reg-tests /connection/b0.vtc is failing after 
this recent commit.


http://git.haproxy.org/?p=haproxy.git;a=commit;h=3e1f68bcf9adfcd30e3316b0822c2626cc2a6a84

Using HA-Proxy version 1.9-dev8-3e1f68b 2018/11/29 Some of the output 
looks like this:


***  h1    0.0 debug|Using kqueue() as the polling mechanism.
 Slog_1  0.0 syslog|<133>Nov 29 22:44:25 haproxy[79765]: Proxy http 
started.
 Slog_1  0.0 syslog|<133>Nov 29 22:44:25 haproxy[79765]: Proxy 
ssl-offload-http started.
***  h1    0.0 debug|:ssl-offload-http.accept(0005)=000d from 
[::1:59078] ALPN=h2
***  h1    0.0 debug|:ssl-offload-http.clireq[000d:]: 
POST /1 HTTP/1.1
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
user-agent: curl/7.60.0
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
accept: */*
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
content-length: 3
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
content-type: application/x-www-form-urlencoded
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
host: [::1]:37611

***  h1    0.0 debug|:ssl-offload-http.srvcls[000d:adfd]
***  h1    0.0 debug|:ssl-offload-http.clicls[000d:adfd]
***  h1    0.0 debug|:ssl-offload-http.closed[000d:adfd]
 Slog_1  0.0 syslog|<134>Nov 29 22:44:25 haproxy[79765]: ::1:59078 
[29/Nov/2018:22:44:25.752] ssl-offload-http~ ssl-offload-http/http 
0/0/0/-1/1 400 187 - - CH-- 1/1/0/0/0 0/0 "POST /1 HTTP/1.1"
**   Slog_1  0.0 === expect ~ "Connect from .* to 
${h1_ssl_addr}:${h1_ssl_port}"

 Slog_1  0.0 EXPECT FAILED ~ "Connect from .* to ::1:37611"
...
 top   0.1 shell_out|  % Total    % Received % Xferd  Average 
Speed   Time    Time Time  Current
 top   0.1 shell_out| Dload Upload   
Total   Spent    Left  Speed
 top   0.1 shell_out|\r  0 0    0 0    0 0 0  0 
--:--:-- --:--:-- --:--:-- 0\r100    93    0    90 100 3   
5625    187 --:--:-- --:--:-- --:--:--  5812

 top   0.1 shell_out|HTTP/2 400 \r
 top   0.1 shell_out|cache-control: no-cache\r
 top   0.1 shell_out|content-type: text/html\r
 top   0.1 shell_out|\r
 top   0.1 shell_out|400 Bad request
 top   0.1 shell_out|Your browser sent an invalid request.
 top   0.1 shell_out|


While it should look like this where offloaded traffic is forwarded to a 
second http frontend:


***  h1    0.0 debug|:ssl-offload-http.accept(0005)=000d from 
[::1:48710] ALPN=h2
***  h1    0.0 debug|:ssl-offload-http.clireq[000d:]: 
POST /1 HTTP/1.1
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
user-agent: curl/7.60.0
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
accept: */*
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
content-length: 3
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
content-type: application/x-www-form-urlencoded
***  h1    0.0 debug|:ssl-offload-http.clihdr[000d:]: 
host: [::1]:48188
***  h1    0.0 debug|0001:http.accept(0008)=0017 from [::1:48710] 
ALPN=

***  h1    0.0 debug|0001:http.clireq[0017:]: POST /1 HTTP/1.1
***  h1    0.0 debug|0001:http.clihdr[0017:]: user-agent: 
curl/7.60.0

***  h1    0.0 debug|0001:http.clihdr[0017:]: accept: */*
***  h1    0.0 debug|0001:http.clihdr[0017:]: content-length: 3
***  h1    0.0 debug|0001:http.clihdr[0017:]: content-type: 
application/x-www-form-urlencoded

***  h1    0.0 debug|0001:http.clihdr[0017:]: host: [::1]:48188
 Slog_1  0.0 syslog|<134>Nov 29 22:18:35 haproxy[70605]: Connect 
from ::1:48710 to ::1:48188 (http/HTTP)
**   Slog_1  0.0 === expect ~ "Connect from .* to 
${h1_ssl_addr}:${h1_ssl_port}"


Do you see the same? Is more info needed?

Thanks in advance :)

Regards,

PiBa-NL (Pieter)

Re: [PATCH] REGTEST/MINOR: script: add run-regtests.sh script

[Frederic Lecaille]

On 11/29/18 5:36 AM, Willy Tarreau wrote:

Hi guys,

On Wed, Nov 28, 2018 at 11:17:22AM +0100, Frederic Lecaille wrote:

Perhaps we should "chmod +x" this script.


Good point, done here.

However I'm now seeing this when starting it :

## Starting varnishtest ##
Testing with haproxy version: 1.8-dev0-3b0a6d-66
Assert error in start_test(), vtc_main.c line 283:
   Condition((mkdir(tmpdir, 0711)) == 0) not true.
   errno = 13 (Permission denied)
./scripts/run-regtests.sh: line 299: 21484 Aborted 
$VARNISHTEST_PROGRAM $varnishtestparams $verbose $jobcount -l -k -t 10 $testlist
## Gathering failed results ##
find: `/proc/tty/driver': Permission denied
find: `/proc/1/task/1/fd': Permission denied
find: `/proc/1/task/1/fdinfo': Permission denied
find: `/proc/1/task/1/ns': Permission denied
find: `/proc/1/fd': Permission denied
find: `/proc/1/map_files': Permission denied
(...)

Then I stopped it using Ctrl-C.

I'm seeing a few minor issues we must still address :

   - haproxy is started from the path, which means that on all systems
 where it's installed, it will be the one provided by the system and
 not the just built one which will be tested (as happened above),
 which is confusing. I think we shouldn't search for haproxy in the
 path but use $PWD/haproxy or ./haproxy or something like this.

   - it seems there are some issues in the way TMPDIR/TESTDIR are created,
 as it ended up with an empty TESTDIR. In my case, TMPDIR is not set,
 so TESTDIR was set to /tmp/varnishtest_haproxy. mkdir worked (I now
 have this directory), but a test is missing on this mkdir.

   - the way the sub-directory is created is problematic on shared
 development machines, as only the first user will own the directory
 and will thus prevent other users from creating their own overthere,
 thus it's probably preferable not to create an intermediary directory
 in the end.

   - in my case it's mktemp which failed :

 ++ date +%Y-%m-%d_%H-%M-%S
 + TESTRUNDATETIME=2018-11-29_05-03-43
 + TESTDIR=/tmp/varnishtest_haproxy
 + mkdir -p /tmp/varnishtest_haproxy
 ++ mktemp -d /tmp/varnishtest_haproxy/2018-11-29_05-03-43.
 mktemp: cannot make temp dir 
/tmp/varnishtest_haproxy/2018-11-29_05-03-43.: Invalid argument
 + TESTDIR=

 I haven't checked why yet, but we definitely need to test the status
 code for success as well.

   - in my opinion the script is still missing a number of quotes when using
 string variables, especially in the directory names. If someone is crazy
 enough to have TMPDIR="/tmp/Temp Dir", then he'll have some fun.

   - I'm seeing a linux-specific "rm -d" at the end, it's be better to
 replace it with rmdir.

   - there's "/usr/bin/env sh" at the top of the shell script. /bin/sh is
 the portable one, I've spent lots of time in the past editing scripts
 where env was forced to /usr/bin while it was placed in /bin on some
 systems, so I'm pretty certain that this one is not portable at all :-)


Here is a patch for haproxy (named 0001-REGTEST*) to fix these issues.

Note the other patch for varnishtest which is also required (sent to PHK).

Fred.
>From 7d316b74e65be4d75802feefe3d17b37afddf04e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= 
Date: Thu, 29 Nov 2018 21:51:55 +0100
Subject: [PATCH] Add the support for spaces in filenames for the temporaty
 working directory.

---
 bin/varnishtest/vtc_haproxy.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/bin/varnishtest/vtc_haproxy.c b/bin/varnishtest/vtc_haproxy.c
index 9d12ed671..e361aad78 100644
--- a/bin/varnishtest/vtc_haproxy.c
+++ b/bin/varnishtest/vtc_haproxy.c
@@ -477,7 +477,7 @@ haproxy_new(const char *name)
 	h->cli = haproxy_cli_new(h);
 	AN(h->cli);
 
-	bprintf(buf, "rm -rf %s ; mkdir -p %s", h->workdir, h->workdir);
+	bprintf(buf, "rm -rf %s ; mkdir -p \"%s\"", h->workdir, h->workdir);
 	AZ(system(buf));
 
 	VTAILQ_INSERT_TAIL(, h, list);
@@ -561,7 +561,7 @@ haproxy_start(struct haproxy *h)
 
 	VSB_printf(vsb, " %s", VSB_data(h->args));
 
-	VSB_printf(vsb, " -f %s ", h->cfg_fn);
+	VSB_printf(vsb, " -f \"%s\" ", h->cfg_fn);
 
 	if (h->opt_worker || h->opt_daemon) {
 		bprintf(buf, "%s/pid", h->workdir);
@@ -754,7 +754,7 @@ haproxy_write_conf(const struct haproxy *h, const char *cfg, int auto_be)
 	vsb2 = VSB_new_auto();
 	AN(vsb2);
 
-	VSB_printf(vsb, "global\n\tstats socket %s "
+	VSB_printf(vsb, "global\n\tstats socket \"%s\" "
 		   "level admin mode 600\n", h->cli_fn);
 	VSB_printf(vsb, "stats socket \"fd@${cli}\" level admin\n");
 	AZ(VSB_cat(vsb, cfg));
-- 
2.11.0

>From bd6bc4956f3bd005af7c27cf83e78c8cd14aea02 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= 
Date: Thu, 29 Nov 2018 21:41:42 +0100
Subject: [PATCH] REGTEST: Fix several 

Re: [PATCH] REGTEST/MINOR: script: add run-regtests.sh script

[PiBa-NL]

Hi Frederic,

Op 29-11-2018 om 19:18 schreef Frederic Lecaille:

On 11/29/18 8:47 AM, Willy Tarreau wrote:

On Thu, Nov 29, 2018 at 05:36:35AM +0100, Willy Tarreau wrote:
However I'm well aware that it's easier to work on improvements once 
the

script is merged, so what I've done now is to merge it and create a
temporary "reg-tests2" target in the makefile to use it without losing
the existing one. This way everyone can work in parallel, and once the
few issues seem reliably addressed, we can definitely replace the make
target.


Unfortunately ENOCOFFEE struck me this morning and I forgot to commit
my local changes so I merged the unmodified version which replaces the
"reg-test" target.

Thus now we're condemned to quickly fix these small issues :-)


Pieter,

I am having a look at all these issues.

Regards,

Fred.


If that means i don't have to do anything at this moment, thank you! (i 
suppose your turnaround time from issue>fix will also be shorter than 
waiting for my spare evening hour..)
Ill start checking some requirements of existing .vtc . And perhaps 
writing new one's.


Regards,

PiBa-NL (Pieter)

Re: [PATCH] REGTEST/MINOR: script: add run-regtests.sh script

[Frederic Lecaille]

On 11/29/18 8:47 AM, Willy Tarreau wrote:

On Thu, Nov 29, 2018 at 05:36:35AM +0100, Willy Tarreau wrote:

However I'm well aware that it's easier to work on improvements once the
script is merged, so what I've done now is to merge it and create a
temporary "reg-tests2" target in the makefile to use it without losing
the existing one. This way everyone can work in parallel, and once the
few issues seem reliably addressed, we can definitely replace the make
target.


Unfortunately ENOCOFFEE struck me this morning and I forgot to commit
my local changes so I merged the unmodified version which replaces the
"reg-test" target.

Thus now we're condemned to quickly fix these small issues :-)


Pieter,

I am having a look at all these issues.

Regards,

Fred.

Re: [PATCH] BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores session id

[Willy Tarreau]

On Thu, Nov 29, 2018 at 04:33:21PM +0100, Baptiste wrote:
> (first, before I forget, the patch should be backported in 1.8)

OK, I've added this in the commit message.

> Test case was very fun: I have a HAProxy listening on 127.1:443 with my
> cert. When I "curl" it, I could get the list of ciphers sent by the client
> in the log using the relevant fetch and when I used Chrome or Firefox, I
> could not.
(...)

Thanks for the detailed description, it indeed makes sense. I've just merged
it right now.

Cheers,
Willy

Re: [PATCH] BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores session id

[Baptiste]

Hi Willy,

(first, before I forget, the patch should be backported in 1.8)

Test case was very fun: I have a HAProxy listening on 127.1:443 with my
cert. When I "curl" it, I could get the list of ciphers sent by the client
in the log using the relevant fetch and when I used Chrome or Firefox, I
could not.
After reading the code (of both HAProxy and OpenSSL) and tcpduming the
traffic, I saw that curl did not send the SSL session ID while
Chrome/firefox did (I might had one generated long time ago).
When I do run curl/chrome/firefox with the patch enabled, I can get the
cipher list in the log line.

I just RTFM curl, and I can see it now has a ssl session id cache when you
ask for multiple requests on the same line:
Add the following line in a frontend:
 http-request set-header X-TLS %[ssl_fc_cipherlist_xxh,debug]
Run HAProxy in debug mode.
Run the following curl command:
 curl  https://127.0.0.1/ https://127.0.0.1/
and check the output:
0006:f_public.accept(0005)=000b from [127.0.0.1:40747] ALPN=
0006:f_public.clireq[000b:]: GET / HTTP/1.1
0006:f_public.clihdr[000b:]: Host: 127.0.0.1
0006:f_public.clihdr[000b:]: User-Agent: curl/7.47.0
0006:f_public.clihdr[000b:]: Accept: */*
[debug converter] type: sint <4615001522986040631>
0006:default.srvrep[000b:adfd]: HTTP/1.0 503 Service Unavailable
0006:default.srvhdr[000b:adfd]: Cache-Control: no-cache
0006:default.srvhdr[000b:adfd]: Content-Type: text/html
0006:default.srvcls[000b:adfd]
0006:default.clicls[000b:adfd]
0006:default.closed[000b:adfd]
0007:f_public.accept(0005)=000b from [127.0.0.1:40793] ALPN=
0007:f_public.clireq[000b:]: GET / HTTP/1.1
0007:f_public.clihdr[000b:]: Host: 127.0.0.1
0007:f_public.clihdr[000b:]: User-Agent: curl/7.47.0
0007:f_public.clihdr[000b:]: Accept: */*
0007:default.srvrep[000b:adfd]: HTTP/1.0 503 Service Unavailable
0007:default.srvhdr[000b:adfd]: Cache-Control: no-cache
0007:default.srvhdr[000b:adfd]: Content-Type: text/html
0007:default.srvcls[000b:adfd]
0007:default.clicls[000b:adfd]
0007:default.closed[000b:adfd]

You can see the debug converter which displays the list of ciphers for the
first request, when there are no session id, and does not display it for
the second request.

Now, the output when the patch is applied:

:f_public.accept(0005)=000b from [127.0.0.1:45115] ALPN=
:f_public.clireq[000b:]: GET / HTTP/1.1
:f_public.clihdr[000b:]: Host: 127.0.0.1
:f_public.clihdr[000b:]: User-Agent: curl/7.47.0
:f_public.clihdr[000b:]: Accept: */*
[debug converter] type: sint <4615001522986040631>
:default.clicls[000b:adfd]
:default.closed[000b:adfd]
0001:f_public.accept(0005)=000b from [127.0.0.1:45145] ALPN=
0001:f_public.clireq[000b:]: GET / HTTP/1.1
0001:f_public.clihdr[000b:]: Host: 127.0.0.1
0001:f_public.clihdr[000b:]: User-Agent: curl/7.47.0
0001:f_public.clihdr[000b:]: Accept: */*
[debug converter] type: sint <4615001522986040631>
0001:default.srvcls[000b:adfd]
0001:default.clicls[000b:adfd]
0001:default.closed[000b:adfd]

You can see the cipher list for both connections.

I am unfortunately not familiar with reg-test, but I can have a look at it
and contribute one if you want.

Baptiste



On Thu, Nov 29, 2018 at 9:01 AM Willy Tarreau  wrote:

> Hi Baptiste,
>
> On Wed, Nov 28, 2018 at 03:33:20PM +0100, Baptiste wrote:
> > Hi there,
> >
> > There is a small bug with the function involved in the
> ssl_fc_cipherlist_*
> > fetches.
> > ssl_sock_parse_clienthello() improperly parses the session id, which
> leads
> > to not return any client side cipher list when a client send a SSL
> session
> > ID to resume a session.
> > This patch aims at fixing this issue.
>
> Thanks for this. Would you happen to have a test case for this ? I *think*
> it's correct based on some other sample fetch functions which also take a
> single byte for the session ID length, but if you have a simple test case
> it would be great.
>
> Also CCing Thierry who developed it in case he can test it.
>
> > From f2c79803c6bcb69866f54c8a5833bd0178bea64c Mon Sep 17 00:00:00 2001
> > From: Baptiste Assmann 
> > Date: Wed, 28 Nov 2018 15:20:25 +0100
> > Subject: [PATCH] BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores
> session id
> >
> > In ssl_sock_parse_clienthello(), the code considers that SSL Sessionid
> > size is '1', and then considers that the SSL cipher suite is availble
> > right after the session id size information.
> > This actually works in a single case, when the client does not send a
> > session id.
> >
> > This patch fixes this issue by introducing the a propoer way to parse
> > the session id and move forward the cursor by the session id length when
> > required.
> > ---
> >  src/ssl_sock.c | 15 ---
> >  1 file changed, 12 

Re: BUG: Lua tasks can't use client sockets after bf89ff3d

[Frederic Lecaille]

Hi Adis,

On 11/29/18 10:03 AM, Adis Nezirovic wrote:

On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote:

OK thanks, I'll take a look at it once I've flushed my pending stuff on
H2+HTX :-(


Great, I had my morning coffee and visited my optometrist, so here is
a fixed test script (correctly setting Host header).

P.S.
Lua usually suffers trying to do things in tasks, I don't think this is
the first time something gets broken. Can we make reg test with Lua
script (maybe strip out LuaSocket requirement)?



Yes. There already exist LUA reg tests in reg-tests/lua directory.

Fred.

[PATCH] REGTEST: Fix LEVEL 4 script 0 of "connection" module.

[Frederic Lecaille]

Here is a little reg test fix.

Fred.
>From 61fd6486eee833b42a342993706b656537079242 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20L=C3=A9caille?= 
Date: Thu, 29 Nov 2018 14:23:32 +0100
Subject: [PATCH] REGTEST: Fix LEVEL 4 script 0 of "connection" module.

Prevent this script from creating a UNIX socket in ${testdir} which
is the parent directory of the script. Prefer use ${tmpdir} which
is the temporary working directory for the script.
---
 reg-tests/connection/b0.vtc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/reg-tests/connection/b0.vtc b/reg-tests/connection/b0.vtc
index cbb8a7b0..50bb7494 100644
--- a/reg-tests/connection/b0.vtc
+++ b/reg-tests/connection/b0.vtc
@@ -36,14 +36,14 @@ haproxy h1 -conf {
 
 listen http
 bind-process 1
-bind unix@${testdir}/http.socket accept-proxy name ssl-offload-http
+bind unix@${tmpdir}/http.socket accept-proxy name ssl-offload-http
 option forwardfor
 
 listen ssl-offload-http
 option httplog
 bind-process 2-4
 bind "fd@${ssl}" ssl crt ${testdir}/common.pem ssl no-sslv3 alpn h2,http/1.1
-server http unix@${testdir}/http.socket send-proxy
+server http unix@${tmpdir}/http.socket send-proxy
 } -start
 
 
-- 
2.11.0

Re: SSL certs

[Johan Hendriks]

Op 27-11-18 om 02:53 schreef Azim Siddiqui:
> Hello,
>
> Hope you are doing good. We are using HAproxy in our company. But the ssl 
> certs has been expired. I want to renew it. As i can see HAproxy only takes 
> .pem format for certs. So what files should be included in that .pem file ? 
> And can you please tell me how to convert the certs in .pem ?
>
> Thanks & Regards,
> Azeem
>
>
>
To create a .pem file you need to create a file with all the certs an keys.

-BEGIN RSA PRIVATE KEY-

(Your Private Key: your_domain_name.key)

-END RSA PRIVATE KEY-

-BEGIN CERTIFICATE-

(Your Primary SSL certificate: your_domain_name.crt)

-END CERTIFICATE- 

-BEGIN CERTIFICATE-

(Your Intermediate certificate: your_domain_name.bundle) 

-END CERTIFICATE- 

 


You can do it with cat
# cat your_domain_name.key your_domain_name.crt your_domain_name.bundle
> your_domain_name.pem

regards
Johan

Re: BUG: Lua tasks can't use client sockets after bf89ff3d

[Adis Nezirovic]

On Thu, Nov 29, 2018 at 09:03:34AM +0100, Willy Tarreau wrote:
> OK thanks, I'll take a look at it once I've flushed my pending stuff on
> H2+HTX :-(

Great, I had my morning coffee and visited my optometrist, so here is
a fixed test script (correctly setting Host header).

P.S.
Lua usually suffers trying to do things in tasks, I don't think this is
the first time something gets broken. Can we make reg test with Lua
script (maybe strip out LuaSocket requirement)?

Best regards,
-- 
Adis Nezirovic
Software Engineer
HAProxy Technologies - Powering your uptime!
375 Totten Pond Road, Suite 302 | Waltham, MA 02451, US
+1 (844) 222-4340 | https://www.haproxy.com
local http = require 'socket.http'
local ltn12 = require 'ltn12'

local function main(applet)
end

local function cron()
local headers = {
host = 'www.haproxy.org'
}
local content = {}

while true do
core.Debug('CRON')
local b, c, h = http.request {
method='GET',
headers=headers,
url = 'http://51.15.8.218',
sink = ltn12.sink.table(content),
redirect = false,
create = core.tcp
}
core.Debug(tostring(c))
core.msleep(1000)
end
end

core.register_service("cron", "http", main)
core.register_task(cron)
global
log /dev/log local0 debug
nbproc 1
daemon
lua-load cron.lua

defaults
log global
mode http
option httplog
timeout connect 5s
timeout client 10s
timeout server 10s

frontend cron
bind 127.0.0.1:9000
http-request use-service lua.cron

Re: BUG: Lua tasks can't use client sockets after bf89ff3d

[Willy Tarreau]

Hi Adis,

On Wed, Nov 28, 2018 at 05:35:28PM +0100, Adis Nezirovic wrote:
> Hey guys,
> 
> After commit bf89ff3db8be1a8f87de305c144467bbc2503036
> "MEDIUM: stream-int: make stream_int_update() aware of the lower layers"
> I'm not able to use client sockets from tasks created with Lua (see
> attached configuration and Lua script)

Ah crap :-(

I tried to be careful, but was worried at the same time, seeing that
Lua directly manipulates the stream interface's update functions. But
I don't know how to test for this, so I only relied on the reg tests
at the moment.

> You will need LuaSocket library to run the test code, but I also get the
> same issue without it (using raw core.tcp() and custom http library).
> 
> Before the commit cron works fine, connection and http layer (returns
> 404 since http.socket is a little bit stupid, but ignore that).
> 
> After the commit, http.socket reports 'Can't connect' (and 'connection
> closed' afterwards). Debug log shows significant time spent in
> 'stream_int_chk_rcv_applet()'

OK thanks, I'll take a look at it once I've flushed my pending stuff on
H2+HTX :-(

Cheers,
Willy

Re: [PATCH] BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores session id

[Willy Tarreau]

Hi Baptiste,

On Wed, Nov 28, 2018 at 03:33:20PM +0100, Baptiste wrote:
> Hi there,
> 
> There is a small bug with the function involved in the ssl_fc_cipherlist_*
> fetches.
> ssl_sock_parse_clienthello() improperly parses the session id, which leads
> to not return any client side cipher list when a client send a SSL session
> ID to resume a session.
> This patch aims at fixing this issue.

Thanks for this. Would you happen to have a test case for this ? I *think*
it's correct based on some other sample fetch functions which also take a
single byte for the session ID length, but if you have a simple test case
it would be great.

Also CCing Thierry who developed it in case he can test it.

> From f2c79803c6bcb69866f54c8a5833bd0178bea64c Mon Sep 17 00:00:00 2001
> From: Baptiste Assmann 
> Date: Wed, 28 Nov 2018 15:20:25 +0100
> Subject: [PATCH] BUG/MINOR: ssl: ssl_sock_parse_clienthello ignores session id
> 
> In ssl_sock_parse_clienthello(), the code considers that SSL Sessionid
> size is '1', and then considers that the SSL cipher suite is availble
> right after the session id size information.
> This actually works in a single case, when the client does not send a
> session id.
> 
> This patch fixes this issue by introducing the a propoer way to parse
> the session id and move forward the cursor by the session id length when
> required.
> ---
>  src/ssl_sock.c | 15 ---
>  1 file changed, 12 insertions(+), 3 deletions(-)
> 
> diff --git a/src/ssl_sock.c b/src/ssl_sock.c
> index a73fb2d..c48c4af 100644
> --- a/src/ssl_sock.c
> +++ b/src/ssl_sock.c
> @@ -1561,10 +1561,19 @@ void ssl_sock_parse_clienthello(int write_p, int 
> version, int content_type,
>  
>   /* Expect 2 bytes for protocol version (1 byte for major and 1 byte
>* for minor, the random, composed by 4 bytes for the unix time and
> -  * 28 bytes for unix payload, and them 1 byte for the session id. So
> -  * we jump 1 + 1 + 4 + 28 + 1 bytes.
> +  * 28 bytes for unix payload. So we jump 1 + 1 + 4 + 28.
>*/
> - msg += 1 + 1 + 4 + 28 + 1;
> + msg += 1 + 1 + 4 + 28;;

Just a minor detail, extra semi-column above. I'll fix it once merged,
don't resend for this.

> + if (msg > end)
> + return;
> +
> + /* Next, is session id:
> +  * if present, we have to jump by is length + 1 for the size information

extra "is" here, same, no need to resend.

> +  * if not present, we have to jump by 1 only
> +  */
> + if (msg[0] > 0)
> + msg += msg[0];
> + msg += 1;
>   if (msg > end)
>   return;

Thank you,
Willy