Re: [RFC PATCH] couple of reg-tests

2019-01-14 Thread Christopher Faulet 

Le 09/01/2019 à 15:42, Frederic Lecaille a écrit :

On 1/9/19 3:22 PM, Jarno Huuskonen wrote:

Hello Frederic,

On Mon, Jan 07, Frederic Lecaille wrote:


reg-tests/http-rules/h3.vtc fails on my side due to a typo in
the regex with this error:

 h10.0 CLI regexp error: 'missing opening brace after \o'
(@48) (^0x[a-f0-9]+ example\.org
https://www\.example.\org\n0x[a-f0-9]+ subdomain\.example\.org
https://www\.subdomain\.example\.org\n$)

.\org shoulb be replaced by \.org

Could you check on your side why you did not notice this issue please?


For some reason the buggy regex works for me, maybe it depends on
pcre version.
My varnishtest links to centos7 default pcre (pcre-8.32-17.el7.x86_64).


Not that weird. This POSIX is not supported

   9.3.2 BRE Ordinary Characters

An ordinary character is a BRE that matches itself: any character in the
supported character set, except for the BRE special characters listed in
BRE Special Characters.

The interpretation of an ordinary character preceded by a backslash (
'\' ) is undefined, except for:

  The characters ')', '(', '{', and '}'

  The digits 1 to 9 inclusive (see BREs Matching Multiple Characters)

  A character inside a bracket expression

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html


After checking this issue we will merge your patches. Great work!


I'm attaching the patches again, with fixed regex in h3.vtc.
The patches are for recent 2.0dev.

Should reg-tests/README default to [-Dno-htx='#'] instead of -Dno-htx= ?


Perhaps [-Dno-htx=[#]] is even better as the # here is optional? I do
not care on my side.

Do not modify anymore your patch for that.

Anybody to merge Jarno's patches?

Thank you for replying Jarno.



Thanks guys, now merged !

--
Christopher Faulet

Re: [RFC PATCH] couple of reg-tests

2019-01-09 Thread Frederic Lecaille 

On 1/9/19 3:22 PM, Jarno Huuskonen wrote:

Hello Frederic,

On Mon, Jan 07, Frederic Lecaille wrote:


reg-tests/http-rules/h3.vtc fails on my side due to a typo in
the regex with this error:

 h10.0 CLI regexp error: 'missing opening brace after \o'
(@48) (^0x[a-f0-9]+ example\.org
https://www\.example.\org\n0x[a-f0-9]+ subdomain\.example\.org
https://www\.subdomain\.example\.org\n$)

.\org shoulb be replaced by \.org

Could you check on your side why you did not notice this issue please?


For some reason the buggy regex works for me, maybe it depends on
pcre version.
My varnishtest links to centos7 default pcre (pcre-8.32-17.el7.x86_64).


Not that weird. This POSIX is not supported

 9.3.2 BRE Ordinary Characters

An ordinary character is a BRE that matches itself: any character in the 
supported character set, except for the BRE special characters listed in 
BRE Special Characters.


The interpretation of an ordinary character preceded by a backslash ( 
'\' ) is undefined, except for:


The characters ')', '(', '{', and '}'

The digits 1 to 9 inclusive (see BREs Matching Multiple Characters)

A character inside a bracket expression

http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap09.html


After checking this issue we will merge your patches. Great work!


I'm attaching the patches again, with fixed regex in h3.vtc.
The patches are for recent 2.0dev.

Should reg-tests/README default to [-Dno-htx='#'] instead of -Dno-htx= ?


Perhaps [-Dno-htx=[#]] is even better as the # here is optional? I do 
not care on my side.


Do not modify anymore your patch for that.

Anybody to merge Jarno's patches?

Thank you for replying Jarno.

Fred.

Re: [RFC PATCH] couple of reg-tests

2019-01-09 Thread Jarno Huuskonen 
Hello Frederic,

On Mon, Jan 07, Frederic Lecaille wrote:
> 
> reg-tests/http-rules/h3.vtc fails on my side due to a typo in
> the regex with this error:
> 
>  h10.0 CLI regexp error: 'missing opening brace after \o'
> (@48) (^0x[a-f0-9]+ example\.org
> https://www\.example.\org\n0x[a-f0-9]+ subdomain\.example\.org
> https://www\.subdomain\.example\.org\n$)
> 
> .\org shoulb be replaced by \.org
> 
> Could you check on your side why you did not notice this issue please?

For some reason the buggy regex works for me, maybe it depends on
pcre version.
My varnishtest links to centos7 default pcre (pcre-8.32-17.el7.x86_64).

> After checking this issue we will merge your patches. Great work!

I'm attaching the patches again, with fixed regex in h3.vtc.
The patches are for recent 2.0dev.

Should reg-tests/README default to [-Dno-htx='#'] instead of -Dno-htx= ?

-Jarno

-- 
Jarno Huuskonen
>From 1a5a90641ec072d62babbb8ed65c6831998bbdee Mon Sep 17 00:00:00 2001
From: Jarno Huuskonen 
Date: Wed, 9 Jan 2019 13:41:19 +0200
Subject: [PATCH 1/4] REGTESTS: test case for map_regm commit 271022150d

Minimal test case for map_regm commit 271022150d7961b9aa39dbfd88e0c6a4bc48c3ee.
Config and test is adapted from: Daniel Schneller's example
(https://www.mail-archive.com/haproxy@formilux.org/msg30523.html).
---
 reg-tests/http-rules/b0.map |  1 +
 reg-tests/http-rules/b0.vtc | 77 +
 2 files changed, 78 insertions(+)
 create mode 100644 reg-tests/http-rules/b0.map
 create mode 100644 reg-tests/http-rules/b0.vtc

diff --git a/reg-tests/http-rules/b0.map b/reg-tests/http-rules/b0.map
new file mode 100644
index 000..08ffcfb
--- /dev/null
+++ b/reg-tests/http-rules/b0.map
@@ -0,0 +1 @@
+^(.*)\.(.*)$ \1_AND_\2
diff --git a/reg-tests/http-rules/b0.vtc b/reg-tests/http-rules/b0.vtc
new file mode 100644
index 000..897c3b4
--- /dev/null
+++ b/reg-tests/http-rules/b0.vtc
@@ -0,0 +1,77 @@
+#commit 271022150d7961b9aa39dbfd88e0c6a4bc48c3ee
+#BUG/MINOR: map: fix map_regm with backref
+#
+#Due to a cascade of get_trash_chunk calls the sample is
+#corrupted when we want to read it.
+#
+#The fix consist to use a temporary chunk to copy the sample
+#value and use it.
+
+varnishtest "map_regm get_trash_chunk test"
+feature ignore_unknown_macro
+
+#REQUIRE_VERSION=1.6
+syslog S1 -level notice {
+recv
+expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Proxy (fe|be)1 started."
+recv
+expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Proxy (fe|be)1 started."
+recv info
+# not expecting ${h1_pid} with master-worker
+expect ~ "[^:\\[ ]\\[[[:digit:]]+\\]: .* fe1 be1/s1 
[[:digit:]]+/[[:digit:]]+/[[:digit:]]+/[[:digit:]]+/[[:digit:]]+ 200 
[[:digit:]]+ - -  .* \"GET / HTTP/(1|2)(\\.1)?\""
+} -start
+
+server s1 {
+   rxreq
+   expect req.method == "GET"
+   expect req.http.x-mapped-from-header == example_AND_org
+   expect req.http.x-mapped-from-var == example_AND_org
+   txresp
+
+   rxreq
+   expect req.method == "GET"
+   expect req.http.x-mapped-from-header == www.example_AND_org
+   expect req.http.x-mapped-from-var == www.example_AND_org
+   txresp
+} -start
+
+haproxy h1 -conf {
+  global
+log ${S1_addr}:${S1_port} local0 debug err
+
+  defaults
+mode http
+${no-htx} option http-use-htx
+log global
+option httplog
+timeout connect 15ms
+timeout client  20ms
+timeout server  20ms
+
+  frontend fe1
+bind "fd@${fe1}"
+# Remove port from Host header
+http-request replace-value Host '(.*):.*' '\1'
+# Store host header in variable
+http-request set-var(txn.host) req.hdr(Host)
+# This works correctly
+http-request set-header X-Mapped-From-Header 
%[req.hdr(Host),map_regm(${testdir}/b0.map,"unknown")]
+# This breaks before commit 271022150d7961b9aa39dbfd88e0c6a4bc48c3ee
+http-request set-header X-Mapped-From-Var 
%[var(txn.host),map_regm(${testdir}/b0.map,"unknown")]
+
+default_backend be1
+
+backend be1
+server s1 ${s1_addr}:${s1_port}
+} -start
+
+client c1 -connect ${h1_fe1_sock} {
+txreq -hdr "Host: example.org:8443"
+rxresp
+expect resp.status == 200
+
+txreq -hdr "Host: www.example.org"
+rxresp
+expect resp.status == 200
+} -run
+
-- 
1.8.3.1

>From 27b305721d62d5809f8ec400f0c236dd6a51e149 Mon Sep 17 00:00:00 2001
From: Jarno Huuskonen 
Date: Wed, 9 Jan 2019 13:44:44 +0200
Subject: [PATCH 2/4] REGTESTS: Basic tests for concat,strcmp,word,field,ipmask
 converters

---
 reg-tests/http-rules/h2.map |   1 +
 reg-tests/http-rules/h2.vtc | 220 
 2 files changed, 221 insertions(+)
 create mode 100644 reg-tests/http-rules/h2.map
 create mode 100644 reg-tests/http-rules/h2.vtc

diff --git a/reg-tests/http-rules/h2.map b/reg-tests/http-rules/h2.map
new file mode 100644
index 000..9a3e8e6
--- /dev/null
+++

Re: [RFC PATCH] couple of reg-tests

2019-01-07 Thread Frederic Lecaille 

On 1/2/19 2:17 PM, Jarno Huuskonen wrote:

Hello,


Hello Jarno,

Sorry for this late reply.


I started playing with reg-tests and came up with couple of regtests.
Is there a better subdirectory for these than http-rules ? Maybe
map/b0.vtc and converter/h* ?


No, at this time it is ok.



I'm attaching the tests for comments.



reg-tests/http-rules/h3.vtc fails on my side due to a typo in the 
regex with this error:


 h10.0 CLI regexp error: 'missing opening brace after \o' (@48) 
(^0x[a-f0-9]+ example\.org https://www\.example.\org\n0x[a-f0-9]+ 
subdomain\.example\.org https://www\.subdomain\.example\.org\n$)


.\org shoulb be replaced by \.org

Could you check on your side why you did not notice this issue please?

After checking this issue we will merge your patches. Great work!


Thank you a lot.

Fred.

[RFC PATCH] couple of reg-tests

2019-01-02 Thread Jarno Huuskonen 
Hello,

I started playing with reg-tests and came up with couple of regtests.
Is there a better subdirectory for these than http-rules ? Maybe
map/b0.vtc and converter/h* ?

I'm attaching the tests for comments.

-Jarno

-- 
Jarno Huuskonen
>From e75f2ef8b461caa164e81e2d39630e3b2e8791f4 Mon Sep 17 00:00:00 2001
From: Jarno Huuskonen 
Date: Thu, 27 Dec 2018 11:58:13 +0200
Subject: [PATCH 1/4] REGTESTS: test case for map_regm commit 271022150d

Minimal test case for map_regm commit 271022150d7961b9aa39dbfd88e0c6a4bc48c3ee.
Config and test is adapted from: Daniel Schneller's example
(https://www.mail-archive.com/haproxy@formilux.org/msg30523.html).
---
 reg-tests/http-rules/b0.map |  1 +
 reg-tests/http-rules/b0.vtc | 77 +
 2 files changed, 78 insertions(+)
 create mode 100644 reg-tests/http-rules/b0.map
 create mode 100644 reg-tests/http-rules/b0.vtc

diff --git a/reg-tests/http-rules/b0.map b/reg-tests/http-rules/b0.map
new file mode 100644
index 000..08ffcfb
--- /dev/null
+++ b/reg-tests/http-rules/b0.map
@@ -0,0 +1 @@
+^(.*)\.(.*)$ \1_AND_\2
diff --git a/reg-tests/http-rules/b0.vtc b/reg-tests/http-rules/b0.vtc
new file mode 100644
index 000..bdc3b34
--- /dev/null
+++ b/reg-tests/http-rules/b0.vtc
@@ -0,0 +1,77 @@
+#commit 271022150d7961b9aa39dbfd88e0c6a4bc48c3ee
+#BUG/MINOR: map: fix map_regm with backref
+#
+#Due to a cascade of get_trash_chunk calls the sample is
+#corrupted when we want to read it.
+#
+#The fix consist to use a temporary chunk to copy the sample
+#value and use it.
+
+varnishtest "map_regm get_trash_chunk test"
+feature ignore_unknown_macro
+
+#REQUIRE_VERSION=1.6
+syslog S1 -level notice {
+recv
+expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Proxy (fe|be)1 started."
+recv
+expect ~ "[^:\\[ ]\\[${h1_pid}\\]: Proxy (fe|be)1 started."
+recv info
+# not expecting ${h1_pid} with master-worker
+expect ~ "[^:\\[ ]\\[[[:digit:]]+\\]: .* fe1 be1/s1 
[[:digit:]]+/[[:digit:]]+/[[:digit:]]+/[[:digit:]]+/[[:digit:]]+ 200 
[[:digit:]]+ - -  .* \"GET / HTTP/(1|2)(\\.1)?\""
+} -start
+
+server s1 {
+   rxreq
+   expect req.method == "GET"
+   expect req.http.x-mapped-from-header == example_AND_org
+   expect req.http.x-mapped-from-var == example_AND_org
+   txresp
+
+   rxreq
+   expect req.method == "GET"
+   expect req.http.x-mapped-from-header == www.example_AND_org
+   expect req.http.x-mapped-from-var == www.example_AND_org
+   txresp
+} -start
+
+haproxy h1 -conf {
+  global
+log ${S1_addr}:${S1_port} local0 debug err
+
+  defaults
+mode http
+${no-htx} option http-use-htx
+log global
+option httplog
+timeout connect 15ms
+timeout client  20ms
+timeout server  20ms
+
+  frontend fe1
+bind "fd@${fe1}"
+# Remove port from Host header
+http-request replace-value Host '(.*):.*' '\1'
+# Store host header in variable
+http-request set-var(txn.host) req.hdr(Host)
+# This works correctly
+http-request set-header X-Mapped-From-Header 
%[req.hdr(Host),map_regm(${testdir}/b0.map,"unknown")]
+# This breaks before commit 271022150d7961b9aa39dbfd88e0c6a4bc48c3ee
+http-request set-header X-Mapped-From-Var 
%[var(txn.host),map_regm(${testdir}/b0.map,"unknown")]
+
+default_backend be1
+
+backend be1
+server s1 ${s1_addr}:${s1_port}
+} -start
+
+client c1 -connect ${h1_fe1_sock} {
+txreq -hdr "Host: example.org:8443"
+rxresp
+expect resp.status == 200
+
+txreq -hdr "Host: www.example.org"
+rxresp
+expect resp.status == 200
+} -run
+
-- 
1.8.3.1

>From cd8c246769267bfcf69acef29104cef86ace4032 Mon Sep 17 00:00:00 2001
From: Jarno Huuskonen 
Date: Tue, 1 Jan 2019 13:39:52 +0200
Subject: [PATCH 2/4] REGTESTS: Basic tests for using maps to redirect requests
 / select backend

---
 reg-tests/http-rules/h3-be.map |   4 +
 reg-tests/http-rules/h3.map|   3 +
 reg-tests/http-rules/h3.vtc| 174 +
 3 files changed, 181 insertions(+)
 create mode 100644 reg-tests/http-rules/h3-be.map
 create mode 100644 reg-tests/http-rules/h3.map
 create mode 100644 reg-tests/http-rules/h3.vtc

diff --git a/reg-tests/http-rules/h3-be.map 
b/reg-tests/http-rules/h3-be.map
new file mode 100644
index 000..c8822fc
--- /dev/null
+++ b/reg-tests/http-rules/h3-be.map
@@ -0,0 +1,4 @@
+# These entries are used for use_backend rules
+test1.example.com  test1_be
+test1.example.invalid  test1_be
+test2.example.com  test2_be
diff --git a/reg-tests/http-rules/h3.map b/reg-tests/http-rules/h3.map
new file mode 100644
index 000..a0cc02d
--- /dev/null
+++ b/reg-tests/http-rules/h3.map
@@ -0,0 +1,3 @@
+# These entries are used for http-request redirect rules
+example.org https://www.example.org
+subdomain.example.org https://www.subdomain.example.org
diff --git