Re: [PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread Thayne McCombs



Just wondering (maybe something to add to the doc or test): Should 
this handle URL encoded parameter names or parameter values? It 
probably should not, because that's makes the converter less general.


But it would certainly be useful to explain how to properly retrieve 
those values. Simply url-decoding the full query string before doesn't 
do the job, because then the additional delimiters might be 
introduced. This likely needs to be combined with the URI 
normalization feature, as the encoding of a parameter name is not a 
1:1 relationship.


Hmm, Initially I was thinking that it would be sufficient to use 
`urldec` on the result to handle url encoding, but I didn't think about 
the name itself being encoded. I'll add something to the docs 
recommending using uri-normalization for now. Although, I suppose one 
downside to that is it doesn't help if the input doesn't come from the 
uri (for example if it is in the body or a header).






Re: [PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread Tim Düsterhus

Thayne,

On 6/3/22 09:51, astrotha...@gmail.com wrote:

Add a converter that extracts a parameter from string of delimited
key/value pairs.


Just wondering (maybe something to add to the doc or test): Should this 
handle URL encoded parameter names or parameter values? It probably 
should not, because that's makes the converter less general.


But it would certainly be useful to explain how to properly retrieve 
those values. Simply url-decoding the full query string before doesn't 
do the job, because then the additional delimiters might be introduced. 
This likely needs to be combined with the URI normalization feature, as 
the encoding of a parameter name is not a 1:1 relationship.


Best regards
Tim Düsterhus



Re: [PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread Thayne McCombs

There were a couple of things I wasn't entirely sure about:

1. Should this allow specifying the separator between key and value, 
rather than always using "="?


2. How should it handle the case where there isn't a value given, the 
current implementation treats "a&b" as equivalent to "a=&b"





[PATCH 2/2] MINOR : converter: add param converter

2022-06-03 Thread astrothayne
From: Thayne McCombs 

Add a converter that extracts a parameter from string of delimited
key/value pairs.

Fixes: #1697
---
 doc/configuration.txt | 19 +
 reg-tests/converter/param.vtc | 80 +++
 src/sample.c  | 64 ++--
 3 files changed, 160 insertions(+), 3 deletions(-)
 create mode 100644 reg-tests/converter/param.vtc

diff --git a/doc/configuration.txt b/doc/configuration.txt
index 927c97ce3..d9f47c2eb 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -17411,6 +17411,25 @@ or()
   This prefix is followed by a name. The separator is a '.'. The name may only
   contain characters 'a-z', 'A-Z', '0-9', '.' and '_'.

+param(,[])
+  This extracts the first occurence of the parameter  in the input string
+  where parameters are delimited by , which defaults to "&", and the 
name
+  and value of the parameter are separated by a "=". If there is no "=" and 
value
+  before the end of the parameter segment, it is treated as equivalent to a 
value
+  of an empty string.
+
+  This can be useful for extracting parameters from a query string, or 
possibly a
+  x-www-form-urlencoded body. In particular, `query,param()` can be used 
as
+  an alternative to `urlp()` which only uses "&" as a delimiter, whereas 
urlp
+  also uses "?" and ";".
+
+  Example :
+  str(a=b&c=d&a=r),param(a)   # b
+  str(a&b=c),param(a) # ""
+  str(a=&b&c=a),param(b)  # ""
+  str(a=1;b=2;c=4),param(b,;) # 2
+  query,param(redirect_uri)
+
 protobuf(,[])
   This extracts the protocol buffers message field in raw mode of an input 
binary
   sample representation of a protocol buffer message with  as 
field
diff --git a/reg-tests/converter/param.vtc b/reg-tests/converter/param.vtc
new file mode 100644
index 0..163360382
--- /dev/null
+++ b/reg-tests/converter/param.vtc
@@ -0,0 +1,80 @@
+varnishtest "param converter Test"
+
+feature ignore_unknown_macro
+
+server s1 {
+   rxreq
+   txresp -hdr "Connection: close"
+} -repeat 10 -start
+
+haproxy h1 -conf {
+   defaults
+   mode http
+   timeout connect "${HAPROXY_TEST_TIMEOUT-5s}"
+   timeout client  "${HAPROXY_TEST_TIMEOUT-5s}"
+   timeout server  "${HAPROXY_TEST_TIMEOUT-5s}"
+
+   frontend fe
+   bind "fd@${fe}"
+
+   ### requests
+   http-request set-var(txn.query) query
+   http-response set-header Found %[var(txn.query),param(test)] if { 
var(txn.query),param(test) -m found }
+
+   default_backend be
+
+   backend be
+   server s1 ${s1_addr}:${s1_port}
+} -start
+
+client c1 -connect ${h1_fe_sock} {
+   txreq -url "/foo/?test=1&b=4&d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "1"
+
+   txreq -url "/?a=1&b=4&test=34"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "34"
+
+   txreq -url "/?test=bar"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "bar"
+
+   txreq -url "/?a=b&c=d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == ""
+
+   txreq -url "/?a=b&test=t&c=d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == "t"
+
+   txreq -url "/?a=b&test&c=d"
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == ""
+
+   txreq -url "/?test="
+   rxresp
+   expect resp.status == 200
+   expect resp.http.found == ""
+
+txreq -url "/?a=b&test"
+rxresp
+expect resp.status == 200
+expect resp.http.found == ""
+
+txreq -url "/?testing=123"
+rxresp
+expect resp.status == 200
+expect resp.http.found == ""
+
+txreq -url "/?testing=123&test=4"
+rxresp
+expect resp.status == 200
+expect resp.http.found == "4"
+} -run
diff --git a/src/sample.c b/src/sample.c
index 237b88056..b2c80b6c8 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -2582,6 +2582,65 @@ static int sample_conv_word(const struct arg *arg_p, 
struct sample *smp, void *p
return 1;
 }

+static int sample_conv_param_check(struct arg *arg, struct sample_conv *conv,
+   const char *file, int line, char **err)
+{
+   if (arg[1].type == ARGT_STR && arg[1].data.str.data != 1) {
+   memprintf(err, "Delimiter must be exactly 1 character.");
+   return 0;
+   }
+
+   return 1;
+}
+
+static int sample_conv_param(const struct arg *arg_p, struct sample *smp, void 
*private)
+{
+   char *pos, *end, *pend, *equal;
+   char delim = '&';
+   const char *name = arg_p[0].data.str.area;
+   size_t name_l = arg_p[0].data.str.data;
+
+   if (arg_p[1].type == ARGT_STR)
+   delim = *arg_p[1].data.str.area;
+
+   pos = smp->data.u.str.area;
+   end = pos + smp->data.u.str.data;
+   while (pos < end) {
+   equal = pos + name_l;
+   /* Parameter not