On 12/04/2021 23:13, Aleksandar Lazic wrote:
> Hi Moemen,
>
> any chance to get this feature before 2.4 will be realeased?
>
> Regards
> Aleks
>
Hi Aleksandar,
I have updated the patch (attached) so it gets reviewed and eventually
merged.
I know this is going to be useful with what you are trying to do with
the json converter so I will try to be more active on this.
On 06/04/2021 09:13, Willy Tarreau wrote:
>> in such case should we rather use dynamic allocation ?
>
> No, there are two possible approaches. One of them is to use a trash
> buffer using get_trash_chunk(). The trash buffers are "large enough"
> for anything that comes from outside. A second, cleaner solution
> simply consists in not using a temporary buffer but doing the conversion
> on the fly. Indeed, looking closer, what the function does is to first
> replace a few chars on the whole chain to then call the base64 conversion
> function. So it doubles the work on the string and one side effect of
> this double work is that you need a temporary storage.
The url variant is not only about a different alphabet that needs to be
replaced but also is a non padding variant. So the straightforward
algorithm to decoding it is to add the padding to the input encoded in
url variant and then use the standard base64 decoder.
Even doing this on the fly requires extending input with two more bytes
max. Unless I am missing smth but in such case on the fly conversion
will result in a out of bound array access. That's why I have copied
input in a "inlen+2" string.
In the end I have updated patch to handle extending input in decoding
function via get_trash_chunk to make sure a buffer of size input+2 is
available.
You can find attached the patches 0001 and 0002 for this implementation.
> Other approaches would consist in either reimplementing the functions
> with a different alphabet, or modifying the existing ones to take an
> extra argument for the conversion table, and make one set of functions
> making use of the current table and another set making use of your new
> table.
>
> Willy
>
You can find attached the patches 0001-bis and 0002-bis modifying the
existing functions (introducing an url flag) to see how it looks like.
This solution may be cleaner (no chunk allocation and we don't loop
twice over input string) but has the drawbacks of being intrusive with
the rest of the code and less clearer imo regarding how url variant is
different from standard base64.
Feel free to pick the one that looks better otherwise I can continue
with a different implementation if needbe.
--
Moemen
>From cf0a43dab4f5f88ddf5e5e736127132721b7f18e Mon Sep 17 00:00:00 2001
From: Moemen MHEDHBI <mmhed...@haproxy.com>
Date: Thu, 1 Apr 2021 20:53:59 +0200
Subject: [PATCH 1/2] MINOR: sample: add ub64dec and ub64enc converters
ub64dec and ub64enc are the base64url equivalent of b64dec and base64
converters. base64url encoding is the "URL and Filename Safe Alphabet"
variant of base64 encoding. It is also used in in JWT (JSON Web Token)
standard.
RFC1421 mention in base64.c file is deprecated so it was replaced with
RFC4648 to which existing converters, base64/b64dec, still apply.
Example:
HAProxy:
http-request return content-type text/plain lf-string %[req.hdr(Authorization),word(2,.),ub64dec]
Client:
Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiZm9vIiwia2V5IjoiY2hhZTZBaFhhaTZlIn0.5VsVj7mdxVvo1wP5c0dVHnr-S_khnIdFkThqvwukmdg
$ curl -H "Authorization: Bearer ${TOKEN}" http://haproxy.local
{"user":"foo","key":"chae6AhXai6e"}
---
doc/configuration.txt | 12 ++++++
include/haproxy/base64.h | 2 +
reg-tests/sample_fetches/ubase64.vtc | 45 +++++++++++++++++++
src/base64.c | 64 +++++++++++++++++++++++++++-
src/sample.c | 38 +++++++++++++++++
5 files changed, 160 insertions(+), 1 deletion(-)
create mode 100644 reg-tests/sample_fetches/ubase64.vtc
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 7048fb63e..c7fe416e5 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -16393,6 +16393,18 @@ table_trackers(<table>)
connections there are from a given address for example. See also the
sc_trackers sample fetch keyword.
+ub64dec
+ This converter is the base64url variant of b64dec converter. base64url
+ encoding is the "URL and Filename Safe Alphabet" variant of base64 encoding.
+ It is also the encoding used in JWT (JSON Web Token) standard.
+
+ Example:
+ # Decoding a JWT payload:
+ http-request set-var(txn.token_payload) req.hdr(Authorization),word(2,.),ub64dec
+
+ub64enc
+ This converter is the base64url variant of base64 converter.
+
upper
Convert a string sample to upper case. This can only be placed after a string
sample fetch function or after a transformation keyword returning a string
diff --git a/include/haproxy/base64.h b/include/haproxy/base64.h
index 1756bc058..532c46a44 100644
--- a/include/haproxy/base64.h
+++ b/include/haproxy/base64.h
@@ -17,7 +17,9 @@
#include <haproxy/api.h>
int a2base64(char *in, int ilen, char *out, int olen);
+int a2base64url(char *in, size_t ilen, char *out, size_t olen);
int base64dec(const char *in, size_t ilen, char *out, size_t olen);
+int base64urldec(const char *in, size_t ilen, char *out, size_t olen);
const char *s30tob64(int in, char *out);
int b64tos30(const char *in);
diff --git a/reg-tests/sample_fetches/ubase64.vtc b/reg-tests/sample_fetches/ubase64.vtc
new file mode 100644
index 000000000..dd32cd526
--- /dev/null
+++ b/reg-tests/sample_fetches/ubase64.vtc
@@ -0,0 +1,45 @@
+varnishtest "ub64dec sample fetche Test"
+
+#REQUIRE_VERSION=2.4
+
+feature ignore_unknown_macro
+
+haproxy h1 -conf {
+ defaults
+ mode http
+ timeout connect 1s
+ timeout client 1s
+ timeout server 1s
+
+ frontend fe
+ bind "fd@${fe}"
+ http-request return content-type text/plain hdr encode %[hdr(encode),ub64enc] hdr decode %[hdr(decode),ub64dec]
+
+} -start
+
+client c1 -connect ${h1_fe_sock} {
+ txreq -hdr "encode: f" -hdr "decode: Zg"
+ rxresp
+ expect resp.http.encode == "Zg"
+ expect resp.http.decode == "f"
+ txreq -hdr "encode: fo" -hdr "decode: Zm8"
+ rxresp
+ expect resp.http.encode == "Zm8"
+ expect resp.http.decode == "fo"
+ txreq -hdr "encode: foo" -hdr "decode: Zm9v"
+ rxresp
+ expect resp.http.encode == "Zm9v"
+ expect resp.http.decode == "foo"
+ txreq -hdr "encode: foob" -hdr "decode: Zm9vYg"
+ rxresp
+ expect resp.http.encode == "Zm9vYg"
+ expect resp.http.decode == "foob"
+ txreq -hdr "encode: fooba" -hdr "decode: Zm9vYmE"
+ rxresp
+ expect resp.http.encode == "Zm9vYmE"
+ expect resp.http.decode == "fooba"
+ txreq -hdr "encode: foobar" -hdr "decode: Zm9vYmFy"
+ rxresp
+ expect resp.http.encode == "Zm9vYmFy"
+ expect resp.http.decode == "foobar"
+} -run
diff --git a/src/base64.c b/src/base64.c
index 53e4d65b2..6e911bb29 100644
--- a/src/base64.c
+++ b/src/base64.c
@@ -1,5 +1,5 @@
/*
- * ASCII <-> Base64 conversion as described in RFC1421.
+ * ASCII <-> Base64 conversion as described in RFC4648.
*
* Copyright 2006-2010 Willy Tarreau <w...@1wt.eu>
* Copyright 2009-2010 Krzysztof Piotr Oledzki <o...@ans.pl>
@@ -16,6 +16,7 @@
#include <haproxy/api.h>
#include <haproxy/base64.h>
+#include <haproxy/chunk.h>
#define B64BASE '#' /* arbitrary chosen base value */
#define B64CMIN '+'
@@ -138,6 +139,67 @@ int base64dec(const char *in, size_t ilen, char *out, size_t olen) {
return convlen;
}
+/* url variant of a2base64 */
+int a2base64url(char *in, size_t ilen, char *out, size_t olen)
+{
+ int convlen, i;
+
+ convlen = a2base64(in, ilen, out, olen);
+ while (out[convlen - 1] == '=') {
+ convlen--;
+ out[convlen] = '\0';
+ }
+ for (i = 0; i < convlen; i++) {
+ if (out[i] == '+')
+ out[i] = '-';
+ if (out[i] == '/')
+ out[i] = '_';
+ }
+ return convlen;
+}
+
+/* url variant of base64dec */
+int base64urldec(const char *in, size_t ilen, char *out, size_t olen)
+{
+ struct buffer *trash = get_trash_chunk();
+ char *conv = trash->area;
+
+ if (trash->size < (ilen + 2))
+ return -1;
+
+ for(int i = 0; i < ilen; i++) {
+ switch (in[i]) {
+ case '-':
+ conv[i] = '+';
+ break;
+ case '_':
+ conv[i] = '/';
+ break;
+ default:
+ conv[i] = in[i];
+ }
+ }
+ switch (ilen % 4) {
+ case 0:
+ break;
+ case 2:
+ conv[ilen] = '=';
+ conv[ilen + 1] = '=';
+ conv[ilen + 2] = '\0';
+ ilen += 2;
+ break;
+ case 3:
+ conv[ilen] = '=';
+ conv[ilen + 1] = '\0';
+ ilen += 1;
+ break;
+ default:
+ return -1;
+ }
+
+ return base64dec(conv, ilen, out, olen);
+}
+
/* Converts the lower 30 bits of an integer to a 5-char base64 string. The
* caller is responsible for ensuring that the output buffer can accept 6 bytes
diff --git a/src/sample.c b/src/sample.c
index 835a18115..e36e2e2ec 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -1567,6 +1567,24 @@ static int sample_conv_base642bin(const struct arg *arg_p, struct sample *smp, v
return 1;
}
+static int sample_conv_base64url2bin(const struct arg *arg_p, struct sample *smp, void *private)
+{
+ struct buffer *trash = get_trash_chunk();
+ int bin_len;
+
+ trash->data = 0;
+ bin_len = base64urldec(smp->data.u.str.area, smp->data.u.str.data,
+ trash->area, trash->size);
+ if (bin_len < 0)
+ return 0;
+
+ trash->data = bin_len;
+ smp->data.u.str = *trash;
+ smp->data.type = SMP_T_BIN;
+ smp->flags &= ~SMP_F_CONST;
+ return 1;
+}
+
static int sample_conv_bin2base64(const struct arg *arg_p, struct sample *smp, void *private)
{
struct buffer *trash = get_trash_chunk();
@@ -1585,6 +1603,24 @@ static int sample_conv_bin2base64(const struct arg *arg_p, struct sample *smp, v
return 1;
}
+static int sample_conv_bin2base64url(const struct arg *arg_p, struct sample *smp, void *private)
+{
+ struct buffer *trash = get_trash_chunk();
+ int b64_len;
+
+ trash->data = 0;
+ b64_len = a2base64url(smp->data.u.str.area, smp->data.u.str.data,
+ trash->area, trash->size);
+ if (b64_len < 0)
+ return 0;
+
+ trash->data = b64_len;
+ smp->data.u.str = *trash;
+ smp->data.type = SMP_T_STR;
+ smp->flags &= ~SMP_F_CONST;
+ return 1;
+}
+
static int sample_conv_sha1(const struct arg *arg_p, struct sample *smp, void *private)
{
blk_SHA_CTX ctx;
@@ -4096,6 +4132,8 @@ static struct sample_conv_kw_list sample_conv_kws = {ILH, {
{ "debug", sample_conv_debug, ARG2(0,STR,STR), smp_check_debug, SMP_T_ANY, SMP_T_ANY },
{ "b64dec", sample_conv_base642bin,0, NULL, SMP_T_STR, SMP_T_BIN },
{ "base64", sample_conv_bin2base64,0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "ub64dec", sample_conv_base64url2bin,0, NULL, SMP_T_STR, SMP_T_BIN },
+ { "ub64enc", sample_conv_bin2base64url,0, NULL, SMP_T_BIN, SMP_T_STR },
{ "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
--
2.31.1
>From bbc4b93d1c6cb23ef057b642a5b0cbb88e7a0719 Mon Sep 17 00:00:00 2001
From: Moemen MHEDHBI <mmhed...@haproxy.com>
Date: Fri, 2 Apr 2021 01:05:07 +0200
Subject: [PATCH 2/2] CLEANUP: align samples list in sample.c
---
src/sample.c | 54 ++++++++++++++++++++++++++--------------------------
1 file changed, 27 insertions(+), 27 deletions(-)
diff --git a/src/sample.c b/src/sample.c
index e36e2e2ec..9dd05f648 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -4129,33 +4129,33 @@ INITCALL1(STG_REGISTER, sample_register_fetches, &smp_kws);
/* Note: must not be declared <const> as its list will be overwritten */
static struct sample_conv_kw_list sample_conv_kws = {ILH, {
- { "debug", sample_conv_debug, ARG2(0,STR,STR), smp_check_debug, SMP_T_ANY, SMP_T_ANY },
- { "b64dec", sample_conv_base642bin,0, NULL, SMP_T_STR, SMP_T_BIN },
- { "base64", sample_conv_bin2base64,0, NULL, SMP_T_BIN, SMP_T_STR },
- { "ub64dec", sample_conv_base64url2bin,0, NULL, SMP_T_STR, SMP_T_BIN },
- { "ub64enc", sample_conv_bin2base64url,0, NULL, SMP_T_BIN, SMP_T_STR },
- { "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
- { "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
- { "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
- { "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
- { "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT },
- { "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_IPV4 },
- { "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
- { "utime", sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
- { "crc32", sample_conv_crc32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "crc32c", sample_conv_crc32c, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "djb2", sample_conv_djb2, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "sdbm", sample_conv_sdbm, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "wt6", sample_conv_wt6, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "xxh3", sample_conv_xxh3, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "xxh32", sample_conv_xxh32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "xxh64", sample_conv_xxh64, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "json", sample_conv_json, ARG1(1,STR), sample_conv_json_check, SMP_T_STR, SMP_T_STR },
- { "bytes", sample_conv_bytes, ARG2(1,SINT,SINT), NULL, SMP_T_BIN, SMP_T_BIN },
- { "field", sample_conv_field, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
- { "word", sample_conv_word, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
- { "regsub", sample_conv_regsub, ARG3(2,REG,STR,STR), sample_conv_regsub_check, SMP_T_STR, SMP_T_STR },
- { "sha1", sample_conv_sha1, 0, NULL, SMP_T_BIN, SMP_T_BIN },
+ { "debug", sample_conv_debug, ARG2(0,STR,STR), smp_check_debug, SMP_T_ANY, SMP_T_ANY },
+ { "b64dec", sample_conv_base642bin, 0, NULL, SMP_T_STR, SMP_T_BIN },
+ { "base64", sample_conv_bin2base64, 0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "ub64enc", sample_conv_bin2base64url,0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "ub64dec", sample_conv_base64url2bin,0, NULL, SMP_T_STR, SMP_T_BIN },
+ { "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
+ { "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
+ { "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
+ { "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT },
+ { "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_IPV4 },
+ { "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
+ { "utime", sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
+ { "crc32", sample_conv_crc32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "crc32c", sample_conv_crc32c, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "djb2", sample_conv_djb2, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "sdbm", sample_conv_sdbm, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "wt6", sample_conv_wt6, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "xxh3", sample_conv_xxh3, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "xxh32", sample_conv_xxh32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "xxh64", sample_conv_xxh64, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "json", sample_conv_json, ARG1(1,STR), sample_conv_json_check, SMP_T_STR, SMP_T_STR },
+ { "bytes", sample_conv_bytes, ARG2(1,SINT,SINT), NULL, SMP_T_BIN, SMP_T_BIN },
+ { "field", sample_conv_field, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
+ { "word", sample_conv_word, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
+ { "regsub", sample_conv_regsub, ARG3(2,REG,STR,STR), sample_conv_regsub_check, SMP_T_STR, SMP_T_STR },
+ { "sha1", sample_conv_sha1, 0, NULL, SMP_T_BIN, SMP_T_BIN },
#ifdef USE_OPENSSL
{ "sha2", sample_conv_sha2, ARG1(0, SINT), smp_check_sha2, SMP_T_BIN, SMP_T_BIN },
#ifdef EVP_CIPH_GCM_MODE
--
2.31.1
>From 83418c368abd74f8a53a6929c62c48b9201be1f7 Mon Sep 17 00:00:00 2001
From: Moemen MHEDHBI <mmhed...@haproxy.com>
Date: Thu, 1 Apr 2021 20:53:59 +0200
Subject: [PATCH 1/2] MINOR: sample: add ub64dec and ub64enc converters
ub64dec and ub64enc are the base64url equivalent of b64dec and base64
converters. base64url encoding is the "URL and Filename Safe Alphabet"
variant of base64 encoding. It is also used in in JWT (JSON Web Token)
standard.
RFC1421 mention in base64.c file is deprecated so it was replaced with
RFC4648 to which existing converters, base64/b64dec, still apply.
Example:
HAProxy:
http-request return content-type text/plain lf-string %[req.hdr(Authorization),word(2,.),ub64dec]
Client:
Token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyIjoiZm9vIiwia2V5IjoiY2hhZTZBaFhhaTZlIn0.5VsVj7mdxVvo1wP5c0dVHnr-S_khnIdFkThqvwukmdg
$ curl -H "Authorization: Bearer ${TOKEN}" http://haproxy.local
{"user":"foo","key":"chae6AhXai6e"}
---
doc/configuration.txt | 12 +++
include/haproxy/base64.h | 4 +-
reg-tests/sample_fetches/ubase64.vtc | 45 +++++++++++
src/base64.c | 116 +++++++++++++++++++--------
src/h1.c | 4 +-
src/http_fetch.c | 2 +-
src/sample.c | 44 +++++++++-
7 files changed, 188 insertions(+), 39 deletions(-)
create mode 100644 reg-tests/sample_fetches/ubase64.vtc
diff --git a/doc/configuration.txt b/doc/configuration.txt
index 7048fb63e..c7fe416e5 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -16393,6 +16393,18 @@ table_trackers(<table>)
connections there are from a given address for example. See also the
sc_trackers sample fetch keyword.
+ub64dec
+ This converter is the base64url variant of b64dec converter. base64url
+ encoding is the "URL and Filename Safe Alphabet" variant of base64 encoding.
+ It is also the encoding used in JWT (JSON Web Token) standard.
+
+ Example:
+ # Decoding a JWT payload:
+ http-request set-var(txn.token_payload) req.hdr(Authorization),word(2,.),ub64dec
+
+ub64enc
+ This converter is the base64url variant of base64 converter.
+
upper
Convert a string sample to upper case. This can only be placed after a string
sample fetch function or after a transformation keyword returning a string
diff --git a/include/haproxy/base64.h b/include/haproxy/base64.h
index 1756bc058..e63234fd0 100644
--- a/include/haproxy/base64.h
+++ b/include/haproxy/base64.h
@@ -16,8 +16,8 @@
#include <haproxy/api.h>
-int a2base64(char *in, int ilen, char *out, int olen);
-int base64dec(const char *in, size_t ilen, char *out, size_t olen);
+int a2base64(char *in, int ilen, char *out, int olen, int url);
+int base64dec(const char *in, size_t ilen, char *out, size_t olen, int url);
const char *s30tob64(int in, char *out);
int b64tos30(const char *in);
diff --git a/reg-tests/sample_fetches/ubase64.vtc b/reg-tests/sample_fetches/ubase64.vtc
new file mode 100644
index 000000000..dd32cd526
--- /dev/null
+++ b/reg-tests/sample_fetches/ubase64.vtc
@@ -0,0 +1,45 @@
+varnishtest "ub64dec sample fetche Test"
+
+#REQUIRE_VERSION=2.4
+
+feature ignore_unknown_macro
+
+haproxy h1 -conf {
+ defaults
+ mode http
+ timeout connect 1s
+ timeout client 1s
+ timeout server 1s
+
+ frontend fe
+ bind "fd@${fe}"
+ http-request return content-type text/plain hdr encode %[hdr(encode),ub64enc] hdr decode %[hdr(decode),ub64dec]
+
+} -start
+
+client c1 -connect ${h1_fe_sock} {
+ txreq -hdr "encode: f" -hdr "decode: Zg"
+ rxresp
+ expect resp.http.encode == "Zg"
+ expect resp.http.decode == "f"
+ txreq -hdr "encode: fo" -hdr "decode: Zm8"
+ rxresp
+ expect resp.http.encode == "Zm8"
+ expect resp.http.decode == "fo"
+ txreq -hdr "encode: foo" -hdr "decode: Zm9v"
+ rxresp
+ expect resp.http.encode == "Zm9v"
+ expect resp.http.decode == "foo"
+ txreq -hdr "encode: foob" -hdr "decode: Zm9vYg"
+ rxresp
+ expect resp.http.encode == "Zm9vYg"
+ expect resp.http.decode == "foob"
+ txreq -hdr "encode: fooba" -hdr "decode: Zm9vYmE"
+ rxresp
+ expect resp.http.encode == "Zm9vYmE"
+ expect resp.http.decode == "fooba"
+ txreq -hdr "encode: foobar" -hdr "decode: Zm9vYmFy"
+ rxresp
+ expect resp.http.encode == "Zm9vYmFy"
+ expect resp.http.decode == "foobar"
+} -run
diff --git a/src/base64.c b/src/base64.c
index 53e4d65b2..f2768d980 100644
--- a/src/base64.c
+++ b/src/base64.c
@@ -1,5 +1,5 @@
/*
- * ASCII <-> Base64 conversion as described in RFC1421.
+ * ASCII <-> Base64 conversion as described in RFC4648.
*
* Copyright 2006-2010 Willy Tarreau <w...@1wt.eu>
* Copyright 2009-2010 Krzysztof Piotr Oledzki <o...@ans.pl>
@@ -17,50 +17,70 @@
#include <haproxy/api.h>
#include <haproxy/base64.h>
-#define B64BASE '#' /* arbitrary chosen base value */
-#define B64CMIN '+'
-#define B64CMAX 'z'
-#define B64PADV 64 /* Base64 chosen special pad value */
+#define B64BASE '#' /* arbitrary chosen base value */
+#define B64CMIN '+'
+#define UB64CMIN '-'
+#define B64CMAX 'z'
+#define B64PADV 64 /* Base64 chosen special pad value */
const char base64tab[65]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+const char base64urltab[65]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_";
const char base64rev[]="b###cXYZ[\\]^_`a###d###$%&'()*+,-./0123456789:;<=######>?@ABCDEFGHIJKLMNOPQRSTUVW";
+const char base64urlrev[]="b##XYZ[\\]^_`a###c###$%&'()*+,-./0123456789:;<=####c#>?@ABCDEFGHIJKLMNOPQRSTUVW";
/* Encodes <ilen> bytes from <in> to <out> for at most <olen> chars (including
* the trailing zero). Returns the number of bytes written. No check is made
* for <in> or <out> to be NULL. Returns negative value if <olen> is too short
* to accept <ilen>. 4 output bytes are produced for 1 to 3 input bytes.
+ * When <url> param is set url safe variant of base64 encoding is used.
*/
-int a2base64(char *in, int ilen, char *out, int olen)
+int a2base64(char *in, int ilen, char *out, int olen, int url)
{
int convlen;
+ const char *enctab = base64tab;
convlen = ((ilen + 2) / 3) * 4;
if (convlen >= olen)
return -1;
+ if (url)
+ enctab = base64urltab;
/* we don't need to check olen anymore */
while (ilen >= 3) {
- out[0] = base64tab[(((unsigned char)in[0]) >> 2)];
- out[1] = base64tab[(((unsigned char)in[0] & 0x03) << 4) | (((unsigned char)in[1]) >> 4)];
- out[2] = base64tab[(((unsigned char)in[1] & 0x0F) << 2) | (((unsigned char)in[2]) >> 6)];
- out[3] = base64tab[(((unsigned char)in[2] & 0x3F))];
+ out[0] = enctab[(((unsigned char)in[0]) >> 2)];
+ out[1] = enctab[(((unsigned char)in[0] & 0x03) << 4) | (((unsigned char)in[1]) >> 4)];
+ out[2] = enctab[(((unsigned char)in[1] & 0x0F) << 2) | (((unsigned char)in[2]) >> 6)];
+ out[3] = enctab[(((unsigned char)in[2] & 0x3F))];
out += 4;
in += 3; ilen -= 3;
}
-
if (!ilen) {
out[0] = '\0';
- } else {
- out[0] = base64tab[((unsigned char)in[0]) >> 2];
- if (ilen == 1) {
- out[1] = base64tab[((unsigned char)in[0] & 0x03) << 4];
- out[2] = '=';
+ return convlen;
+ }
+
+ out[0] = enctab[((unsigned char)in[0]) >> 2];
+ if (ilen == 1) {
+ out[1] = enctab[((unsigned char)in[0] & 0x03) << 4];
+ if (url) {
+ out[2] = '\0';
+ convlen -= 2;
} else {
- out[1] = base64tab[(((unsigned char)in[0] & 0x03) << 4) |
- (((unsigned char)in[1]) >> 4)];
- out[2] = base64tab[((unsigned char)in[1] & 0x0F) << 2];
+ out[2] = '=';
+ out[3] = '=';
+ out[4] = '\0';
}
+ return convlen;
+ }
+
+ out[1] = enctab[(((unsigned char)in[0] & 0x03) << 4) |
+ (((unsigned char)in[1]) >> 4)];
+ out[2] = enctab[((unsigned char)in[1] & 0x0F) << 2];
+ if (url) {
+ out[3] = '\0';
+ convlen -= 1;
+ } else {
out[3] = '=';
out[4] = '\0';
}
@@ -73,29 +93,59 @@ int a2base64(char *in, int ilen, char *out, int olen)
* <in> or <out> to be NULL. Returns -1 if <in> is invalid or ilen
* has wrong size, -2 if <olen> is too short.
* 1 to 3 output bytes are produced for 4 input bytes.
+ * The reverse tab used to decode base64 is generated via /contrib/base64/base64rev-gen.c
+ * When <url> param is set url safe variant of base64 encoding is used.
*/
-int base64dec(const char *in, size_t ilen, char *out, size_t olen) {
+int base64dec(const char *in, size_t ilen, char *out, size_t olen, int url)
+{
unsigned char t[4];
+ const char *dectab = base64rev;
+ char b64min = B64CMIN, b64max = B64CMAX;
signed char b;
- int convlen = 0, i = 0, pad = 0;
-
- if (ilen % 4)
- return -1;
+ int convlen = 0, i = 0, pad = 0, padlen = 0;
+
+ switch (ilen % 4) {
+ case 0:
+ break;
+ case 2:
+ if (!url)
+ return -1;
+ padlen = 2;
+ break;
+ case 3:
+ if (!url)
+ return -1;
+ padlen = 1;
+ break;
+ default:
+ return -1;
+ }
if (olen < ((ilen / 4 * 3)
- (in[ilen-1] == '=' ? 1 : 0)
- (in[ilen-2] == '=' ? 1 : 0)))
return -2;
- while (ilen) {
+ if (url) {
+ dectab = base64urlrev;
+ b64min = UB64CMIN;
+ }
- /* if (*p < B64CMIN || *p > B64CMAX) */
- b = (signed char)*in - B64CMIN;
- if ((unsigned char)b > (B64CMAX-B64CMIN))
- return -1;
+ while (ilen + padlen) {
+
+ if (ilen) {
+ /* if (*p < B64CMIN || *p > B64CMAX) */
+ b = (signed char) * in - b64min;
+ if ((unsigned char)b > (b64max - b64min))
+ return -1;
+
+ b = dectab[b] - B64BASE - 1;
+ } else {
+ b = B64PADV;
+ padlen--;
+ }
- b = base64rev[b] - B64BASE - 1;
/* b == -1: invalid character */
if (b < 0)
@@ -131,8 +181,10 @@ int base64dec(const char *in, size_t ilen, char *out, size_t olen) {
pad = i = 0;
}
- in++;
- ilen--;
+ if (ilen) {
+ in++;
+ ilen--;
+ }
}
return convlen;
diff --git a/src/h1.c b/src/h1.c
index 3a6c1c330..148b1d2df 100644
--- a/src/h1.c
+++ b/src/h1.c
@@ -1069,7 +1069,7 @@ void h1_generate_random_ws_input_key(char key_out[25])
memcpy(key, &rand1, 8);
memcpy(&key[8], &rand2, 8);
- a2base64(key, 16, key_out, 25);
+ a2base64(key, 16, key_out, 25, 0);
}
#define H1_WS_KEY_SUFFIX_GUID "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"
@@ -1094,5 +1094,5 @@ void h1_calculate_ws_output_key(const char *key, char *result)
blk_SHA1_Final((unsigned char *)hash_out, &sha1_ctx);
/* encode in base64 the hash */
- a2base64(hash_out, 20, result, 29);
+ a2base64(hash_out, 20, result, 29, 0);
}
diff --git a/src/http_fetch.c b/src/http_fetch.c
index 6c569a75b..547da74b5 100644
--- a/src/http_fetch.c
+++ b/src/http_fetch.c
@@ -127,7 +127,7 @@ static int get_http_auth(struct sample *smp, struct htx *htx)
len = base64dec(txn->auth.method_data.area,
txn->auth.method_data.data,
- http_auth->area, global.tune.bufsize - 1);
+ http_auth->area, global.tune.bufsize - 1, 0);
if (len < 0)
return 0;
diff --git a/src/sample.c b/src/sample.c
index 835a18115..16abda467 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -1556,7 +1556,27 @@ static int sample_conv_base642bin(const struct arg *arg_p, struct sample *smp, v
trash->data = 0;
bin_len = base64dec(smp->data.u.str.area, smp->data.u.str.data,
- trash->area, trash->size);
+ trash->area, trash->size,0);
+ if (bin_len < 0)
+ return 0;
+
+ trash->data = bin_len;
+ smp->data.u.str = *trash;
+ smp->data.type = SMP_T_BIN;
+ smp->flags &= ~SMP_F_CONST;
+ return 1;
+}
+
+static int sample_conv_base64url2bin(const struct arg *arg_p, struct sample *smp, void *private)
+{
+ struct buffer *trash = get_trash_chunk();
+ int bin_len;
+
+ if (b_room(&smp->data.u.str) < 2)
+ return 0;
+ trash->data = 0;
+ bin_len = base64dec(smp->data.u.str.area, smp->data.u.str.data,
+ trash->area, trash->size,1);
if (bin_len < 0)
return 0;
@@ -1574,7 +1594,25 @@ static int sample_conv_bin2base64(const struct arg *arg_p, struct sample *smp, v
trash->data = 0;
b64_len = a2base64(smp->data.u.str.area, smp->data.u.str.data,
- trash->area, trash->size);
+ trash->area, trash->size,0);
+ if (b64_len < 0)
+ return 0;
+
+ trash->data = b64_len;
+ smp->data.u.str = *trash;
+ smp->data.type = SMP_T_STR;
+ smp->flags &= ~SMP_F_CONST;
+ return 1;
+}
+
+static int sample_conv_bin2base64url(const struct arg *arg_p, struct sample *smp, void *private)
+{
+ struct buffer *trash = get_trash_chunk();
+ int b64_len;
+
+ trash->data = 0;
+ b64_len = a2base64(smp->data.u.str.area, smp->data.u.str.data,
+ trash->area, trash->size,1);
if (b64_len < 0)
return 0;
@@ -4096,6 +4134,8 @@ static struct sample_conv_kw_list sample_conv_kws = {ILH, {
{ "debug", sample_conv_debug, ARG2(0,STR,STR), smp_check_debug, SMP_T_ANY, SMP_T_ANY },
{ "b64dec", sample_conv_base642bin,0, NULL, SMP_T_STR, SMP_T_BIN },
{ "base64", sample_conv_bin2base64,0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "ub64dec", sample_conv_base64url2bin,0, NULL, SMP_T_STR, SMP_T_BIN },
+ { "ub64enc", sample_conv_bin2base64url,0, NULL, SMP_T_BIN, SMP_T_STR },
{ "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
{ "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
--
2.31.1
>From e67d16c8edd58551d24ca740e17ee9bdd1223f63 Mon Sep 17 00:00:00 2001
From: Moemen MHEDHBI <mmhed...@haproxy.com>
Date: Fri, 2 Apr 2021 01:05:07 +0200
Subject: [PATCH 2/2] CLEANUP: align samples list in sample.c
---
src/sample.c | 54 ++++++++++++++++++++++++++--------------------------
1 file changed, 27 insertions(+), 27 deletions(-)
diff --git a/src/sample.c b/src/sample.c
index 16abda467..c17d9abc7 100644
--- a/src/sample.c
+++ b/src/sample.c
@@ -4131,33 +4131,33 @@ INITCALL1(STG_REGISTER, sample_register_fetches, &smp_kws);
/* Note: must not be declared <const> as its list will be overwritten */
static struct sample_conv_kw_list sample_conv_kws = {ILH, {
- { "debug", sample_conv_debug, ARG2(0,STR,STR), smp_check_debug, SMP_T_ANY, SMP_T_ANY },
- { "b64dec", sample_conv_base642bin,0, NULL, SMP_T_STR, SMP_T_BIN },
- { "base64", sample_conv_bin2base64,0, NULL, SMP_T_BIN, SMP_T_STR },
- { "ub64dec", sample_conv_base64url2bin,0, NULL, SMP_T_STR, SMP_T_BIN },
- { "ub64enc", sample_conv_bin2base64url,0, NULL, SMP_T_BIN, SMP_T_STR },
- { "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
- { "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
- { "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
- { "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
- { "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT },
- { "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_IPV4 },
- { "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
- { "utime", sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
- { "crc32", sample_conv_crc32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "crc32c", sample_conv_crc32c, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "djb2", sample_conv_djb2, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "sdbm", sample_conv_sdbm, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "wt6", sample_conv_wt6, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "xxh3", sample_conv_xxh3, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "xxh32", sample_conv_xxh32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "xxh64", sample_conv_xxh64, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
- { "json", sample_conv_json, ARG1(1,STR), sample_conv_json_check, SMP_T_STR, SMP_T_STR },
- { "bytes", sample_conv_bytes, ARG2(1,SINT,SINT), NULL, SMP_T_BIN, SMP_T_BIN },
- { "field", sample_conv_field, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
- { "word", sample_conv_word, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
- { "regsub", sample_conv_regsub, ARG3(2,REG,STR,STR), sample_conv_regsub_check, SMP_T_STR, SMP_T_STR },
- { "sha1", sample_conv_sha1, 0, NULL, SMP_T_BIN, SMP_T_BIN },
+ { "debug", sample_conv_debug, ARG2(0,STR,STR), smp_check_debug, SMP_T_ANY, SMP_T_ANY },
+ { "b64dec", sample_conv_base642bin, 0, NULL, SMP_T_STR, SMP_T_BIN },
+ { "base64", sample_conv_bin2base64, 0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "ub64enc", sample_conv_bin2base64url,0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "ub64dec", sample_conv_base64url2bin,0, NULL, SMP_T_STR, SMP_T_BIN },
+ { "upper", sample_conv_str2upper, 0, NULL, SMP_T_STR, SMP_T_STR },
+ { "lower", sample_conv_str2lower, 0, NULL, SMP_T_STR, SMP_T_STR },
+ { "length", sample_conv_length, 0, NULL, SMP_T_STR, SMP_T_SINT },
+ { "hex", sample_conv_bin2hex, 0, NULL, SMP_T_BIN, SMP_T_STR },
+ { "hex2i", sample_conv_hex2int, 0, NULL, SMP_T_STR, SMP_T_SINT },
+ { "ipmask", sample_conv_ipmask, ARG2(1,MSK4,MSK6), NULL, SMP_T_ADDR, SMP_T_IPV4 },
+ { "ltime", sample_conv_ltime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
+ { "utime", sample_conv_utime, ARG2(1,STR,SINT), NULL, SMP_T_SINT, SMP_T_STR },
+ { "crc32", sample_conv_crc32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "crc32c", sample_conv_crc32c, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "djb2", sample_conv_djb2, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "sdbm", sample_conv_sdbm, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "wt6", sample_conv_wt6, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "xxh3", sample_conv_xxh3, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "xxh32", sample_conv_xxh32, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "xxh64", sample_conv_xxh64, ARG1(0,SINT), NULL, SMP_T_BIN, SMP_T_SINT },
+ { "json", sample_conv_json, ARG1(1,STR), sample_conv_json_check, SMP_T_STR, SMP_T_STR },
+ { "bytes", sample_conv_bytes, ARG2(1,SINT,SINT), NULL, SMP_T_BIN, SMP_T_BIN },
+ { "field", sample_conv_field, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
+ { "word", sample_conv_word, ARG3(2,SINT,STR,SINT), sample_conv_field_check, SMP_T_STR, SMP_T_STR },
+ { "regsub", sample_conv_regsub, ARG3(2,REG,STR,STR), sample_conv_regsub_check, SMP_T_STR, SMP_T_STR },
+ { "sha1", sample_conv_sha1, 0, NULL, SMP_T_BIN, SMP_T_BIN },
#ifdef USE_OPENSSL
{ "sha2", sample_conv_sha2, ARG1(0, SINT), smp_check_sha2, SMP_T_BIN, SMP_T_BIN },
#ifdef EVP_CIPH_GCM_MODE
--
2.31.1
