Hi,
I did this patch for dev19 some time ago but I am still not sure whether
it is the best way to do it or not, and did not have the time to discuss
it since. As the latest changes broke it and forced me to rebase it, and
it's very useful for us, I'd like to propose it for inclusion before the
final release if you think it's OK, or to discuss how it should be done.
Main purpose wanted to achieve is it be able to use many backends
without the need to declare each routing process from frontend to
backend and instead use generic and dynamic switching when a sane
parameter can be used from user request using the logformat logic. For
example when we have a backend farm dedicated to each 'Host: ' http-header,
it's pain in the ass to have to declare the backend and the relevant
use_backend.
With the proposed solution, you first need to declare a dynamic
use_backend as the following :
use_backend bk_cust_%[hdr(Host)] if { hdr(Host) -m found }
And then to declare the needed backend. For every new vhost hosted will only
need to add the backend section to the configuration.
More detailed usage and implementation in patch itself.
It was rebased on commit 0e9b1b4d1f0efc5e46a10371d9be21e97581faab.
A current limitation of that patch can be that if a dynamic use_backend
is evaluated and the named backend is not found, the default_backend is
then used. This is not a need we have, but some may complain about it.
Future enhancement could be to use the same logic for use-server and
many other part such as reqadd..
Bertrand
--
Bertrand Jacquin, EXOSEC (http://www.exosec.fr/)
ZAC des Metz - 3 Rue du petit robinson - 78350 JOUY EN JOSAS
Tel: +33 1 30 67 60 65 - Fax: +33 1 75 43 40 70
mailto:[email protected]
>From bc306e460752244e73ca7c04dcf3df7506f1c013 Mon Sep 17 00:00:00 2001
From: Bertrand Jacquin <[email protected]>
Date: Tue, 19 Nov 2013 11:43:06 +0100
Subject: [PATCH] MEDIUM: proxy: support use_backend with dynamic names
We have a use case where we look up a customer ID in an HTTP header
and direct it to the corresponding server. This can easily be done
using ACLs and use_backend rules, but the configuration becomes
painful to maintain when the number of customers grows to a few
tens or even a several hundreds.
We realized it would be nice if we could make the use_backend
resolve its name at run time instead of config parsing time, and
use a similar expression as http-request add-header to decide on
the proper backend to use. This permits the use of prefixes or
even complex names in backend expressions. If no name matches,
then the default backend is used. Doing so allowed us to get rid
of all the use_backend rules.
Since there are some config checks on the use_backend rules to see
if the referenced backend exists, we want to keep them to detect
config errors in normal config. So this patch does not modify the
default behaviour and proceeds this way :
- if the backend name in the use_backend directive parses as a log
format rule, it's used as-is and is resolved at run time ;
- otherwise it's a static name which must be valid at config time.
There was the possibility of doing this with the use-server directive
instead of use_backend, but it seems like use_backend is more suited
to this task, as it can be used for other purposes. For example, it
becomes easy to serve a customer-specific proxy.pac file based on the
customer ID by abusing the errorfile primitive :
use_backend bk_cust_%[hdr(X-Cust-Id)] if { hdr(X-Cust-Id) -m found }
default_backend bk_err_404
backend bk_cust_1
errorfile 200 /etc/haproxy/static/proxy.pac.cust1
Signed-off-by: Bertrand Jacquin <[email protected]>
---
doc/configuration.txt | 24 +++++++++++++++++++++---
include/types/proxy.h | 2 ++
src/cfgparse.c | 29 ++++++++++++++++++++++++++++-
src/session.c | 19 ++++++++++++++++++-
4 files changed, 69 insertions(+), 5 deletions(-)
diff --git a/doc/configuration.txt b/doc/configuration.txt
index ba8057f..661f537 100644
--- a/doc/configuration.txt
+++ b/doc/configuration.txt
@@ -2463,7 +2463,9 @@ fullconn <conns>
Since it's hard to get this value right, haproxy automatically sets it to
10% of the sum of the maxconns of all frontends that may branch to this
- backend. That way it's safe to leave it unset.
+ backend (based on "use_backend" and "default_backend" rules). That way it's
+ safe to leave it unset. However, "use_backend" involving dynamic names are
+ not counted since there is no way to know if they could match or not.
Example :
# The servers will accept between 100 and 1000 concurrent connections each
@@ -7191,7 +7193,8 @@ use_backend <backend> unless <condition>
May be used in sections : defaults | frontend | listen | backend
no | yes | yes | no
Arguments :
- <backend> is the name of a valid backend or "listen" section.
+ <backend> is the name of a valid backend or "listen" section, or a
+ "log-format" string resolving to a backend name.
<condition> is a condition composed of ACLs, as described in section 7.
@@ -7220,7 +7223,22 @@ use_backend <backend> unless <condition>
a complete HTTP request to get in. This feature is useful when a frontend
must decode several protocols on a unique port, one of them being HTTP.
- See also: "default_backend", "tcp-request", and section 7 about ACLs.
+ When <backend> is a simple name, it is resolved at configuration time, and an
+ error is reported if the specified backend does not exist. If <backend> is
+ a log-format string instead, no check may be done at configuration time, so
+ the backend name is resolved dynamically at run time. If the resulting
+ backend name does not correspond to any valid backend, no other rule is
+ evaluated, and the default_backend directive is applied instead. Note that
+ when using dynamic backend names, it is highly recommended to use a prefix
+ that no other backend uses in order to ensure that an unauthorized backend
+ cannot be forced from the request.
+
+ It is worth mentionning that "use_backend" rules with an explicit name are
+ used to detect the association between frontends and backends to compute the
+ backend's "fullconn" setting. This cannot be done for dynamic names.
+
+ See also: "default_backend", "tcp-request", "fullconn", "log-format", and
+ section 7 about ACLs.
use-server <server> if <condition>
diff --git a/include/types/proxy.h b/include/types/proxy.h
index e6bc755..085129c 100644
--- a/include/types/proxy.h
+++ b/include/types/proxy.h
@@ -364,9 +364,11 @@ struct proxy {
struct switching_rule {
struct list list; /* list linked to from the proxy */
struct acl_cond *cond; /* acl condition to meet */
+ int dynamic; /* this is a dynamic rule using the logformat expression */
union {
struct proxy *backend; /* target backend */
char *name; /* target backend name during config parsing */
+ struct list expr; /* logformat expression to use for dynamic rules */
} be;
};
diff --git a/src/cfgparse.c b/src/cfgparse.c
index 41c1949..4ec5aeb 100644
--- a/src/cfgparse.c
+++ b/src/cfgparse.c
@@ -6259,6 +6259,33 @@ int check_config_validity()
/* find the target proxy for 'use_backend' rules */
list_for_each_entry(rule, &curproxy->switching_rules, list) {
struct proxy *target;
+ struct logformat_node *node;
+ char *pxname;
+
+ /* Try to parse the string as a log format expression. If the result
+ * of the parsing is only one entry containing a simple string, then
+ * it's a standard string corresponding to a static rule, thus the
+ * parsing is cancelled and be.name is restored to be resolved.
+ */
+ pxname = rule->be.name;
+ LIST_INIT(&rule->be.expr);
+ parse_logformat_string(pxname, curproxy, &rule->be.expr, 0, SMP_VAL_FE_HRQ_HDR,
+ curproxy->conf.args.file, curproxy->conf.args.line);
+ node = LIST_NEXT(&rule->be.expr, struct logformat_node *, list);
+
+ if (!LIST_ISEMPTY(&rule->be.expr)) {
+ if (node->type != LOG_FMT_TEXT || node->list.n != &rule->be.expr) {
+ rule->dynamic = 1;
+ free(pxname);
+ continue;
+ }
+ /* simple string: free the expression and fall back to static rule */
+ free(node->arg);
+ free(node);
+ }
+
+ rule->dynamic = 0;
+ rule->be.name = pxname;
target = findproxy_mode(rule->be.name, curproxy->mode, PR_CAP_BE);
@@ -7210,7 +7237,7 @@ out_uri_auth_compat:
/* check if a "use_backend" rule matches */
if (!found) {
list_for_each_entry(rule, &fe->switching_rules, list) {
- if (rule->be.backend == curproxy) {
+ if (!rule->dynamic && rule->be.backend == curproxy) {
found = 1;
break;
}
diff --git a/src/session.c b/src/session.c
index ed55ca4..e0d2cd1 100644
--- a/src/session.c
+++ b/src/session.c
@@ -1234,7 +1234,24 @@ static int process_switching_rules(struct session *s, struct channel *req, int a
ret = !ret;
if (ret) {
- if (!session_set_backend(s, rule->be.backend))
+ /* If the backend name is dynamic, try to resolve the name.
+ * If we can't resolve the name, or if any error occurs, break
+ * the loop and fallback to the default backend.
+ */
+ struct proxy *backend;
+
+ if (rule->dynamic) {
+ struct chunk *tmp = get_trash_chunk();
+ if (!build_logline(s, tmp->str, tmp->size, &rule->be.expr))
+ break;
+ backend = findproxy(tmp->str, PR_CAP_BE);
+ if (!backend)
+ break;
+ }
+ else
+ backend = rule->be.backend;
+
+ if (!session_set_backend(s, backend))
goto sw_failed;
break;
}
