On Wed, 11 Feb 2009, Willy Tarreau wrote:
Hi Krzysztof,
Hi Willy,
First, please excuse that it took me nearly one moth to replay to your
letter, shame on me. :(
On Wed, Feb 11, 2009 at 05:58:42PM +0100, Krzysztof Oledzki wrote:
As you are probably aware, recently there was a mail quoted below, asking
about the redirect feature. It encouraged me to think a little more about
it, so: shouldn't we rather put the feature into use_backend chain instead
of "req allow/deny/block"? This would simply allow to do something like:
--- cut here ---
use_backend www_php4 if payment
redirect prefix https://pay.xxx.bg if pay_xxx
default_backend www_php4
--- cut here ---
instead of:
--- cut here ---
use_backend www_php4 if payment
redirect prefix https://pay.xxx.bg if pay_xxx !payment
default_backend www_php4
--- cut here ---
It would also allow to create more complicated rules, without duplicating
whole "use_backend xxx if abc" section, which is sometimes *very*
complicated, into "req allow if abc".
What is your opinion?
To be fair, this is how I believed it worked. But I now remember there
were is a specific "redirect rules" list for them, so I believed wrong.
I've just looked at the diagram I have posted yesterday, and use_backend
is separated there. Now that I'm thinking about it, I believe the reason
I wanted use_backend in a different list was because it is not really an
action. I mean, "allow", "deny", "redirect", "tarpit", ... all terminate
processing or rule evaluation. Having "use_backend" stop evaluation is
problematic, as it's not an action, it's a dynamic configuration.
In fact, it's problematic for existing setups mostly, because new setups
could be written by carefully interleaving use_backend statements in the
middle of the allow/redirect/deny rules.
Yes, your concern is perfectly right. Changing such behavior would be
wrong and most likely useless.
And I agree that for the long term, it's better to remember that rule
ordering is what matters than remembering which one applies before or
after which one.
I doubt it will be ever possible to ensure 100% rule ordering, especially
it is quite inconvenience for automatically generated configs allowing
to include manually created rulesets.
In fact, I think that having use_backend evaluated last makes sense
since it's really how it's supposed to work. But I agree that for the
poor guy writing the rules, it would be easier to be able to put it
before other rules. In fact, there's a solution consisting in using
"allow" to escape the rules, but it requires that the rule is
duplicated for the use_backend one, which is not always very convenient.
So, after so long time of thinking (1 month, right? ;) ) I believe we
should keep it as-is and prevent duplication by simply writing a "best
practices" chapter in the docs and suggesting to use a dedicated backend
to handle redirects:
backend pax_redirects
redirect prefix https://pay.xxx.bg if pay_xxx
(...)
backend XX
use_backend www_php4 if payment
use_backend pax_redirects if pay_xxx
default_backend www_php4
We may also add a warinig or even disallow to use use_backend and redirect
in the same proxy. I believe it is the best solution - we already have
everything what is needed, let's use it.
Hmmm now I remember why redirect was in another list. I'm pretty sure
the reason was that we wanted to be able to redirect after an allow
(which is not necessarily a strong requirement).
There are other aspects to consider :
- eventually, tcp will support use_backend/allow/deny/tarpit
- outgoing processing will obviously ever support use_backend
- the backend's incoming rules will support use_server (to force
a server) but not use_backend.
All these elements may easily be solved by a simple rule list and
strict action enforcing to ensure we never have the wrong action
at the wrong place.
I'm still afraid of breaking existing setups. Same problem as for the
"block" keyword after all.
Don't you think we should create a new "set-backend" keyword to merge
it with the whole list, and let "use_backend" slowly die (for instance,
we mark it deprecated in version 1.4 with a big warning) ?
This means we would have :
block
allow|deny|redirect|tarpit|set-backend
use_backend
I'm not sure. I like the idea of two step request processing:
- first decide if we need to "allow|deny|tarpit" a request
- then decide which backend to use (frontend) or which server/redirect
(backend)
We may add set-backend directive but I think use_backend will be still
useful, so keeping both could be hard to maintain in the long term.
Another idea would consist in splitting access rules from traffic
management rules. I mean, "allow", "deny", "tarpit" grant or deny
access. Even the tarpit could be considered as an extended deny.
Then we have "use_backend", "redirect", and maybe later things
about QoS, logging, etc... which would make sense in a separate
list.
Yes. This is definitely the way I think we would like to go currently!
Since we can't ask the user to remember which keyword works in which
group, the syntax should make it possible to explicitly state where
the processing ought to happen. That's the principle of the "http-req in"
etc...
This is good, however I'm not sure about this "http-req in". First: why we
need to explicitly state that it is a http processing? The same syntax
should be used for different backends (tcp, http, smtp) if that is
possible. The only thing that matters are acls. So I would rather use
"req" here. Then: why "in"? Do you expect we may ever need something
like "req out" processing?
So we could have two explicit groups for "http-req" : one just for
access (allow/deny/tarpit), and another one for traffic control which
would be the default one, with all traffic management rules. So we
could have something like this :
http-req in allow if acl(hdr(host) -i www)
http-req in deny if ANY
http-req redirect /index.html if acl(url /)
http-req redirect /index.html if acl(url /)
http-req use_backend static if acl(url_beg /static)
So this is more or less what we have now (or we will have implemented by
my ACL patch), I don't find big value in requiring "http-req" prefix.
The more I'm thinking about it, the more I think that the problem
only comes from those rules which do not terminate processing : "allow"
and "use_backend". Users expect to combine them with other rules. There's
even already an exception for allow/deny rules in the processing so that
the first action is granted in case of an allow.
So, my current proposal is to:
- add "req (allow|deny|tarpit) [(if|unless) (...)]"
- keep "block" but warn if it is used
- keep use_backend and redirect as is
- warn when use_backend and redirect is used ith the same proxy
The problem will only get worse once we have QoS settings, header
addition or replacements, etc..., as those will not be terminal rules
anymore. Note that we could have an optional keyword associated to
all of those to indicate if the eval is to be the last one or not
when it matches, but I'm not sure it will make the config any simpler
because some people will still want to run a few rule sets.
I'd want to avoid the principle of the "goto" + rule number as used in
the Alteon for instance, because while it solves the issue, it makes
administration very hard.
I agree, goto is rather a poor design option, we should avoid it.
A long time ago, I wanted to support rule sets (and IIRC, the keyword
"ruleset" is accepted in the config). Those ones were dedicated to
rules and switching. The idea was to use them approximately the same
way as iptables with jumps between rulesets. It could help a lot, but
seems very complicated for people who just want to do a few things.
The principle of separating processing in small groups (eg: http-req,
etc...) approximately mimmics this behaviour without adding too much
hassle. We could add also other groups after all.
Maybe we should then start by considering that only "allow" and
"use_backend" sometimes need to take immediate action and exit
rules evaluation, and sometimes go on to the end. Then we may
write some variants for them, eg "allow_now, use_backend_now" or
"allow!, use_backend!" to indicate that we want that rule to act
immediately.
I believe we still need some thinking and to collect users' opinions
on those variants. It would be annoying to make something very poor
and having to support it for a long time.
Exactly!
Best regards,
Krzysztof Olędzki
