Hi Krzysztof, On Mon, Mar 09, 2009 at 01:13:31PM +0100, Krzysztof Oledzki wrote: > Hi Willy, > > First, please excuse that it took me nearly one moth to replay to your > letter, shame on me. :(
no problem, I know we're all facing the same issues trying to find time :-) > >In fact, I think that having use_backend evaluated last makes sense > >since it's really how it's supposed to work. But I agree that for the > >poor guy writing the rules, it would be easier to be able to put it > >before other rules. In fact, there's a solution consisting in using > >"allow" to escape the rules, but it requires that the rule is > >duplicated for the use_backend one, which is not always very convenient. > > So, after so long time of thinking (1 month, right? ;) ) I believe we > should keep it as-is and prevent duplication by simply writing a "best > practices" chapter in the docs and suggesting to use a dedicated backend > to handle redirects: > > backend pax_redirects > redirect prefix https://pay.xxx.bg if pay_xxx > (...) Indeed, it may be one way of solving difficult ordering for complex setups, I like the idea. Maybe we should see the backend more as the place where most of the processing should be performed once it has been selected by the frontend. This is already more or less the case after all. > backend XX > use_backend www_php4 if payment > use_backend pax_redirects if pay_xxx > default_backend www_php4 > > We may also add a warinig or even disallow to use use_backend and redirect > in the same proxy. I believe it is the best solution - we already have > everything what is needed, let's use it. I want to keep the ability to redirect from the frontend, as I already have several configs making use of it. It's useful to catch unexpected conditions, such as : frontend XXX acl local_site_down nbsrv(bk) lt 3 redirect prefix http://backup-site.domain.com if local_site_down use_backend bk if ... See ? However, I think that the root cause of the problem is having redirect *after* use_backend. This is indeed misleading and should cause a warning to be emitted. I'll try to enumerate all cases of mis-placed conditions which can lead to unexpected behaviours, and report warnings in such cases. The example above should be fine, but the one below should send a warning : frontend XXX acl local_site_down nbsrv(bk) lt 3 use_backend bk if ... redirect prefix http://backup-site.domain.com if local_site_down > >Don't you think we should create a new "set-backend" keyword to merge > >it with the whole list, and let "use_backend" slowly die (for instance, > >we mark it deprecated in version 1.4 with a big warning) ? > >This means we would have : > > block > > allow|deny|redirect|tarpit|set-backend > > use_backend > > I'm not sure. I like the idea of two step request processing: > - first decide if we need to "allow|deny|tarpit" a request > - then decide which backend to use (frontend) or which server/redirect > (backend) Yes, after one month of thinking too :-) I think it is important to separate the authorisation (allow/deny/...) from the switching rule. And in fact, that's what was on the diagram I sent a month ago. It's in /download/1.3/doc/acl.pdf on the site, in case you don't have it in mind. > We may add set-backend directive but I think use_backend will be still > useful, so keeping both could be hard to maintain in the long term. Agreed. I even remember now that I wanted use_backend in the frontend and use_server in the backend, both processed after allow/deny/... > >Another idea would consist in splitting access rules from traffic > >management rules. I mean, "allow", "deny", "tarpit" grant or deny > >access. Even the tarpit could be considered as an extended deny. > >Then we have "use_backend", "redirect", and maybe later things > >about QoS, logging, etc... which would make sense in a separate > >list. > > Yes. This is definitely the way I think we would like to go currently! I think it too. > >Since we can't ask the user to remember which keyword works in which > >group, the syntax should make it possible to explicitly state where > >the processing ought to happen. That's the principle of the "http-req in" > >etc... > > This is good, however I'm not sure about this "http-req in". First: why we > need to explicitly state that it is a http processing? Because there would be multiple levels. First, we can already match at the TCP level even in HTTP mode. Second, once there will be multiple layers (eg: HTTP on top of SSL on top of TCP), we will want to tell where the rule applies. However, I absolutely want that these boring keywords are not needed for the normal case. Right now we know that most of the rules will be HTTP, so it makes sense not to indicate anything by default for HTTP. Ditto if we implement SMTP, it should not be needed to explitictly state we want to process SMTP. > The same syntax > should be used for different backends (tcp, http, smtp) if that is > possible. The default syntax should apply the expected behaviour depending on the mode. The explicit syntax will be useful to leverage any ambiguity. > The only thing that matters are acls. Not exactly. For instance, I'm already used to block some source IPs at the TCP level, immediately upon accept, while doing other things at HTTP level, once the full request has been received. > So I would rather use "req" here. > Then: why "in"? Do you expect we may ever need something like "req out" > processing? Yes, it was on the diagram too. It would more likely be for final rewriting, and only *sometimes* blocking. But it's something particularly uncommon, to block on output. However, one may want to put filtering rules in a frontend, on the response output, in order to ensure that there is no leaking of sensible information on a specific frontend. But once again, "in" should be the default here. > >So we could have two explicit groups for "http-req" : one just for > >access (allow/deny/tarpit), and another one for traffic control which > >would be the default one, with all traffic management rules. So we > >could have something like this : > > > > http-req in allow if acl(hdr(host) -i www) > > http-req in deny if ANY > > > > http-req redirect /index.html if acl(url /) > > http-req redirect /index.html if acl(url /) > > http-req use_backend static if acl(url_beg /static) > > So this is more or less what we have now (or we will have implemented by > my ACL patch), I don't find big value in requiring "http-req" prefix. I don't want to require it, it's just a way of indicating where we want the action to be performed when there are multiple possibilities. > >The more I'm thinking about it, the more I think that the problem > >only comes from those rules which do not terminate processing : "allow" > >and "use_backend". Users expect to combine them with other rules. There's > >even already an exception for allow/deny rules in the processing so that > >the first action is granted in case of an allow. > > So, my current proposal is to: > > - add "req (allow|deny|tarpit) [(if|unless) (...)]" yes > - keep "block" but warn if it is used I'm just thinking that as it has only been used for blacklisting right now, we should just document that it's performed before allow/deny/... and warn if it appears after one of those rules. It would be useful later when we can include files, because some people might want to feed blacklists without touching their other rules. > - keep use_backend and redirect as is yes > - warn when use_backend and redirect is used ith the same proxy OK with the order detection. If you look at the diagram, you'll see rectangular boxes in which ACLs are processed. I think I have lost the drafts I had with the list of options there, but from memory (and logical sense) what we need in these blocks is to always follow the same order : 1) block 2) allow/deny/tarpit/... 3) replace/delete (reqadd/reqdel) 4) add (reqadd) 5) redirect Of course that might slightly vary depending on the protocol and the direction, but I think we should be able to overall keep the same ordering. A big advantage of following this sequence is that it should make it possible to mix TCP frontends with HTTP backends, which is particularly handy for instance, when opening multiple protocols on the same port (HTTP+HTTPS). I'll check how we can report warnings for mis-ordered redirects right now. If I can release 1.3.16 with that warning in, it'll save a lot of headaches to some people I think :-) Cheers, Willy
