Re: Copying a Header before Modifying it
Baptiste wrote: On Thu, Mar 29, 2012 at 11:42 PM, William Lewis m...@wlewis.co.uk wrote: Hi Cyril, Cyril Bont wrote: Hi William, Le 29/03/2012 14:30, William Lewis a crit : Hi, So I use Haproxy to rewrite some URL requests infront of my java webservers, but I also want my java webservers to be able to issue redirects relative to the url that hit haproxy. Specifically I want the developers that have access to application platform but not the haproxy to be able to enforce a resource is only accessible over https without me having to write a rule in the haproxy config. In this case they just need to be able to get the original request and send back a 403 redirect with https:// on the front, of course they don't see the original url so this is a problem. I tried solving it with this rule reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 run before any of the rewrite rules e.g. reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 This results in a request to the webserver which looks like GET /tomcatcontext/ HTTP/1.1 X-Original-Request: / Host: example.com Connection: keep-alive ... This all works great until you then try and do some acl matching in the haproxy, because an acl like acl example-com hdr_end(host) -i example.com will no longer match. It should (tested quicky here), can you provide us the version of haproxy you're using ? I remember there was a bug in old 1.4 versions concerning headers manipulation. I was using 1.4.19 but have just updated to 1.4.20 and still having the same problem, complete example config below global daemon quiet maxconn 1024 pidfile /home/haproxyblue/haproxy.pid uid 20003 gid 20003 chroot /home/haproxyblue log 127.0.0.1 local0 log 127.0.0.1 local1 notice defaults log global option httplog balance roundrobin mode http retries 3 option redispatch timeout connect 30 timeout client 30 timeout server 30 frontend http-in bind *:80 reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 acl test hdr_end(host) -i example.com reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 if test use_backend echo if test backend echo mode http option http-server-close option forwardfor server echo1 127.0.0.1: Hi, You hould enable http-server-close on the frontend side as well. Or better, put it in the defaults. cheers I've added http-server-close to the defaults and the acl still doesn't match when I write the X-Original-Request header.
Re: Copying a Header before Modifying it
Hi all, Le 30/03/2012 12:27, William Lewis a écrit : Baptiste wrote: (...) You hould enable http-server-close on the frontend side as well. Or better, put it in the defaults. cheers I've added http-server-close to the defaults and the acl still doesn't match when I write the X-Original-Request header. Actually, there's a comment in the code which talks about this issue : /* FIXME: if the user adds a newline in the replacement, the * index will not be recalculated for now, and the new line * will not be counted as a new header. */ As a side effect, headers indexes do not point to the right position. This means that acls do not match anymore, which is what you observe. It appears several times in the code, the one that concerns this issue in the function apply_filter_to_req_line() in src/proto_http.c -- Cyril Bonté
Re: Copying a Header before Modifying it
Hi again,
Le 30/03/2012 13:24, Cyril Bonté a écrit :
Hi all,
Le 30/03/2012 12:27, William Lewis a écrit :
Baptiste wrote:
(...)
You hould enable http-server-close on the frontend side as well.
Or better, put it in the defaults.
cheers
I've added http-server-close to the defaults and the acl still doesn't
match when I write the X-Original-Request header.
Actually, there's a comment in the code which talks about this issue :
/* FIXME: if the user adds a newline in the replacement, the
* index will not be recalculated for now, and the new line
* will not be counted as a new header.
*/
As a side effect, headers indexes do not point to the right position.
This means that acls do not match anymore, which is what you observe.
It appears several times in the code, the one that concerns this issue
in the function apply_filter_to_req_line() in src/proto_http.c
Please find in attachment a quick and *dirty* fix to realign the headers
indexes for your case (based on haproxy 1.4.20).
Maybe this can help you.
--
Cyril Bonté
--- ./haproxy-1.4.20/src/proto_http.c 2012-03-10 21:06:21.0 +0100
+++ ./haproxy-1.4.20-newline/src/proto_http.c 2012-03-30 15:14:31.127457646 +0200
@@ -5849,12 +5849,20 @@
http_msg_move_end(txn-req, delta);
cur_end += delta;
- cur_end = (char *)http_parse_reqline(txn-req, req-data,
- HTTP_MSG_RQMETH,
- cur_ptr, cur_end + 1,
- NULL, NULL);
- if (unlikely(!cur_end))
-return -1;
+ /* Quick and dirty hack to realign headers indexes if the user adds a newline
+ * in the replacement.
+ */
+ {
+char *cur_end2 = (char *)http_parse_reqline(txn-req, req-data,
+ HTTP_MSG_RQMETH,
+ cur_ptr, cur_end + 1,
+ NULL, NULL);
+if (unlikely(!cur_end2))
+ return -1;
+if (cur_end - cur_end2 0) {
+ txn-req.sl.rq.l += cur_end - cur_end2;
+}
+ }
/* we have a full request and we know that we have either a CR
* or an LF at ptr.
Copying a Header before Modifying it
Hi, So I use Haproxy to rewrite some URL requests infront of my java webservers, but I also want my java webservers to be able to issue redirects relative to the url that hit haproxy. Specifically I want the developers that have access to application platform but not the haproxy to be able to enforce a resource is only accessible over https without me having to write a rule in the haproxy config. In this case they just need to be able to get the original request and send back a 403 redirect with https:// on the front, of course they don't see the original url so this is a problem. I tried solving it with this rule reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 run before any of the rewrite rules e.g. reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 This results in a request to the webserver which looks like GET /tomcatcontext/ HTTP/1.1 X-Original-Request: / Host: example.com Connection: keep-alive ... This all works great until you then try and do some acl matching in the haproxy, because an acl like acl example-com hdr_end(host) -i example.com will no longer match. Looks like a bug to me but I'd be interested in hearing any other ways of getting the original request through to the backend or otherwise allowing the backend to signal the haproxy that request needs to be redirected onto https. Thanks Will Lewis
Re: Copying a Header before Modifying it
W dniu 29 marca 2012 14:48 użytkownik Łukasz Jagiełło jagiello.luk...@gmail.com napisał: From what I notice, as long as keepalive working only requests going to backend from first request. After keepalive ends, again backend is choice by ACL. Should be: From what I notice, as long as keepalive working all requests going to backend from first request. After keepalive ends, again backend is choice by ACL. -- Łukasz Jagiełło lukaszatjagiellodotorg
Re: Copying a Header before Modifying it
Hi William, Le 29/03/2012 14:30, William Lewis a écrit : Hi, So I use Haproxy to rewrite some URL requests infront of my java webservers, but I also want my java webservers to be able to issue redirects relative to the url that hit haproxy. Specifically I want the developers that have access to application platform but not the haproxy to be able to enforce a resource is only accessible over https without me having to write a rule in the haproxy config. In this case they just need to be able to get the original request and send back a 403 redirect with https:// on the front, of course they don't see the original url so this is a problem. I tried solving it with this rule reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 run before any of the rewrite rules e.g. reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 This results in a request to the webserver which looks like GET /tomcatcontext/ HTTP/1.1 X-Original-Request: / Host: example.com Connection: keep-alive ... This all works great until you then try and do some acl matching in the haproxy, because an acl like acl example-com hdr_end(host) -i example.com will no longer match. It should (tested quicky here), can you provide us the version of haproxy you're using ? I remember there was a bug in old 1.4 versions concerning headers manipulation. -- Cyril Bonté
Re: Copying a Header before Modifying it
Hi, Le 29/03/2012 14:48, Łukasz Jagiełło a écrit : (...) Problem happen when timeout http-keep-alive 1s and without option httpclose. When I turn option httpclose problem gone, and start working normal. From what I notice, as long as keepalive working only requests going to backend from first request. After keepalive ends, again backend is choice by ACL. It looks like you forgot to use option http-server-close (timeout http-keep-alive is not sufficient). Without one of the options httpclose or http-server-close, haproxy works in what is called tunnel mode, which means only the first request of a connection is analyzed, everthing else is considered as data to be directly streamed to the server. -- Cyril Bonté
Re: Copying a Header before Modifying it
Hi Cyril, Cyril Bont wrote: Hi William, Le 29/03/2012 14:30, William Lewis a crit : Hi, So I use Haproxy to rewrite some URL requests infront of my java webservers, but I also want my java webservers to be able to issue redirects relative to the url that hit haproxy. Specifically I want the developers that have access to application platform but not the haproxy to be able to enforce a resource is only accessible over https without me having to write a rule in the haproxy config. In this case they just need to be able to get the original request and send back a 403 redirect with https:// on the front, of course they don't see the original url so this is a problem. I tried solving it with this rule reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 run before any of the rewrite rules e.g. reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 This results in a request to the webserver which looks like GET /tomcatcontext/ HTTP/1.1 X-Original-Request: / Host: example.com Connection: keep-alive ... This all works great until you then try and do some acl matching in the haproxy, because an acl like acl example-com hdr_end(host) -i example.com will no longer match. It should (tested quicky here), can you provide us the version of haproxy you're using ? I remember there was a bug in old 1.4 versions concerning headers manipulation. I was using 1.4.19 but have just updated to 1.4.20 and still having the same problem, complete example config below global daemon quiet maxconn 1024 pidfile /home/haproxyblue/haproxy.pid uid 20003 gid 20003 chroot /home/haproxyblue log 127.0.0.1 local0 log 127.0.0.1 local1 notice defaults log global option httplog balance roundrobin mode http retries 3 option redispatch timeout connect 30 timeout client 30 timeout server 30 frontend http-in bind *:80 reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 acl test hdr_end(host) -i example.com reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 if test use_backend echo if test backend echo mode http option http-server-close option forwardfor server echo1 127.0.0.1:
Re: Copying a Header before Modifying it
On Thu, Mar 29, 2012 at 11:42 PM, William Lewis m...@wlewis.co.uk wrote: Hi Cyril, Cyril Bonté wrote: Hi William, Le 29/03/2012 14:30, William Lewis a écrit : Hi, So I use Haproxy to rewrite some URL requests infront of my java webservers, but I also want my java webservers to be able to issue redirects relative to the url that hit haproxy. Specifically I want the developers that have access to application platform but not the haproxy to be able to enforce a resource is only accessible over https without me having to write a rule in the haproxy config. In this case they just need to be able to get the original request and send back a 403 redirect with https:// on the front, of course they don't see the original url so this is a problem. I tried solving it with this rule reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 run before any of the rewrite rules e.g. reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 This results in a request to the webserver which looks like GET /tomcatcontext/ HTTP/1.1 X-Original-Request: / Host: example.com Connection: keep-alive ... This all works great until you then try and do some acl matching in the haproxy, because an acl like acl example-com hdr_end(host) -i example.com will no longer match. It should (tested quicky here), can you provide us the version of haproxy you're using ? I remember there was a bug in old 1.4 versions concerning headers manipulation. I was using 1.4.19 but have just updated to 1.4.20 and still having the same problem, complete example config below global daemon quiet maxconn 1024 pidfile /home/haproxyblue/haproxy.pid uid 20003 gid 20003 chroot /home/haproxyblue log 127.0.0.1 local0 log 127.0.0.1 local1 notice defaults log global option httplog balance roundrobin mode http retries 3 option redispatch timeout connect 30 timeout client 30 timeout server 30 frontend http-in bind *:80 reqirep ^((HEAD|GET|POST|PUT|DELETE|TRACE|OPTIONS|CONNECT|PATCH)\ ([^\ ]*)\ HTTP/1.[01]) \1\nX-Original-Request:\ \3 acl test hdr_end(host) -i example.com reqrep ^([^\ \t]*[\ \t])(.*) \1/tomcatcontext\2 if test use_backend echo if test backend echo mode http option http-server-close option forwardfor server echo1 127.0.0.1: Hi, You hould enable http-server-close on the frontend side as well. Or better, put it in the defaults. cheers
