Le lundi 27 décembre 2010 07:10:37, Willy Tarreau a écrit :
> It's nice that you managed to reproduce it ! I'm not sure how you managed
> to get it though because you're saying that you have it with and without
> nginx.
Oh sorry, I thought it was clear enough with the first mail.
Ok, here is the full test.
nginx is in front of haproxy with this configuration :
events {
worker_connections 1000;
}
http {
server {
listen 8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location /httpclose {
proxy_pass http://127.0.0.1:8081;
}
location /tunnel {
proxy_pass http://127.0.0.1:8082;
}
location /forceclose {
proxy_pass http://127.0.0.1:8083;
}
}
}
haproxy is run with :
listen httpclose_test :8081
mode http
option httpclose
option http-pretend-keepalive
server www localhost:80
listen tunnel_test :8082
mode http
option http-pretend-keepalive
server www localhost:80
listen forceclose_test :8083
mode http
option forceclose
option http-pretend-keepalive
server www localhost:80
The backend is apache with KeepAliveTimeout 15.
Now some results :
* TEST 1 with "option httpclose" and "option http-pretend-keepalive"
$ time curl -i http://localhost:8080/httpclose
HTTP/1.1 404 Not Found
Server: nginx/0.8.53
Date: Mon, 27 Dec 2010 17:20:56 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: keep-alive
Vary: Accept-Encoding
Content-Length: 282
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /httpclose was not found on this server.</p>
<hr>
<address>Apache/2.2.16 (Ubuntu) Server at localhost Port 80</address>
</body></html>
real 0m15.027s
user 0m0.008s
sys 0m0.000s
During this test, the haproxy debug is :
00000000:httpclose_test.accept(0004)=0007 from [127.0.0.1:42571]
00000000:httpclose_test.clireq[0007:ffff]: GET /httpclose HTTP/1.0
00000000:httpclose_test.clihdr[0007:ffff]: Host: localhost
00000000:httpclose_test.clihdr[0007:ffff]: X-Real-IP: 127.0.0.1
00000000:httpclose_test.clihdr[0007:ffff]: X-Forwarded-For: 127.0.0.1
00000000:httpclose_test.clihdr[0007:ffff]: Connection: close
00000000:httpclose_test.clihdr[0007:ffff]: User-Agent: curl/7.21.2 (i686-pc-
linux-gnu) libcurl/7.21.2 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
00000000:httpclose_test.clihdr[0007:ffff]: Accept: */*
00000000:httpclose_test.srvrep[0007:0008]: HTTP/1.1 404 Not Found
00000000:httpclose_test.srvhdr[0007:0008]: Date: Mon, 27 Dec 2010 17:14:01 GMT
00000000:httpclose_test.srvhdr[0007:0008]: Server: Apache/2.2.16 (Ubuntu)
00000000:httpclose_test.srvhdr[0007:0008]: Vary: Accept-Encoding
00000000:httpclose_test.srvhdr[0007:0008]: Content-Length: 282
00000000:httpclose_test.srvhdr[0007:0008]: Keep-Alive: timeout=15, max=100
00000000:httpclose_test.srvhdr[0007:0008]: Connection: Keep-Alive
00000000:httpclose_test.srvhdr[0007:0008]: Content-Type: text/html;
charset=iso-8859-1
00000000:httpclose_test.srvcls[0007:0008]
00000000:httpclose_test.clicls[0007:0008]
00000000:httpclose_test.closed[0007:0008]
* TEST 2 with only "option http-pretend-keepalive"
$ time curl -i http://localhost:8080/tunnel
HTTP/1.1 404 Not Found
Server: nginx/0.8.53
Date: Mon, 27 Dec 2010 17:24:57 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: keep-alive
Vary: Accept-Encoding
Content-Length: 279
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /tunnel was not found on this server.</p>
<hr>
<address>Apache/2.2.16 (Ubuntu) Server at localhost Port 80</address>
</body></html>
real 0m15.024s
user 0m0.012s
sys 0m0.000s
haproxy debug :
00000001:tunnel_test.accept(0005)=0007 from [127.0.0.1:33892]
00000001:tunnel_test.clireq[0007:ffff]: GET /tunnel HTTP/1.0
00000001:tunnel_test.clihdr[0007:ffff]: Host: localhost
00000001:tunnel_test.clihdr[0007:ffff]: X-Real-IP: 127.0.0.1
00000001:tunnel_test.clihdr[0007:ffff]: X-Forwarded-For: 127.0.0.1
00000001:tunnel_test.clihdr[0007:ffff]: Connection: close
00000001:tunnel_test.clihdr[0007:ffff]: User-Agent: curl/7.21.2 (i686-pc-linux-
gnu) libcurl/7.21.2 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
00000001:tunnel_test.clihdr[0007:ffff]: Accept: */*
00000001:tunnel_test.srvrep[0007:0008]: HTTP/1.1 404 Not Found
00000001:tunnel_test.srvhdr[0007:0008]: Date: Mon, 27 Dec 2010 17:24:57 GMT
00000001:tunnel_test.srvhdr[0007:0008]: Server: Apache/2.2.16 (Ubuntu)
00000001:tunnel_test.srvhdr[0007:0008]: Vary: Accept-Encoding
00000001:tunnel_test.srvhdr[0007:0008]: Content-Length: 279
00000001:tunnel_test.srvhdr[0007:0008]: Keep-Alive: timeout=15, max=100
00000001:tunnel_test.srvhdr[0007:0008]: Connection: Keep-Alive
00000001:tunnel_test.srvhdr[0007:0008]: Content-Type: text/html;
charset=iso-8859-1
00000001:tunnel_test.srvcls[0007:0008]
00000001:tunnel_test.clicls[0007:0008]
00000001:tunnel_test.closed[0007:0008]
* TEST 3 with "option forceclose" and "option http-pretend-keepalive"
$ time curl -i http://localhost:8080/forceclose
HTTP/1.1 404 Not Found
Server: nginx/0.8.53
Date: Mon, 27 Dec 2010 17:28:53 GMT
Content-Type: text/html; charset=iso-8859-1
Connection: keep-alive
Vary: Accept-Encoding
Content-Length: 283
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /forceclose was not found on this server.</p>
<hr>
<address>Apache/2.2.16 (Ubuntu) Server at localhost Port 80</address>
</body></html>
real 0m0.012s
user 0m0.000s
sys 0m0.008s
haproxy debug :
00000002:forceclose_test.accept(0006)=0007 from [127.0.0.1:33795]
00000002:forceclose_test.clireq[0007:ffff]: GET /forceclose HTTP/1.0
00000002:forceclose_test.clihdr[0007:ffff]: Host: localhost
00000002:forceclose_test.clihdr[0007:ffff]: X-Real-IP: 127.0.0.1
00000002:forceclose_test.clihdr[0007:ffff]: X-Forwarded-For: 127.0.0.1
00000002:forceclose_test.clihdr[0007:ffff]: Connection: close
00000002:forceclose_test.clihdr[0007:ffff]: User-Agent: curl/7.21.2 (i686-pc-
linux-gnu) libcurl/7.21.2 OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.18
00000002:forceclose_test.clihdr[0007:ffff]: Accept: */*
00000002:forceclose_test.srvrep[0007:0008]: HTTP/1.1 404 Not Found
00000002:forceclose_test.srvhdr[0007:0008]: Date: Mon, 27 Dec 2010 17:28:53
GMT
00000002:forceclose_test.srvhdr[0007:0008]: Server: Apache/2.2.16 (Ubuntu)
00000002:forceclose_test.srvhdr[0007:0008]: Vary: Accept-Encoding
00000002:forceclose_test.srvhdr[0007:0008]: Content-Length: 283
00000002:forceclose_test.srvhdr[0007:0008]: Keep-Alive: timeout=15, max=100
00000002:forceclose_test.srvhdr[0007:0008]: Connection: Keep-Alive
00000002:forceclose_test.srvhdr[0007:0008]: Content-Type: text/html;
charset=iso-8859-1
00000002:forceclose_test.srvcls[0007:0008]
00000002:forceclose_test.clicls[0007:0008]
00000002:forceclose_test.closed[0007:0008]
(Note that when disabling proxy buffering in nginx, the 15s delay is fixed).
Now, I can also observe the same behaviour using ab directly on haproxy :
$ ab http://localhost:8081/
=> 15s delay
$ ab http://localhost:8082/
=> 15s delay
$ ab http://localhost:8083/
=> immediate response
> [ I unwrapped your patch because apparently your mailer did not like it ]
Yes sorry for that, I'm thinking of changing my mailer :-/
> > ((s->fe->options2 & PR_O2_FAKE_KA) != (s->be->options & >
> > PR_O2_FAKE_KA))) {
>
> ^^^
> Here above it should be "options2".
Oops, fixed in my code.
> I must say I'm having difficulties to understand the logics. The rule is
> becoming quite complex, so we should probably first try to define how
> we'd like pretend-keep-alive to work in various cases.
Same here, maybe it should/can be simplified in the 1.5 branch (or future 1.6).
And i fear it will become more and more complex the day haproxy supports
client side keep-alive (if it will).
Due to this complexity, I really wanted a second look...which was needed (the
typo in the variable name confirmed it :-)).
> In my opinion, this option's goal is to make a server think we're sending
> it a keep-alive request so that it can announce a valid content-length or
> chunking to the client, eventhough we intend to close its connection. So
> what I'm thinking is that this option should only be considered if either
> "httpclose", "forceclose" or "http-server-close" is enabled in either the
> frontend or backend, or if the request indicates a close.
>
> I also think that if we have httpclose (or the request indicated a close)
> with the option set, then we can assume the server will not close after
> its response, which can make the client wait for a timeout. Thus, we should
> probably turn the response to a forced close in this case.
>
> Do you agree with that ? Is it what you wanted to do ?
I agree. You precisely explained what I tried to fix.
> Do you see any other case that must be covered ?
I only see the "tunnel mode" case, where http-pretend-keepalive also
introduces this timeout side effect. With this patch, a forced close is applied
in that case too.
If it's ok for you, I'll resend the updated patch in a correct way.
--
Cyril Bonté
