Context: SSL stuff, haproxy HA-Proxy version 1.5-dev15 2012/12/12 ;
complete haproxy info at the bottom.
Our app does large file uploads via an ancillary java applet thingy;
these look like so:
00000009:https.accept(0006)=000f from [64.236.139.254:35763]
00000009:https.clireq[000f:ffff]: POST /cytobank/uploadServlet HTTP/1.1
00000009:https.clihdr[000f:ffff]: User-Agent: Cytobank Upload
00000009:https.clihdr[000f:ffff]: Content-Type: multipart/form-data;
boundary=756zrpPEjQyIs1KS9cHjGzvox7mMfeCZR0
00000009:https.clihdr[000f:ffff]: Cache-Control: no-cache
00000009:https.clihdr[000f:ffff]: Pragma: no-cache
00000009:https.clihdr[000f:ffff]: Host: dvs-staging.cytobank.org
00000009:https.clihdr[000f:ffff]: Accept: text/html, image/gif, image/jpeg, *;
q=.2, */*; q=.2
00000009:https.clihdr[000f:ffff]: Connection: keep-alive
00000009:https.clihdr[000f:ffff]: Content-Length: 6451523
00000009:https.clihdr[000f:ffff]: Cookie:
__utma=51498683.1371821700.1307729573.1357763653.1357770695.223;
__utmz=51498683.1348737696.200.4.utmcsr=localhost|utmccn=(referral)|utmcmd=referral|utmcct=/cytobank/subscriptions/subscribe;
__utma=20047853.812597310.1357601503.1357799042.1357864337.4;
__utmz=20047853.1357601503.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
__utmb=20047853.32.10.1357864337; __utmc=20047853; openid_provider=Google;
user_credentials=ff9ed09a5a08f5103cdce0bcf9c7eda9d9bcee3e2e5a392b0e0b066780f5cfdb2a505a8bdd17d505057b751b20a96989ed61f6435a057d751958f034c095be50%3A%3A2
00000009:dvs-staging.cytobank.org-https-tomcat.srvrep[000f:0010]: HTTP/1.1 200
OK
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Server:
Apache-Coyote/1.1
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Content-Type:
text/html;charset=ISO-8859-1
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]:
Transfer-Encoding: chunked
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Date: Fri, 11
Jan 2013 01:13:19 GMT
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Connection:
close
00000009:https.clicls[000f:ffff]
00000009:https.closed[000f:ffff]
I'm using haproxy's ssl, like so:
bind 0.0.0.0:443 ssl crt /opt/haproxy/etc/wildcard.cert
If the config for this backend is like this:
use_backend dvs-staging.cytobank.org-https-tomcat if { path_reg -i
^/cytobank/ }
everything works fine, but if it's like this:
use_backend dvs-staging.cytobank.org-https-tomcat if { ssl_fc_sni -i
dvs-staging.cytobank.org } { path_reg -i ^/cytobank/ }
The upload goes very slowly and then fails.
There is no obvious difference between the outputs in these cases;
the above was successful, this is a failed one:
00000009:https.accept(0006)=000f from [64.236.139.254:45900]
00000009:https.clireq[000f:ffff]: POST /cytobank/uploadServlet HTTP/1.1
00000009:https.clihdr[000f:ffff]: User-Agent: Cytobank Upload
00000009:https.clihdr[000f:ffff]: Content-Type: multipart/form-data;
boundary=qKulbQFAOI0yUOyRv5tJ8Itzvy4aMPqP58
00000009:https.clihdr[000f:ffff]: Cache-Control: no-cache
00000009:https.clihdr[000f:ffff]: Pragma: no-cache
00000009:https.clihdr[000f:ffff]: Host: dvs-staging.cytobank.org
00000009:https.clihdr[000f:ffff]: Accept: text/html, image/gif, image/jpeg, *;
q=.2, */*; q=.2
00000009:https.clihdr[000f:ffff]: Connection: keep-alive
00000009:https.clihdr[000f:ffff]: Content-Length: 6451523
00000009:https.clihdr[000f:ffff]: Cookie:
__utma=51498683.1371821700.1307729573.1357763653.1357770695.223;
__utmz=51498683.1348737696.200.4.utmcsr=localhost|utmccn=(referral)|utmcmd=referral|utmcct=/cytobank/subscriptions/subscribe;
__utma=20047853.812597310.1357601503.1357799042.1357864337.4;
__utmz=20047853.1357601503.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);
__utmb=20047853.28.10.1357864337; __utmc=20047853; openid_provider=Google;
user_credentials=ff9ed09a5a08f5103cdce0bcf9c7eda9d9bcee3e2e5a392b0e0b066780f5cfdb2a505a8bdd17d505057b751b20a96989ed61f6435a057d751958f034c095be50%3A%3A2
00000007:http.clicls[000e:ffff]
00000007:http.closed[000e:ffff]
00000005:https.clicls[000a:ffff]
00000005:https.closed[000a:ffff]
00000002:https.clicls[0009:ffff]
00000002:https.closed[0009:ffff]
00000004:https.clicls[000c:ffff]
00000004:https.closed[000c:ffff]
00000003:https.clicls[000b:ffff]
00000003:https.closed[000b:ffff]
00000001:https.clicls[0008:ffff]
00000001:https.closed[0008:ffff]
00000006:https.clicls[000d:ffff]
00000006:https.closed[000d:ffff]
00000009:dvs-staging.cytobank.org-https-tomcat.srvrep[000f:0010]: HTTP/1.1 200
OK
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Server:
Apache-Coyote/1.1
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Content-Type:
text/html;charset=ISO-8859-1
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]:
Transfer-Encoding: chunked
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Date: Fri, 11
Jan 2013 01:08:39 GMT
00000009:dvs-staging.cytobank.org-https-tomcat.srvhdr[000f:0010]: Connection:
close
00000009:dvs-staging.cytobank.org-https-tomcat.srvcls[000f:0010]
00000009:dvs-staging.cytobank.org-https-tomcat.clicls[000f:0010]
00000009:dvs-staging.cytobank.org-https-tomcat.closed[000f:0010]
(all the extra closes are because it took so long, AFAICT).
I'm pretty sure this is an haproxy bug. My *guess* is that what's
happening is that the first chunk is sni-ing properly, but that
everything else on the same keep-alive session is failing the sni
check, and effectively getting thrown away.
If it's PEBKAC, I'd love some assistance.
Thanks!
-Robin
[rlpowell@ds1007 ~]$ sudo /opt/haproxy/usr/local/sbin/haproxy -vv
HA-Proxy version 1.5-dev15 2012/12/12
Copyright 2000-2012 Willy Tarreau <[email protected]>
Build options :
TARGET = linux26
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing
OPTIONS = USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built without zlib support (USE_ZLIB not set)
Compression algorithms supported : identity
Built with OpenSSL version : OpenSSL 1.0.1c 10 May 2012
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
