[ https://issues.apache.org/jira/browse/TS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phil Sorber updated TS-3283: ---------------------------- Backport to Version: 4.2.3 > Certain SSL handshake error during client-hello hangs the client and leaves > network connection open > --------------------------------------------------------------------------------------------------- > > Key: TS-3283 > URL: https://issues.apache.org/jira/browse/TS-3283 > Project: Traffic Server > Issue Type: Bug > Components: SSL > Reporter: Joe Chung > Assignee: Susan Hinrichs > Fix For: 5.3.0 > > > h3. Problem Description > Send an SSLv2 Client Hello with an old cipher suite request against Traffic > Server 4.2.2, and the connection will freeze on the client side and > eventually time out after 120 seconds. > The Traffic Server detects the SSL error, but instead of closing the > connection, goes on to accept new connections. > h3. Reproduction > === Client: Macbook Pro running OSX Mavericks 10.9.5 === > {code:none} > $ openssl version -a > OpenSSL 0.9.8za 5 Jun 2014 > built on: Aug 10 2014 > platform: darwin64-x86_64-llvm > options: bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) blowfish(idx) > compiler: -arch x86_64 -fmessage-length=0 -pipe -Wno-trigraphs > -fpascal-strings -fasm-blocks -O3 -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H > -DL_ENDIAN -DMD32_REG_T=int -DOPENSSL_NO_IDEA -DOPENSSL_PIC -DOPENSSL_THREADS > -DZLIB -mmacosx-version-min=10.6 > OPENSSLDIR: "/System/Library/OpenSSL" > {code} > h4. The following command triggers the bad behavior on the 4.2.2 server. > {code:none} > $ openssl s_client -connect 192.168.20.130:443 -ssl2 -debug > CONNECTED(00000003) > write to 0x7fb9f2508610 [0x7fb9f300f201] (45 bytes => 45 (0x2D)) > 0000 - 80 2b 01 00 02 00 12 00-00 00 10 07 00 c0 03 00 .+.............. > 0010 - 80 01 00 80 06 00 40 04-00 80 02 00 80 f4 71 1a ......@.......q. > 0020 - ad 23 06 59 4d f8 d2 c5-b2 57 a9 66 4c .#.YM....W.fL > ^C > {code} > At this point, the client is hung, and I have to hit ctrl-c to interrupt it > or wait 120 seconds for tcp timeout. > h3. Server: Lubuntu 13.10 on VMware > {code:none} > $ openssl version -a > OpenSSL 1.0.1e 11 Feb 2013 > built on: Fri Jun 20 18:52:25 UTC 2014 > platform: debian-i386 > options: bn(64,32) rc4(8x,mmx) des(ptr,risc1,16,long) blowfish(idx) > compiler: cc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT > -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector > --param=ssp-buffer-size=4 -Wformat -Werror=format-security > -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack > -Wall -DOPENSSL_NO_TLS1_2_CLIENT -DOPENSSL_MAX_TLS1_2_CIPHER_LENGTH=50 > -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT > -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM > -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM > OPENSSLDIR: "/usr/lib/ssl" > {code} > {code:none} > $ diff /usr/local/etc/trafficserver/records.config.422 > /usr/local/etc/trafficserver/records.config > 113c113 > < CONFIG proxy.config.http.server_ports STRING 8080 > --- > > CONFIG proxy.config.http.server_ports STRING 8080 443:ssl > 594,595c594,595 > < CONFIG proxy.config.diags.debug.enabled INT 0 > < CONFIG proxy.config.diags.debug.tags STRING http.*|dns.* > --- > > CONFIG proxy.config.diags.debug.enabled INT 1 > > CONFIG proxy.config.diags.debug.tags STRING ssl.* > {code} > {code:none} > $ /usr/local/bin/traffic_server --version > [TrafficServer] using root directory '/usr/local' > Apache Traffic Server - traffic_server - 4.2.2 - (build # 0723 on Jan 7 2015 > at 23:04:32) > $ sudo /usr/local/bin/traffic_server > [sudo] password for user: > [TrafficServer] using root directory '/usr/local' > [Jan 8 00:53:42.618] Server {0xb702e700} DEBUG: (ssl) setting SNI callbacks > with for ctx 0xa4a7928 > [Jan 8 00:53:42.618] Server {0xb702e700} DEBUG: (ssl) indexed '*' with > SSL_CTX 0xa4a7928 > [Jan 8 00:53:42.619] Server {0xb702e700} DEBUG: (ssl) importing SNI names > from /usr/local/etc/trafficserver > [Jan 8 00:54:02.256] Server {0xb6265b40} DEBUG: (ssl) > [SSLNextProtocolAccept:mainEvent] event 202 netvc 0xb280fa90 > [Jan 8 00:54:02.256] Server {0xb6265b40} DEBUG: (ssl) ssl_callback_info ssl: > 0xb280fcb8 where: 16 ret: 1 > [Jan 8 00:54:02.256] Server {0xb6265b40} DEBUG: (ssl) ssl_callback_info ssl: > 0xb280fcb8 where: 8193 ret: 1 > [Jan 8 00:54:02.256] Server {0xb6265b40} DEBUG: (ssl) ssl_callback_info ssl: > 0xb280fcb8 where: 8194 ret: -1 > [Jan 8 00:54:02.256] Server {0xb6265b40} DEBUG: (ssl) > SSL::3055967040:error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown > protocol:s23_srvr.c:628 > [Jan 8 00:54:02.256] Server {0xb6265b40} DEBUG: <SSLNetVConnection.cc:526 > (sslServerHandShakeEvent)> (ssl) SSL handshake error: SSL_ERROR_SSL (1), > errno=0 > {code} > At this point, the server has not closed the tcp connection, and has instead > gone back to the accept loop, waiting for new connections. > After 120 seconds (TCP timeout) the connection is eventually closed. During > this time, the server process holds onto a file descriptor needlessly for 120 > seconds. I connected to the server from multiple clients and counted the > server's number of open fds after each connection ({{sudo ls -1 /proc/$pid/fd > | wc -l}}). > This behavior makes the server vulnerable to a denial-of-service attack. > h3. Possible Fix > In gdb, I compared the behavior of 5.1.2 vs 4.2.2 in the face of an > {{SSL_ERROR_SSL}} during the Client Hello handshake. I noticed that in 5.1.2, > there is a {{netvc->do_io(VIO::CLOSE)}} in > {{SSLNextProtocolTrampoline::ioCompletionEvent()}} that closes the TCP > connection. > When I apply the following diff to the 4.2.2 code base (essentially making > this function look more like the 5.1.2 version), it fixes the problem; i.e. > the server immediately closes the TCP connection after the error is detected. > {code:none} > $ git diff > diff --git a/iocore/net/SSLNextProtocolAccept.cc > b/iocore/net/SSLNextProtocolAccept.cc > index bfc2f0b..a5612c6 100644 > --- a/iocore/net/SSLNextProtocolAccept.cc > +++ b/iocore/net/SSLNextProtocolAccept.cc > @@ -78,19 +78,22 @@ struct SSLNextProtocolTrampoline : public Continuation > Continuation * plugin; > SSLNetVConnection * netvc; > + vio = static_cast<VIO *>(edata); > + netvc = dynamic_cast<SSLNetVConnection *>(vio->vc_server); > + ink_assert(netvc != NULL); > + > switch (event) { > case VC_EVENT_INACTIVITY_TIMEOUT: > - case VC_EVENT_READ_COMPLETE: > case VC_EVENT_ERROR: > - vio = static_cast<VIO *>(edata); > + netvc->do_io(VIO::CLOSE); > + delete this; > + return EVENT_CONT; > + case VC_EVENT_READ_COMPLETE: > break; > default: > return EVENT_ERROR; > } > - netvc = dynamic_cast<SSLNetVConnection *>(vio->vc_server); > - ink_assert(netvc != NULL); > - > plugin = netvc->endpoint(); > if (plugin) { > send_plugin_event(plugin, NET_EVENT_ACCEPT, netvc); > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)