[
https://issues.apache.org/jira/browse/TS-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728968#comment-13728968
]
Alan M. Carroll commented on TS-2051:
-------------------------------------
A theory on this issue --
There are some very odd things in the stack trace. The logic around frame 25
seems to indicate a having just completed client side handshake, but frame #19
looks like to be on the server side. The crash itself looks to be caused by a
HttpTunnelProducer instance that has already been run (and so lost the start
buffer reader) but we can see the tunnel being set up in frame #4.
Based on this and other things, I think this is coming out of
HttpSM::handle_post_failure. That method sets up the two conditions we need for
this result, a net read of zero (line 4930) and setting up
state_read_server_response (line 4948). Note that the zero read is done to stop
read IO. This would seem to be effective for normal traffic but SSL traffic has
the special case for handshake that will trigger a READ_COMPLETE if there is
data waiting on the socket.
In this view, the relation to TS-1320 is that before that change. all pending
data on the socket would be consumed before handle_post_failure and so
net_read_io would not be triggered. With TS-1320, some data can be left and if
so we get the net_read_io call and things proceed off the cliff.
Things to try:
1) Put a warning in HttpSM::handle_post_failure to see if that goes off.
2) We could check in SSLNetVConnection::net_read_io if the connection was
closed, or only do the readSignalDone when the handshake complets (that is,
signal only if !getSSLHandshakeComplete() when the method was entered).
> ATS crashing shortly after starting when SSL is enabled
> -------------------------------------------------------
>
> Key: TS-2051
> URL: https://issues.apache.org/jira/browse/TS-2051
> Project: Traffic Server
> Issue Type: Bug
> Components: Core, SSL
> Reporter: David Carlin
> Assignee: Leif Hedstrom
> Priority: Blocker
> Fix For: 3.3.5
>
>
> We went to test master because it contains a fix for TS-2035. It seems to
> crash if SSL is enabled shortly after startup. The filenames for the core
> files indicate they are from SSL threads.
> CONFIG proxy.config.http.server_ports STRING 80 81 82 83 84 85 90 80:ipv6
> 81:ipv6 443:ipv6:ssl 444:ipv6:ssl 443:ssl 444:ssl 445:ssl 446:ssl 447:ssl
> 448:ssl 490:ssl
> Stack trace:
> {quote}
> /home/y/bin/traffic_server - STACK TRACE:
> /lib64/libpthread.so.0(+0x3f8240f500)[0x2aafe95f3500]
> /home/y/bin/traffic_server(_ZN10HttpTunnel12producer_runEP18HttpTunnelProducer+0x25f)[0x568daf]
> /home/y/bin/traffic_server(_ZN10HttpTunnel10tunnel_runEP18HttpTunnelProducer+0xd9)[0x569f09]
> /home/y/bin/traffic_server(_ZN6HttpSM23setup_internal_transferEMS_FiiPvE+0x632)[0x526172]
> /home/y/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x487)[0x532357]
> /home/y/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x529220]
> /home/y/bin/traffic_server(_ZN6HttpSM18state_api_callbackEiPv+0x8a)[0x52f1ba]
> /home/y/bin/traffic_server(TSHttpTxnReenable+0x404)[0x4b5ad4]
> /home/y/libexec64/trafficserver/header_filter.so(+0x2d5d)[0x2ab03679ad5d]
> /home/y/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x114)[0x529084]
> /home/y/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x114)[0x530714]
> /home/y/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x3aa)[0x53227a]
> /home/y/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x529220]
> /home/y/bin/traffic_server(_ZN6HttpSM18state_api_callbackEiPv+0x8a)[0x52f1ba]
> /home/y/bin/traffic_server(TSHttpTxnReenable+0x404)[0x4b5ad4]
> /home/y/libexec64/trafficserver/header_filter.so(+0x2d5d)[0x2ab03679ad5d]
> /home/y/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x114)[0x529084]
> /home/y/bin/traffic_server(_ZN6HttpSM33state_read_server_response_headerEiPv+0x390)[0x52c830]
> /home/y/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xd8)[0x52efc8]
> /home/y/bin/traffic_server(_ZN18UnixNetVConnection14readSignalDoneEiP10NetHandler+0x3d)[0x67fc3d]
> /home/y/bin/traffic_server(_ZN17SSLNetVConnection11net_read_ioEP10NetHandlerP7EThread+0xb91)[0x6704b1]
> /home/y/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x1f2)[0x679632]
> /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x6a1aff]
> /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x4a3)[0x6a24e3]
> /home/y/bin/traffic_server[0x6a099a]
> /lib64/libpthread.so.0(+0x3f82407851)[0x2aafe95eb851]
> /lib64/libc.so.6(clone+0x6d)[0x3f820e890d]
> {quote}
> Backtrace:
> {quote}
> #0 operator= (this=<value optimized out>, p=<value optimized out>) at
> ../../lib/ts/Ptr.h:425
> #1 clone_reader (this=<value optimized out>, p=<value optimized out>) at
> ../../iocore/eventsystem/P_IOBuffer.h:873
> #2 HttpTunnel::producer_run (this=<value optimized out>, p=<value optimized
> out>) at HttpTunnel.cc:831
> #3 0x0000000000569f09 in HttpTunnel::tunnel_run (this=0x2ba83d9425e8,
> p_arg=0x0) at HttpTunnel.cc:698
> #4 0x0000000000526172 in HttpSM::setup_internal_transfer
> (this=0x2ba83d940a50, handler_arg=(int (HttpSM::*)(HttpSM *, int, void *))
> 0x5328d0 <HttpSM::tunnel_handler_cache_fill(int, void*)>) at HttpSM.cc:5872
> #5 0x0000000000532357 in HttpSM::handle_api_return (this=0x2ba83d940a50) at
> HttpSM.cc:1598
> #6 0x0000000000529220 in HttpSM::state_api_callout (this=0x2ba83d940a50,
> event=60000, data=0x0) at HttpSM.cc:1484
> #7 0x000000000052f1ba in HttpSM::state_api_callback (this=0x2ba83d940a50,
> event=60000, data=0x0) at HttpSM.cc:1303
> #8 0x00000000004b5ad4 in TSHttpTxnReenable (txnp=0x2ba83d940a50,
> event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5530
> #9 0x00002ba69a79ad5d in cont_header_filter (event=<value optimized out>,
> edata=0x2ba83d940a50) at header_filter.cc:90
> #10 0x0000000000529084 in HttpSM::state_api_callout (this=0x2ba83d940a50,
> event=<value optimized out>, data=<value optimized out>) at HttpSM.cc:1410
> #11 0x0000000000530714 in HttpSM::set_next_state (this=0x2ba83d940a50) at
> HttpSM.cc:7050
> #12 0x000000000053227a in HttpSM::handle_api_return (this=0x2ba83d940a50) at
> HttpSM.cc:1552
> #13 0x0000000000529220 in HttpSM::state_api_callout (this=0x2ba83d940a50,
> event=60000, data=0x0) at HttpSM.cc:1484
> #14 0x000000000052f1ba in HttpSM::state_api_callback (this=0x2ba83d940a50,
> event=60000, data=0x0) at HttpSM.cc:1303
> #15 0x00000000004b5ad4 in TSHttpTxnReenable (txnp=0x2ba83d940a50,
> event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5530
> #16 0x00002ba69a79ad5d in cont_header_filter (event=<value optimized out>,
> edata=0x2ba83d940a50) at header_filter.cc:90
> #17 0x0000000000529084 in HttpSM::state_api_callout (this=0x2ba83d940a50,
> event=<value optimized out>, data=<value optimized out>) at HttpSM.cc:1410
> #18 0x000000000052c830 in do_api_callout (this=0x2ba83d940a50, event=<value
> optimized out>, data=0x2ba6e000dcb8) at HttpSM.cc:504
> #19 HttpSM::state_read_server_response_header (this=0x2ba83d940a50,
> event=<value optimized out>, data=0x2ba6e000dcb8) at HttpSM.cc:1866
> #20 0x000000000052efc8 in HttpSM::main_handler (this=0x2ba83d940a50,
> event=102, data=0x2ba6e000dcb8) at HttpSM.cc:2501
> #21 0x000000000067fc3d in handleEvent (this=0x2ba6e000dbb0, event=<value
> optimized out>, nh=0x2ba66029bbf0) at
> ../../iocore/eventsystem/I_Continuation.h:146
> #22 read_signal_and_update (this=0x2ba6e000dbb0, event=<value optimized out>,
> nh=0x2ba66029bbf0) at UnixNetVConnection.cc:138
> #23 read_signal_done (this=0x2ba6e000dbb0, event=<value optimized out>,
> nh=0x2ba66029bbf0) at UnixNetVConnection.cc:168
> #24 UnixNetVConnection::readSignalDone (this=0x2ba6e000dbb0, event=<value
> optimized out>, nh=0x2ba66029bbf0) at UnixNetVConnection.cc:889
> #25 0x00000000006704b1 in SSLNetVConnection::net_read_io
> (this=0x2ba6e000dbb0, nh=0x2ba66029bbf0, lthread=0x2ba660298010) at
> SSLNetVConnection.cc:245
> #26 0x0000000000679632 in NetHandler::mainNetEvent (this=0x2ba66029bbf0,
> event=<value optimized out>, e=<value optimized out>) at UnixNet.cc:376
> #27 0x00000000006a1aff in handleEvent (this=0x2ba660298010, e=0x27959b0,
> calling_code=5) at I_Continuation.h:146
> #28 EThread::process_event (this=0x2ba660298010, e=0x27959b0, calling_code=5)
> at UnixEThread.cc:141
> #29 0x00000000006a24e3 in EThread::execute (this=0x2ba660298010) at
> UnixEThread.cc:265
> #30 0x00000000006a099a in spawn_thread_internal (a=0x2adec10) at Thread.cc:88
> #31 0x00002ba64c17b851 in start_thread () from /lib64/libpthread.so.0
> #32 0x0000003f820e890d in clone () from /lib64/libc.so.6
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
