[
https://issues.apache.org/jira/browse/TS-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600618#comment-14600618
]
Leif Hedstrom commented on TS-3710:
-----------------------------------
I spent some time testing stuff, and since the crash doesn't happen often, it's
been rather slow process :). But, right now, the best candidate causing these
crashes is the commit c8c9c4fa0dad3.
I did the following on both my boxes, and no crashes so far (about 7 hours in).
{code}
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 81a0be5..74ab417 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -209,15 +209,9 @@ ssl_get_cached_session(SSL *ssl, unsigned char *id, int
len, int *copy)
SSL_SESSION *session = NULL;
if (session_cache->getSession(sid, &session)) {
- // Double check the timeout
- if (session && ssl_session_timed_out(session)) {
- // Due to bug in openssl, the timeout is checked, but only removed
- // from the openssl built-in hash table. The external remove cb is not
called
- ssl_rm_cached_session(SSL_get_SSL_CTX(ssl), session);
- session = NULL;
- }
+ return session;
}
- return session;
+ return NULL;
}
static int
{code}
This is basically reverting the relevant portions from TS-3687.
> ASAN crash in TLS with 6.0.0
> ----------------------------
>
> Key: TS-3710
> URL: https://issues.apache.org/jira/browse/TS-3710
> Project: Traffic Server
> Issue Type: Bug
> Components: SSL
> Reporter: Leif Hedstrom
> Priority: Critical
> Fix For: 6.0.0
>
>
> {code}
> ==9570==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x606000049f48 at pc 0xb9f969 bp 0x2b8dbc348920 sp 0x2b8dbc348918
> READ of size 8 at 0x606000049f48 thread T8 ([ET_NET 7])
> #0 0xb9f968 in Continuation::handleEvent(int, void*)
> ../../iocore/eventsystem/I_Continuation.h:145
> #1 0xb9f968 in read_signal_and_update
> /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:142
> #2 0xb9f968 in UnixNetVConnection::mainEvent(int, Event*)
> /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:1115
> #3 0xb7daf7 in Continuation::handleEvent(int, void*)
> ../../iocore/eventsystem/I_Continuation.h:145
> #4 0xb7daf7 in InactivityCop::check_inactivity(int, Event*)
> /usr/local/src/trafficserver/iocore/net/UnixNet.cc:102
> #5 0xc21ffe in Continuation::handleEvent(int, void*)
> /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145
> #6 0xc21ffe in EThread::process_event(Event*, int)
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128
> #7 0xc241f7 in EThread::execute()
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:207
> #8 0xc20c18 in spawn_thread_internal
> /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85
> #9 0x2b8db3ff6df4 in start_thread (/lib64/libpthread.so.0+0x7df4)
> #10 0x2b8db585f1ac in __clone (/lib64/libc.so.6+0xf61ac)
> 0x606000049f48 is located 8 bytes inside of 56-byte region
> [0x606000049f40,0x606000049f78)
> freed by thread T8 ([ET_NET 7]) here:
> #0 0x2b8db1bf3117 in operator delete(void*)
> ../../.././libsanitizer/asan/asan_new_delete.cc:81
> #1 0xb5b20e in SSLNextProtocolTrampoline::ioCompletionEvent(int, void*)
> /usr/local/src/trafficserver/iocore/net/SSLNextProtocolAccept.cc:89
> #2 0xbb2eef in Continuation::handleEvent(int, void*)
> ../../iocore/eventsystem/I_Continuation.h:145
> #3 0xbb2eef in read_signal_and_update
> /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:142
> #4 0xbb2eef in read_signal_done
> /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:203
> #5 0xbb2eef in UnixNetVConnection::readSignalDone(int, NetHandler*)
> /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:957
> #6 0xb55d6d in SSLNetVConnection::net_read_io(NetHandler*, EThread*)
> /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:480
> #7 0xb748fc in NetHandler::mainNetEvent(int, Event*)
> /usr/local/src/trafficserver/iocore/net/UnixNet.cc:516
> #8 0xc24e89 in Continuation::handleEvent(int, void*)
> /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145
> #9 0xc24e89 in EThread::process_event(Event*, int)
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128
> #10 0xc24e89 in EThread::execute()
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:252
> #11 0xc20c18 in spawn_thread_internal
> /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85
> #12 0x2b8db3ff6df4 in start_thread (/lib64/libpthread.so.0+0x7df4)
> previously allocated by thread T8 ([ET_NET 7]) here:
> #0 0x2b8db1bf2c9f in operator new(unsigned long)
> ../../.././libsanitizer/asan/asan_new_delete.cc:50
> #1 0xb59f8b in SSLNextProtocolAccept::mainEvent(int, void*)
> /usr/local/src/trafficserver/iocore/net/SSLNextProtocolAccept.cc:134
> #2 0xb888e9 in Continuation::handleEvent(int, void*)
> ../../iocore/eventsystem/I_Continuation.h:145
> #3 0xb888e9 in NetAccept::acceptFastEvent(int, void*)
> /usr/local/src/trafficserver/iocore/net/UnixNetAccept.cc:466
> #4 0xc24e89 in Continuation::handleEvent(int, void*)
> /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:145
> #5 0xc24e89 in EThread::process_event(Event*, int)
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:128
> #6 0xc24e89 in EThread::execute()
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:252
> #7 0xc20c18 in spawn_thread_internal
> /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:85
> #8 0x2b8db3ff6df4 in start_thread (/lib64/libpthread.so.0+0x7df4)
> Thread T8 ([ET_NET 7]) created by T0 ([ET_NET 0]) here:
> #0 0x2b8db1bc186a in __interceptor_pthread_create
> ../../.././libsanitizer/asan/asan_interceptors.cc:183
> #1 0xc218a5 in ink_thread_create ../../lib/ts/ink_thread.h:150
> #2 0xc218a5 in Thread::start(char const*, unsigned long, void*
> (*)(void*), void*)
> /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:100
> #3 0xc29e26 in EventProcessor::start(int, unsigned long)
> /usr/local/src/trafficserver/iocore/eventsystem/UnixEventProcessor.cc:140
> #4 0x495e4b in main /usr/local/src/trafficserver/proxy/Main.cc:1621
> #5 0x2b8db578aaf4 in __libc_start_main (/lib64/libc.so.6+0x21af4)
> SUMMARY: AddressSanitizer: heap-use-after-free
> ../../iocore/eventsystem/I_Continuation.h:145 Continuation::handleEvent(int,
> void*)
> Shadow bytes around the buggy address:
> 0x0c0c80001390: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
> 0x0c0c800013a0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
> 0x0c0c800013b0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
> 0x0c0c800013c0: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
> 0x0c0c800013d0: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
> =>0x0c0c800013e0: fd fd fd fa fa fa fa fa fd[fd]fd fd fd fd fd fa
> 0x0c0c800013f0: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
> 0x0c0c80001400: fa fa fa fa fa fa fa fa fa fa fa fa fd fd fd fd
> 0x0c0c80001410: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
> 0x0c0c80001420: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
> 0x0c0c80001430: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Heap right redzone: fb
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack partial redzone: f4
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user: f7
> Contiguous container OOB:fc
> ASan internal: fe
> ==9570==ABORTING
> traffic_server: using root directory '/opt/ats'
> traffic_server: using root directory '/opt/ats'
> {code}
> Update: Seems I didn't get the latest version of the code / ASAN report
> matched up, this should be with 6.0.x proper.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
