[
https://issues.apache.org/jira/browse/TS-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992603#comment-13992603
]
Feifei Cai commented on TS-2782:
--------------------------------
Hi [~sudheerv],
I think it's caused by [TS-2766|https://issues.apache.org/jira/browse/TS-2766].
The
[commit|https://github.com/apache/trafficserver/commit/1630af7f76643c86751398838fbdf8a1363ce292]
in [TS-2766|https://issues.apache.org/jira/browse/TS-2766] changes/corrects
HdrHeap::coalesce_str_heaps(), which is called by
HdrHeap::inherit_string_heaps(). So HdrHeap after this commit is not compliant
with the previous versions. I tested some versions after this commit, and
there's no such kind of crash issues during upgrades and downgrades.
> ats crash in master in HdrHeap::inherit_string_heaps
> ----------------------------------------------------
>
> Key: TS-2782
> URL: https://issues.apache.org/jira/browse/TS-2782
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Reporter: Sudheer Vinukonda
> Labels: spdy, yahoo
>
> When testing master on production hosts, noticed the below crash occuring
> repeatedly every time ats version is changed. This crash stops happening
> after clearing the cache. This needs further investigation, but, I remember a
> discussion between briang and zwoop about duplicate string fields in HdrHeap.
> Not sure if this core is related to that. Would appreciate if briang or zwoop
> can comment. Thank you.
> {code}
> [example_prep.sh] Checking/Moving old cores...
> [TrafficServer] using root directory '/home/y'
> NOTE: Traffic Server received Sig 11: Segmentation fault
> /home/y/bin/traffic_server - STACK TRACE:
> /lib64/libpthread.so.0(+0x30d3c0f500)[0x2aae27e2d500]
> /home/y/bin/traffic_server(_ZN7HdrHeap20inherit_string_heapsEPKS_+0x271)[0x61caa1]
> /home/y/bin/traffic_server(_Z14http_hdr_cloneP11HTTPHdrImplP7HdrHeapS2_+0x93)[0x619f83]
> /home/y/bin/traffic_server(_ZN19HttpTransactHeaders18copy_header_fieldsEP7HTTPHdrS1_bl+0x1be)[0x5c201e]
> /home/y/bin/traffic_server(_ZN12HttpTransact14build_responseEPNS_5StateEP7HTTPHdrS3_11HTTPVersion10HTTPStatusPKc+0x3ed)[0x5a287d]
> /home/y/bin/traffic_server(_ZN12HttpTransact25build_response_from_cacheEPNS_5StateE15HTTPWarningCode+0x354)[0x5b67f4]
> /home/y/bin/traffic_server(_ZN12HttpTransact22HandleCacheOpenReadHitEPNS_5StateE+0x448)[0x5b84c8]
> /home/y/bin/traffic_server(_ZN6HttpSM32call_transact_and_set_next_stateEPFvPN12HttpTransact5StateEE+0x66)[0x573816]
> /home/y/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x2d2)[0x58ba72]
> /home/y/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x583070]
> /home/y/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1ea)[0x58d06a]
> /home/y/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x2d2)[0x58ba72]
> /home/y/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x2b0)[0x583070]
> /home/y/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x1ea)[0x58d06a]
> /home/y/bin/traffic_server(_ZN6HttpSM21state_cache_open_readEiPv+0xfe)[0x58533e]
> /home/y/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xd8)[0x587de8]
> /home/y/bin/traffic_server(_ZN11HttpCacheSM21state_cache_open_readEiPv+0x1b2)[0x566e12]
> /home/y/bin/traffic_server(_ZN7CacheVC8callcontEi+0x53)[0x653453]
> /home/y/bin/traffic_server(_ZN7CacheVC17openReadStartHeadEiP5Event+0x7cf)[0x6be9af]
> /home/y/bin/traffic_server(_ZN7CacheVC14handleReadDoneEiP5Event+0x1ed)[0x69d40d]
> /home/y/bin/traffic_server(_ZN19AIOCallbackInternal11io_completeEiPv+0x35)[0x6539c5]
> /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x714aef]
> /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x61b)[0x71561b]
> /home/y/bin/traffic_server[0x713e9a]
> /lib64/libpthread.so.0(+0x30d3c07851)[0x2aae27e25851]
> /lib64/libc.so.6(clone+0x6d)[0x30d38e890d]
> {code}
> gdb output below:
> {code}
> (gdb) bt
> #0 ink_atomic_increment<int, int> (this=0x2afb60113010,
> inherit_from=0x2afaa824e688) at ../../lib/ts/ink_atomic.h:162
> #1 refcount_inc (this=0x2afb60113010, inherit_from=0x2afaa824e688) at
> ../../lib/ts/Ptr.h:279
> #2 operator= (this=0x2afb60113010, inherit_from=0x2afaa824e688) at
> ../../lib/ts/Ptr.h:408
> #3 attach_str_heap (this=0x2afb60113010, inherit_from=0x2afaa824e688) at
> HdrHeap.cc:1000
> #4 HdrHeap::inherit_string_heaps (this=0x2afb60113010,
> inherit_from=0x2afaa824e688) at HdrHeap.cc:1081
> #5 0x0000000000619f83 in http_hdr_clone (s_hh=0x2afaa824e710,
> s_heap=0x2afaa824e688, d_heap=0x2afb60113010) at HTTP.cc:375
> #6 0x00000000005c201e in copy (src_hdr=0x2afaa824e0b8,
> new_hdr=0x2afac4058c50, retain_proxy_auth_hdrs=false, date=0) at
> ../../proxy/hdrs/HTTP.h:867
> #7 HttpTransactHeaders::copy_header_fields (src_hdr=0x2afaa824e0b8,
> new_hdr=0x2afac4058c50, retain_proxy_auth_hdrs=false, date=0) at
> HttpTransactHeaders.cc:201
> #8 0x00000000005a287d in HttpTransact::build_response (s=0x2afac4058570,
> base_response=0x2afaa824e0b8, outgoing_response=0x2afac4058c50,
> outgoing_version=<value optimized out>,
> status_code=HTTP_STATUS_NONE, reason_phrase=0x7323ac "None") at
> HttpTransact.cc:7926
> #9 0x00000000005b67f4 in HttpTransact::build_response_from_cache
> (s=0x2afac4058570, warning_code=HTTP_WARNING_CODE_NONE) at
> HttpTransact.cc:2925
> #10 0x00000000005b84c8 in HttpTransact::HandleCacheOpenReadHit
> (s=0x2afac4058570) at HttpTransact.cc:2811
> #11 0x0000000000573816 in HttpSM::call_transact_and_set_next_state
> (this=0x2afac4058500, f=<value optimized out>) at HttpSM.cc:6748
> #12 0x000000000058ba72 in HttpSM::handle_api_return (this=0x2afac4058500) at
> HttpSM.cc:1501
> #13 0x0000000000583070 in HttpSM::state_api_callout (this=0x2afac4058500,
> event=60000, data=0x0) at HttpSM.cc:1433
> #14 0x0000000000587fdb in HttpSM::state_api_callback (this=0x2afac4058500,
> event=60000, data=0x0) at HttpSM.cc:1251
> #15 0x00000000004c0acc in TSHttpTxnReenable (txnp=0x2afac4058500,
> event=TS_EVENT_HTTP_CONTINUE) at InkAPI.cc:5548
> #16 0x00002afa68cb19b7 in yca_cache_access_control (contp=<value optimized
> out>, event=<value optimized out>, edata=<value optimized out>) at
> yca_plugin.cpp:747
> #17 yca_auth_plugin_local_handler (contp=<value optimized out>, event=<value
> optimized out>, edata=<value optimized out>) at yca_plugin.cpp:955
> #18 0x0000000000582ed4 in HttpSM::state_api_callout (this=0x2afac4058500,
> event=<value optimized out>, data=<value optimized out>) at HttpSM.cc:1358
> #19 0x000000000058d06a in HttpSM::set_next_state (this=0x2afac4058500) at
> HttpSM.cc:6790
> #20 0x000000000058ba72 in HttpSM::handle_api_return (this=0x2afac4058500) at
> HttpSM.cc:1501
> #21 0x0000000000583070 in HttpSM::state_api_callout (this=0x2afac4058500,
> event=0, data=0x0) at HttpSM.cc:1433
> #22 0x000000000058d06a in HttpSM::set_next_state (this=0x2afac4058500) at
> HttpSM.cc:6790
> #23 0x000000000058533e in HttpSM::state_cache_open_read (this=0x2afac4058500,
> event=1102, data=0x2afbd4006e80) at HttpSM.cc:2423
> #24 0x0000000000587de8 in HttpSM::main_handler (this=0x2afac4058500,
> event=1102, data=0x2afbd4006e80) at HttpSM.cc:2482
> #25 0x0000000000566e12 in handleEvent (this=0x2afac4059ed8, event=<value
> optimized out>, data=0x2afbd4006e80) at
> ../../iocore/eventsystem/I_Continuation.h:146
> #26 HttpCacheSM::state_cache_open_read (this=0x2afac4059ed8, event=<value
> optimized out>, data=0x2afbd4006e80) at HttpCacheSM.cc:118
> #27 0x0000000000653453 in handleEvent (this=0x2afbd4006e80, event=<value
> optimized out>) at ../../iocore/eventsystem/I_Continuation.h:146
> #28 CacheVC::callcont (this=0x2afbd4006e80, event=<value optimized out>) at
> ../../iocore/cache/P_CacheInternal.h:657
> #29 0x00000000006be9af in CacheVC::openReadStartHead (this=0x2afbd4006e80,
> event=3900, e=0x0) at CacheRead.cc:1206
> #30 0x000000000069d40d in handleEvent (this=0x2afbd4006e80, event=<value
> optimized out>, e=<value optimized out>) at
> ../../iocore/eventsystem/I_Continuation.h:146
> #31 CacheVC::handleReadDone (this=0x2afbd4006e80, event=<value optimized
> out>, e=<value optimized out>) at Cache.cc:2521
> #32 0x00000000006539c5 in handleEvent (this=<value optimized out>,
> event=<value optimized out>, data=<value optimized out>) at
> ../../iocore/eventsystem/I_Continuation.h:146
> #33 AIOCallbackInternal::io_complete (this=<value optimized out>,
> event=<value optimized out>, data=<value optimized out>) at
> ../../iocore/aio/P_AIO.h:123
> #34 0x0000000000714aef in handleEvent (this=0x2afa54b18010, e=0x2afaac01ac90,
> calling_code=1) at I_Continuation.h:146
> #35 EThread::process_event (this=0x2afa54b18010, e=0x2afaac01ac90,
> calling_code=1) at UnixEThread.cc:145
> #36 0x000000000071561b in EThread::execute (this=0x2afa54b18010) at
> UnixEThread.cc:196
> #37 0x0000000000713e9a in spawn_thread_internal (a=0x16c3220) at Thread.cc:88
> #38 0x00002afa4f271851 in start_thread () from /lib64/libpthread.so.0
> #39 0x000000361a0e894d in clone () from /lib64/libc.so.6
> (gdb) print mem
> $1 = (volatile int *) 0x70d87d9700000008
> (gdb) print *mem
> Cannot access memory at address 0x70d87d9700000008
> (gdb) up
> #1 refcount_inc (this=0x2afb60113010, inherit_from=0x2afaa824e688) at
> ../../lib/ts/Ptr.h:279
> 279 ../../lib/ts/Ptr.h: No such file or directory.
> in ../../lib/ts/Ptr.h
> (gdb) print this
> $2 = (RefCountObj * const) 0x70d87d9700000000
> (gdb) print *this
> Cannot access memory at address 0x70d87d9700000000
> (gdb) up
> #2 operator= (this=0x2afb60113010, inherit_from=0x2afaa824e688) at
> ../../lib/ts/Ptr.h:408
> 408 in ../../lib/ts/Ptr.h
> (gdb) print this
> $3 = (Ptr<RefCountObj> * const) 0x2afb60113060
> (gdb) print *this
> $4 = {m_ptr = 0x70d87d9700000000}
> (gdb) up
> #3 attach_str_heap (this=0x2afb60113010, inherit_from=0x2afaa824e688) at
> HdrHeap.cc:1000
> 1000 HdrHeap.cc: No such file or directory.
> in HdrHeap.cc
> (gdb) print this
> $5 = (HdrHeap * const) 0x2afb60113010
> (gdb) print *this
> $6 = {m_magic = 2882404077, m_free_start = 0x2afb60113360 "\002p",
> m_data_start = 0x2afb601130e0 "\003\060", m_size = 2048, m_writeable = true,
> m_next = 0x0, m_free_size = 1200,
> m_read_write_heap = {m_ptr = 0x0}, m_ronly_heap = {{m_ref_count_ptr =
> {m_ptr = 0x2afa6c0019a0},
> m_heap_start = 0x2afaa824e990 "HTTP/1.1 200 OK\r\nDate: Mon, 05 May
> 2014 19:02:05 GMT\r\nP3P: policyref=\"http://info.yahoo.com/w3c/p3p.xml\";,
> CP=\"CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi
> SAMi OTRi UNRi PUBi "..., m_heap_len = 539, m_locked = false},
> {m_ref_count_ptr = {m_ptr = 0x70d87d9700000000}, m_heap_start = 0x0,
> m_heap_len = 0,
> m_locked = false}, {m_ref_count_ptr = {m_ptr = 0x0}, m_heap_start =
> 0x0, m_heap_len = 0, m_locked = false}, {m_ref_count_ptr = {m_ptr = 0x0},
> m_heap_start = 0x0, m_heap_len = 0,
> m_locked = false}, {m_ref_count_ptr = {m_ptr = 0x0}, m_heap_start =
> 0x0, m_heap_len = 0, m_locked = false}, {m_ref_count_ptr = {m_ptr = 0x0},
> m_heap_start = 0x0, m_heap_len = 0,
> m_locked = false}}, m_lost_string_space = 0}
> (gdb)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
