[
https://issues.apache.org/jira/browse/TS-4254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383518#comment-15383518
]
tianjidong edited comment on TS-4254 at 7/19/16 3:12 AM:
---------------------------------------------------------
Er, linux-native-aio is disabled in my building (V5.3.2), but this issue
still happend.
Build Log:
configure: Using epoll event interface
checking for 128bit CAS support... yes
checking for library containing cap_set_proc... no
checking whether to enable Linux native AIO... no
error Log:
[E. Mgmt] log ==> [TrafficManager] using root directory '/usr/local'
traffic_server: using root directory '/usr/local'
FATAL: HttpTransactHeaders.cc:180: failed assert `!new_hdr->valid()`
traffic_server: Aborted (Signal sent by tkill() 30262 99)
traffic_server - STACK TRACE:
/usr/local/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0xc3)[0x50b9d0]
/lib64/libpthread.so.0(+0xf100)[0x2b3724e57100]
/lib64/libc.so.6(gsignal+0x37)[0x2b3725e275f7]
/lib64/libc.so.6(abort+0x148)[0x2b3725e28ce8]
/usr/local/lib/libtsutil.so.5(_Z12ink_fatal_vaPKcP13__va_list_tag+0x0)[0x2b3722b09c3d]
/usr/local/lib/libtsutil.so.5(_Z9ink_fatalPKcz+0x0)[0x2b3722b09cf6]
/usr/local/lib/libtsutil.so.5(_Z10ink_pfatalPKcz+0x0)[0x2b3722b09d95]
/usr/local/lib/libtsutil.so.5(+0x3c756)[0x2b3722b07756]
/usr/local/bin/traffic_server(_ZN19HttpTransactHeaders18copy_header_fieldsEP7HTTPHdrS1_bl+0x73)[0x632be1]
/usr/local/bin/traffic_server(_ZN12HttpTransact13build_requestEPNS_5StateEP7HTTPHdrS3_11HTTPVersion+0x128)[0x628a7e]
/usr/local/bin/traffic_server(_ZN12HttpTransact23HandleCacheOpenReadMissEPNS_5StateE+0x69a)[0x6164f2]
/usr/local/bin/traffic_server(_ZN6HttpSM32call_transact_and_set_next_stateEPFvPN12HttpTransact5StateEE+0x71)[0x5fb247]
/usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x102)[0x5e7452]
/usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x875)[0x5e7237]
/usr/local/bin/traffic_server(_ZN6HttpSM23do_api_callout_internalEv+0x1b7)[0x5f4453]
/usr/local/bin/traffic_server(_ZN6HttpSM14do_api_calloutEv+0x27)[0x601b21]
/usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x66)[0x5fb3f2]
/usr/local/bin/traffic_server(_ZN6HttpSM32call_transact_and_set_next_stateEPFvPN12HttpTransact5StateEE+0x1ae)[0x5fb384]
/usr/local/bin/traffic_server(_ZN6HttpSM19state_hostdb_lookupEiPv+0x1c6)[0x5e9a9e]
/usr/local/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x33a)[0x5eb21c]
/usr/local/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x72)[0x50eb22]
/usr/local/bin/traffic_server[0x6c6239]
/usr/local/bin/traffic_server(_ZN18HostDBContinuation8dnsEventEiP7HostEnt+0x1347)[0x6cabb9]
/usr/local/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x72)[0x50eb22]
/usr/local/bin/traffic_server(_ZN8DNSEntry9postEventEiP5Event+0xbf)[0x6e1349]
/usr/local/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x72)[0x50eb22]
/usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x136)[0x7a3276]
/usr/local/bin/traffic_server(_ZN7EThread7executeEv+0xd6)[0x7a34d8]
/usr/local/bin/traffic_server[0x7a27c5]
....
was (Author: kketiantian):
Er, linux-native-aio is disabled In my building (V5.3.2), but this this issue
still happend.
Build Log:
configure: Using epoll event interface
checking for 128bit CAS support... yes
checking for library containing cap_set_proc... no
checking whether to enable Linux native AIO... no
error Log:
[E. Mgmt] log ==> [TrafficManager] using root directory '/usr/local'
traffic_server: using root directory '/usr/local'
FATAL: HttpTransactHeaders.cc:180: failed assert `!new_hdr->valid()`
traffic_server: Aborted (Signal sent by tkill() 30262 99)
traffic_server - STACK TRACE:
/usr/local/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0xc3)[0x50b9d0]
/lib64/libpthread.so.0(+0xf100)[0x2b3724e57100]
/lib64/libc.so.6(gsignal+0x37)[0x2b3725e275f7]
/lib64/libc.so.6(abort+0x148)[0x2b3725e28ce8]
/usr/local/lib/libtsutil.so.5(_Z12ink_fatal_vaPKcP13__va_list_tag+0x0)[0x2b3722b09c3d]
/usr/local/lib/libtsutil.so.5(_Z9ink_fatalPKcz+0x0)[0x2b3722b09cf6]
/usr/local/lib/libtsutil.so.5(_Z10ink_pfatalPKcz+0x0)[0x2b3722b09d95]
/usr/local/lib/libtsutil.so.5(+0x3c756)[0x2b3722b07756]
/usr/local/bin/traffic_server(_ZN19HttpTransactHeaders18copy_header_fieldsEP7HTTPHdrS1_bl+0x73)[0x632be1]
/usr/local/bin/traffic_server(_ZN12HttpTransact13build_requestEPNS_5StateEP7HTTPHdrS3_11HTTPVersion+0x128)[0x628a7e]
/usr/local/bin/traffic_server(_ZN12HttpTransact23HandleCacheOpenReadMissEPNS_5StateE+0x69a)[0x6164f2]
/usr/local/bin/traffic_server(_ZN6HttpSM32call_transact_and_set_next_stateEPFvPN12HttpTransact5StateEE+0x71)[0x5fb247]
/usr/local/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x102)[0x5e7452]
/usr/local/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x875)[0x5e7237]
/usr/local/bin/traffic_server(_ZN6HttpSM23do_api_callout_internalEv+0x1b7)[0x5f4453]
/usr/local/bin/traffic_server(_ZN6HttpSM14do_api_calloutEv+0x27)[0x601b21]
/usr/local/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x66)[0x5fb3f2]
/usr/local/bin/traffic_server(_ZN6HttpSM32call_transact_and_set_next_stateEPFvPN12HttpTransact5StateEE+0x1ae)[0x5fb384]
/usr/local/bin/traffic_server(_ZN6HttpSM19state_hostdb_lookupEiPv+0x1c6)[0x5e9a9e]
/usr/local/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x33a)[0x5eb21c]
/usr/local/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x72)[0x50eb22]
/usr/local/bin/traffic_server[0x6c6239]
/usr/local/bin/traffic_server(_ZN18HostDBContinuation8dnsEventEiP7HostEnt+0x1347)[0x6cabb9]
/usr/local/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x72)[0x50eb22]
/usr/local/bin/traffic_server(_ZN8DNSEntry9postEventEiP5Event+0xbf)[0x6e1349]
/usr/local/bin/traffic_server(_ZN12Continuation11handleEventEiPv+0x72)[0x50eb22]
/usr/local/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x136)[0x7a3276]
/usr/local/bin/traffic_server(_ZN7EThread7executeEv+0xd6)[0x7a34d8]
/usr/local/bin/traffic_server[0x7a27c5]
....
> abort failure when enable --enable-linux-native-aio in production host
> ----------------------------------------------------------------------
>
> Key: TS-4254
> URL: https://issues.apache.org/jira/browse/TS-4254
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 5.3.2
> Reporter: taoyunxing
> Labels: AIO
> Fix For: sometime
>
>
> ATS: 5.3.2 , CentOS 6.6
> compile:
> yum -y libaio libaio-devel
> ./configure --prefix=/opt/ats --enable-linux-native-aio && make -j 4 && make
> install -j 4
> I found a assert errors when ats use linux native aio, occur many times in 12
> hours!
> {code}
> FATAL: HttpTransactHeaders.cc:180: failed assert `!new_hdr->valid()`
> traffic_server: Aborted (Signal sent by tkill() 20759 495)traffic_server -
> STACK TRACE:
> /opt/ats/bin/traffic_server(crash_logger_invoke(int, siginfo*,
> void*)+0xc3)[0x5036f6]
> /lib64/libc.so.6[0x3f1f2326a0]
> /lib64/libc.so.6(gsignal+0x35)[0x3f1f232625]
> /lib64/libc.so.6(abort+0x175)[0x3f1f233e05]
> /opt/ats/lib/libtsutil.so.5(ink_fatal_va(char const*,
> __va_list_tag*)+0x0)[0x2b6a5013a37d]
> /opt/ats/lib/libtsutil.so.5(ink_fatal(char const*, ...)+0x0)[0x2b6a5013a434]
> /opt/ats/lib/libtsutil.so.5(ink_pfatal(char const*, ...)+0x0)[0x2b6a5013a4f9]
> /opt/ats/lib/libtsutil.so.5(+0x39fa2)[0x2b6a50137fa2]
> /opt/ats/bin/traffic_server(HttpTransactHeaders::copy_header_fields(HTTPHdr*,
> HTTPHdr*, bool, long)+0x73)[0x62eddb]
> /opt/ats/bin/traffic_server(HttpTransact::build_request(HttpTransact::State*,
> HTTPHdr*, HTTPHdr*, HTTPVersion)+0x12b)[0x624c9b]
> /opt/ats/bin/traffic_server(HttpTransact::HandleCacheOpenReadMiss(HttpTransact::State*)+0x6a4)[0x612e72]
> /opt/ats/bin/traffic_server(HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*))+0x71)[0x5f7db1]
> /opt/ats/bin/traffic_server(HttpSM::handle_api_return()+0xfa)[0x5e3c82]
> /opt/ats/bin/traffic_server(HttpSM::do_api_callout()+0x40)[0x5fe7c6]
> /opt/ats/bin/traffic_server(HttpSM::set_next_state()+0x5e)[0x5f7f54]
> /opt/ats/bin/traffic_server(HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*))+0x1ae)[0x5f7eee]
> /opt/ats/bin/traffic_server(HttpSM::do_hostdb_lookup()+0x71e)[0x5ed4b6]
> /opt/ats/bin/traffic_server(HttpSM::set_next_state()+0xb34)[0x5f8a2a]
> /opt/ats/bin/traffic_server(HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*))+0x1ae)[0x5f7eee]
> /opt/ats/bin/traffic_server(HttpSM::handle_server_setup_error(int,
> void*)+0x65a)[0x5f227e]
> /opt/ats/bin/traffic_server(HttpSM::state_send_server_request_header(int,
> void*)+0x57c)[0x5e53b8]
> /opt/ats/bin/traffic_server(HttpSM::main_handler(int, void*)+0x270)[0x5e79a4]
> /opt/ats/bin/traffic_server(Continuation::handleEvent(int,
> void*)+0x6c)[0x506956]
> /opt/ats/bin/traffic_server[0x779c97]
> /opt/ats/bin/traffic_server(UnixNetVConnection::mainEvent(int,
> Event*)+0x4de)[0x77d584]
> /opt/ats/bin/traffic_server(Continuation::handleEvent(int,
> void*)+0x6c)[0x506956]
> /opt/ats/bin/traffic_server(InactivityCop::check_inactivity(int,
> Event*)+0x4d3)[0x773ed9]
> /opt/ats/bin/traffic_server(Continuation::handleEvent(int,
> void*)+0x6c)[0x506956]
> /opt/ats/bin/traffic_server(EThread::process_event(Event*,
> int)+0x136)[0x79bee6]
> /opt/ats/bin/traffic_server(EThread::execute()+0x289)[0x79c2f5]
> /opt/ats/bin/traffic_server[0x79b3ec]
> /lib64/libpthread.so.0[0x3f1f6079d1]
> /lib64/libc.so.6(clone+0x6d)[0x3f1f2e89dd]
> {code}
> or
> {code}
> Thread 20825, [ET_NET 41]:
> 0 0x00000000005036cb crash_logger_invoke(int, siginfo*, void*) + 0x98
> 1 0x0000003f1f2326a0 __restore_rt + (nil)
> 2 0x0000003f1f232625 gsignal + 0x35
> 3 0x0000003f1f233e05 abort + 0x175
> 4 0x00002b6a5013a37d ink_fatal_va(char const*, __va_list_tag*) + (nil)
> 5 0x00002b6a5013a434 ink_fatal(char const*, ...) + (nil)
> 6 0x00002b6a5013a4f9 ink_pfatal(char const*, ...) + (nil)
> 7 0x00002b6a50137fa2 _ink_assert + 0x32
> 8 0x000000000062eddb HttpTransactHeaders::copy_header_fields(HTTPHdr*,
> HTTPHdr*, bool, long) + 0x73
> 9 0x0000000000624c9b HttpTransact::build_request(HttpTransact::State*,
> HTTPHdr*, HTTPHdr*, HTTPVersion) + 0x12b
> 10 0x0000000000612e72
> HttpTransact::HandleCacheOpenReadMiss(HttpTransact::State*) + 0x6a4
> 11 0x00000000005f7db1 HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*)) + 0x71
> 12 0x00000000005e3c82 HttpSM::handle_api_return() + 0xfa
> 13 0x00000000005fe7c6 HttpSM::do_api_callout() + 0x40
> 14 0x00000000005f7f54 HttpSM::set_next_state() + 0x5e
> 15 0x00000000005f7eee HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*)) + 0x1ae
> 16 0x00000000005ed4b6 HttpSM::do_hostdb_lookup() + 0x71e
> 17 0x00000000005f8a2a HttpSM::set_next_state() + 0xb34
> 18 0x00000000005f7eee HttpSM::call_transact_and_set_next_state(void
> (*)(HttpTransact::State*)) + 0x1ae
> 19 0x00000000005f227e HttpSM::handle_server_setup_error(int, void*) + 0x65a
> 20 0x00000000005e53b8 HttpSM::state_send_server_request_header(int, void*)
> + 0x57c
> 21 0x00000000005e79a4 HttpSM::main_handler(int, void*) + 0x270
> 22 0x0000000000506956 Continuation::handleEvent(int, void*) + 0x6c
> 23 0x0000000000779c97 read_signal_and_update(int, UnixNetVConnection*) +
> 0x5a
> 24 0x000000000077d584 UnixNetVConnection::mainEvent(int, Event*) + 0x4de
> 25 0x0000000000506956 Continuation::handleEvent(int, void*) + 0x6c
> 26 0x0000000000773ed9 InactivityCop::check_inactivity(int, Event*) + 0x4d3
> 27 0x0000000000506956 Continuation::handleEvent(int, void*) + 0x6c
> 28 0x000000000079bee6 EThread::process_event(Event*, int) + 0x136
> 29 0x000000000079c2f5 EThread::execute() + 0x289
> 30 0x000000000079b3ec spawn_thread_internal(void*) + 0x75
> 31 0x0000003f1f6079d1 start_thread + 0xd1
> 32 0x0000003f1f2e89dd clone + 0x6d
> 33 0x0000000000000000 0x0 + 0x6d
> {code}
> anyone can give me some advices? thanks in advance !
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
