[ https://issues.apache.org/jira/browse/TS-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-4742: ------------------------------ Fix Version/s: sometime > FATAL: ClusterCache.cc:1759: failed assert `tc->tunnel_cont == tc` > ------------------------------------------------------------------ > > Key: TS-4742 > URL: https://issues.apache.org/jira/browse/TS-4742 > Project: Traffic Server > Issue Type: Bug > Components: Clustering > Reporter: Paul Stengel > Fix For: sometime > > > We're running TS in a two-server cluster right now. The clustering works well > and is functioning as you'd expect. However, every so often, TS crashes with > the following backtrace: > {noformat} > FATAL: ClusterCache.cc:1759: failed assert `tc->tunnel_cont == tc` > traffic_server: using root directory '/usr' > traffic_server: Aborted (Signal sent by tkill() 13260 114) > traffic_server - STACK TRACE: > /usr/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0xa0)[0x55bbaed08f00] > /lib/x86_64-linux-gnu/libpthread.so.0(+0x113d0)[0x2b45c2ff13d0] > /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x2b45c3cd7418] > /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x2b45c3cd901a] > /usr/lib/trafficserver/libtsutil.so.6(+0x2430f)[0x2b45c1c8f30f] > /usr/lib/trafficserver/libtsutil.so.6(+0x243a9)[0x2b45c1c8f3a9] > /usr/lib/trafficserver/libtsutil.so.6(+0x22595)[0x2b45c1c8d595] > /usr/bin/traffic_server(+0x2ee7e8)[0x55bbaef207e8] > /usr/bin/traffic_server(_ZN12OneWayTunnel10startEventEiPv+0x16f)[0x55bbaeecd9df] > /usr/bin/traffic_server(_ZN14ClusterHandler32cluster_signal_and_update_lockedEiP18ClusterVConnectionP17ClusterVConnState+0x2f)[0x55bbaef05abf] > /usr/bin/traffic_server(_ZN14ClusterHandler20valid_for_data_writeEP18ClusterVConnection+0x8fa)[0x55bbaeefff1a] > /usr/bin/traffic_server(_ZN14ClusterHandler23build_write_descriptorsEv+0x171)[0x55bbaef00091] > /usr/bin/traffic_server(_ZN14ClusterHandler13process_writeElb+0x578)[0x55bbaef00868] > /usr/bin/traffic_server(_ZN14ClusterHandler16mainClusterEventEiP5Event+0x195)[0x55bbaef00ae5] > /usr/bin/traffic_server(_ZN12ClusterState10IOCompleteEv+0xc7)[0x55bbaef04be7] > /usr/bin/traffic_server(_ZN12ClusterState16doIO_write_eventEiPv+0x119)[0x55bbaef04fd9] > /usr/bin/traffic_server(_Z15write_to_net_ioP10NetHandlerP18UnixNetVConnectionP7EThread+0x1191)[0x55bbaefabfd1] > /usr/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x328)[0x55bbaef98ad8] > /usr/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x92)[0x55bbaefdd362] > /usr/bin/traffic_server(_ZN7EThread7executeEv+0x679)[0x55bbaefde079] > /usr/bin/traffic_server(+0x3aae49)[0x55bbaefdce49] > /lib/x86_64-linux-gnu/libpthread.so.0(+0x76fa)[0x2b45c2fe76fa] > /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2b45c3da8b5d] > {noformat} > We've not found a way to reproduce the crash. Here are the diags.log entries > that show when the server starts back up (indicating a crash and recovery): > {noformat} > [Aug 11 07:23:46.245] Server {0x2afc9cf458c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 11:14:10.272] Server {0x2ba545a318c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 12:08:51.188] Server {0x2b887a0228c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 14:57:23.900] Server {0x2aad8422b8c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 14:58:17.844] Server {0x2b66aad508c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 15:08:38.752] Server {0x2b09a50cb8c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 15:08:51.001] Server {0x2aee8a7c88c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 15:10:46.796] Server {0x2ace4544e8c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 15:25:27.599] Server {0x2b90b94788c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 15:26:13.642] Server {0x2b814d2088c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:03:31.729] Server {0x2b259ff968c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:03:44.057] Server {0x2b1cf99198c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:15:27.307] Server {0x2b2333b1d8c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:39:56.205] Server {0x2b92f018f8c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:40:07.535] Server {0x2aad138ab8c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:40:17.971] Server {0x2b94640748c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 16:41:07.280] Server {0x2b45c1a798c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 17:01:28.328] Server {0x2adec40e68c0} NOTE: <Main.cc:1902 (main)> > traffic server running > [Aug 11 17:06:17.853] Server {0x2aef987758c0} NOTE: <Main.cc:1902 (main)> > traffic server running > {noformat} > {noformat} > root@gateway-us-east-1:/var/log/trafficserver$ traffic_server -V > traffic_server: using root directory '/usr' > Apache Traffic Server - traffic_server - 6.2.0 - (build # 080418 on Aug 4 > 2016 at 18:36:55) > {noformat} > {noformat} > root@gateway-us-east-1:~$ cat /usr/lib/os-release > NAME="Ubuntu" > VERSION="16.04.1 LTS (Xenial Xerus)" > ID=ubuntu > ID_LIKE=debian > PRETTY_NAME="Ubuntu 16.04.1 LTS" > VERSION_ID="16.04" > HOME_URL="http://www.ubuntu.com/" > SUPPORT_URL="http://help.ubuntu.com/" > BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" > UBUNTU_CODENAME=xenial > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)