[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13237651#comment-13237651
]
Otto van der Schaaf commented on TS-857:
i retested with trunk, i cannot reproduce the stack trace i added anymore, so i
think this trace is solved
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.5
Attachments: locking-jp1.patch, ts-857.diff, ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13236722#comment-13236722
]
Otto van der Schaaf commented on TS-857:
another trace, put here as per amc's request:
Program received signal SIGSEGV, Segmentation fault.
ink_atomiclist_push (l=0x100bb0, item=0x11a4f00) at ink_queue.cc:457
457 volatile_void_p *adr_of_next = (volatile_void_p *)
ADDRESS_OF_NEXT(item, l-offset);
(gdb) bt
#0 ink_atomiclist_push (l=0x100bb0, item=0x11a4f00) at ink_queue.cc:457
#1 0x006aab3b in ProtectedQueue::enqueue (this=0x100bb0, e=optimized
out, fast_signal=false) at ProtectedQueue.cc:53
#2 0x00511ff6 in HttpClientSession::do_io_close (this=0x11b89b0,
alerrno=-1) at HttpClientSession.cc:331
#3 0x00522fba in HttpSM::tunnel_handler_ua (this=0x7fffeb726650,
event=103, c=0x7fffeb728270) at HttpSM.cc:3029
#4 0x00566006 in HttpTunnel::consumer_handler (this=0x7fffeb7281c8,
event=103, c=0x7fffeb728270) at HttpTunnel.cc:1232
#5 0x00566800 in HttpTunnel::finish_all_internal (this=0x7fffeb7281c8,
p=0x7fffeb7284b0, chain=false) at HttpTunnel.cc:1359
#6 0x00520680 in local_finish_all (p=0x7fffeb7284b0, this=optimized
out) at HttpTunnel.h:311
#7 HttpSM::tunnel_handler_transform_read (this=0x7fffeb726650, event=104,
p=0x7fffeb7284b0) at HttpSM.cc:3572
#8 0x00565a6c in HttpTunnel::producer_handler (this=0x7fffeb7281c8,
event=104, p=0x7fffeb7284b0) at HttpTunnel.cc:1136
#9 0x00565eb0 in HttpTunnel::main_handler (this=0x7fffeb7281c8,
event=optimized out, data=optimized out) at HttpTunnel.cc:1452
#10 0x004e78c5 in TransformTerminus::handle_event (this=0x7fffe4036998,
event=optimized out, edata=optimized out) at Transform.cc:241
#11 0x006ac450 in handleEvent (data=0x11a4fc0, event=1, this=optimized
out) at I_Continuation.h:146
#12 EThread::process_event (this=0x7474f010, e=0x11a4fc0, calling_code=1)
at UnixEThread.cc:142
#13 0x006ad01e in EThread::execute (this=0x7474f010) at
UnixEThread.cc:234
#14 0x0047d5f2 in main (argc=optimized out, argv=optimized out) at
Main.cc:1939
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.5
Attachments: locking-jp1.patch, ts-857.diff, ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13232159#comment-13232159
]
John Plevyak commented on TS-857:
-
Could the non-locked access be coming from the InactivityCop? See attached
lock patch which handles a race in that code.
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.5
Attachments: ts-857.diff, ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13227664#comment-13227664
]
Zhao Yongming commented on TS-857:
--
we have the codes commit in our tree in:
https://gitorious.org/trafficserver/taobao/commit/c0fdec56bacea139a026412038378d7a8f65a731
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.5
Attachments: ts-857.diff, ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166354#comment-13166354
]
John Plevyak commented on TS-857:
-
So, this patch throws a VC_EVENT_DO_CLOSE event at a NetVC, but it doesn't
record that fact that the event is outstanding? What prevents the NetVC from
being deallocated before the event is processed?
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.3
Attachments: ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166353#comment-13166353
]
John Plevyak commented on TS-857:
-
So, this patch throws a VC_EVENT_DO_CLOSE event at a NetVC, but it doesn't
record that fact that the event is outstanding? What prevents the NetVC from
being deallocated before the event is processed?
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.3
Attachments: ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166513#comment-13166513
]
Alan M. Carroll commented on TS-857:
Where would it record the pending event? The presumption is that the only other
thread that might close it is the one that will process the event.
But the real issue is that sometimes the lock is dropped between the time the
lock try is done and the event is scheduled, leading to scheduling on the null
thread. If you look at TS-934 you can see how that issue is handled there, but
even that's not sufficient because normal VC processing from
NetHandler::mainNetEvent does not lock the VCs, so there isn't any way do
anything safely in this case.
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.3
Attachments: ts-857.diff, ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13148887#comment-13148887
]
Alan M. Carroll commented on TS-857:
I have reviewed the patch and it looks basically correct to me. I had a few
style issues, which I have corrected in my version of the patch.
* I put the new event type in iocore/eventsystem/I_VConnection.h where the
other VConnection events are.
* Added some comments.
* Fixed a broken comment in I_Event.h which misdirected me about schedule_imm
in my earlier work. I found the correct comment in a very old copy of the file
and put it back.
I am currently doing some live testing, but I would like to commit this for
3.1.1.
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Assignee: weijin
Fix For: 3.1.2
Attachments: ts-857.diff
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13 0x006ce6bc in EThread::execute (this=0x2b12c010) at
UnixEThread.cc:262
#14 0x006cd0ee in spawn_thread_internal (a=0x171b58f0) at Thread.cc:88
#15 0x003c33c064a7 in start_thread () from /lib64/libpthread.so.0
#16 0x003c330d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x40e2b790:
rip = 0x2ba641dccdf3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2ba641dccef1
called by frame at 0x40e2bbe0
source language c++.
Arglist at 0x40e2b770, args: stack=value optimized out, len=value
optimized out, signalhandler_frame=value optimized out
Locals at 0x40e2b770, Previous frame's sp is 0x40e2b790
Saved registers:
rbx at 0x40e2b778, rbp at 0x40e2b780, rip at 0x40e2b788
(gdb)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13057723#comment-13057723
]
AdunGaos commented on TS-857:
-
code
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out) at ink_stack_trace.cc:68
68 fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out) at ink_stack_trace.cc:68
#1 0x2b9a63a34ef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004ddc10 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006990d9 in UnixNetVConnection::do_io_close (this=0x2aaab0052f00,
alerrno=value optimized out) at ../../iocore/eventsystem/I_Lock.h:297
#5 0x00513f5a in HttpClientSession::do_io_close (this=0x78046f0,
alerrno=-1) at HttpClientSession.cc:310
#6 0x0052588c in HttpSM::tunnel_handler_ua (this=0x2fcd8c40,
event=103, c=0x2fcda968) at HttpSM.cc:3027
#7 0x0056f9dc in HttpTunnel::consumer_handler (this=0x2fcda930,
event=103, c=0x2fcda968) at HttpTunnel.cc:1232
#8 0x0057095e in HttpTunnel::finish_all_internal (this=0x2fcda930,
p=0x2fcdab28, chain=false) at HttpTunnel.cc:1359
#9 0x00526e07 in HttpSM::tunnel_handler_server (this=0x2fcd8c40,
event=104, p=0x2fcdab28) at HttpTunnel.h:311
#10 0x0056eb21 in HttpTunnel::producer_handler (this=0x2fcda930,
event=104, p=0x2fcdab28) at HttpTunnel.cc:1136
#11 0x0056fca0 in HttpTunnel::main_handler (this=0x2fcda930,
event=1106414096, data=value optimized out) at HttpTunnel.cc:1452
#12 0x0069f097 in read_from_net (nh=0x2b22e688, vc=0x2aaab04fef40,
thread=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:146
#13 0x006940e1 in NetHandler::mainNetEvent (this=0x2b22e688,
event=value optimized out, e=0x2b9b601c) at UnixNet.cc:389
#14 0x006c4fdf in EThread::process_event (this=0x2b22d010,
e=0x693bac0, calling_code=5) at I_Continuation.h:146
#15 0x006c58ec in EThread::execute (this=0x2b22d010) at
UnixEThread.cc:262
#16 0x006c431e in spawn_thread_internal (a=0x692f5d0) at Thread.cc:88
#17 0x0031060064a7 in start_thread () from /lib64/libpthread.so.0
#18 0x0031054d3c2d in clone () from /lib64/libc.so.6
(gdb) info f
Stack level 0, frame at 0x41f22560:
rip = 0x2b9a63a34df3 in ink_stack_trace_get(void**, int, int)
(ink_stack_trace.cc:68); saved rip 0x2b9a63a34ef1
called by frame at 0x41f229b0
source language c++.
Arglist at 0x41f22540, args: stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out
Locals at 0x41f22540, Previous frame's sp is 0x41f22560
Saved registers:
rbx at 0x41f22548, rbp at 0x41f22550, rip at 0x41f22558
/code
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in
[jira] [Commented] (TS-857) Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close - UnixNetVConnection::do_io_close
[
https://issues.apache.org/jira/browse/TS-857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13058160#comment-13058160
]
mohan_zl commented on TS-857:
-
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out) at ink_stack_trace.cc:68
68 fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out) at ink_stack_trace.cc:68
#1 0x0038ff016ef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004ddc10 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006990d9 in UnixNetVConnection::do_io_close (this=0x20efd5e0,
alerrno=value optimized out) at ../../iocore/eventsystem/I_Lock.h:297
#5 0x00513f5a in HttpClientSession::do_io_close (this=0x2aaab405c420,
alerrno=-1) at HttpClientSession.cc:310
#6 0x0052588c in HttpSM::tunnel_handler_ua (this=0x2aaab87fc4f0,
event=103, c=0x2aaab87fe218) at HttpSM.cc:3027
#7 0x0056f9dc in HttpTunnel::consumer_handler (this=0x2aaab87fe1e0,
event=103, c=0x2aaab87fe218) at HttpTunnel.cc:1232
#8 0x0057095e in HttpTunnel::finish_all_internal (this=0x2aaab87fe1e0,
p=0x2aaab87fe3d8, chain=false) at HttpTunnel.cc:1359
#9 0x00526e07 in HttpSM::tunnel_handler_server (this=0x2aaab87fc4f0,
event=104, p=0x2aaab87fe3d8) at HttpTunnel.h:311
#10 0x0056eb21 in HttpTunnel::producer_handler (this=0x2aaab87fe1e0,
event=104, p=0x2aaab87fe3d8) at HttpTunnel.cc:1136
#11 0x0056fca0 in HttpTunnel::main_handler (this=0x2aaab87fe1e0,
event=-1678707408, data=value optimized out) at HttpTunnel.cc:1452
#12 0x0069f097 in read_from_net (nh=0x2adfe688, vc=0x20eb4c40,
thread=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:146
#13 0x006940e1 in NetHandler::mainNetEvent (this=0x2adfe688,
event=value optimized out, e=0x2b706028) at UnixNet.cc:389
#14 0x006c4fdf in EThread::process_event (this=0x2adfd010,
e=0x1fc96f60, calling_code=5) at I_Continuation.h:146
#15 0x006c58ec in EThread::execute (this=0x2adfd010) at
UnixEThread.cc:262
#16 0x004be112 in main (argc=value optimized out, argv=value
optimized out) at Main.cc:1957
Crash Report: HttpTunnel::chain_abort_all - HttpServerSession::do_io_close
- UnixNetVConnection::do_io_close
--
Key: TS-857
URL: https://issues.apache.org/jira/browse/TS-857
Project: Traffic Server
Issue Type: Bug
Components: HTTP, Network
Affects Versions: 3.1.0
Environment: in my branch that is something same as 3.0.x
Reporter: Zhao Yongming
Fix For: 3.1.0
here is the bt from the crash, some of the information is missing due to we
have not enable the --enable-debug configure options.
{code}
[New process 7532]
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
68fp = (void **) (*fp);
(gdb) bt
#0 ink_stack_trace_get (stack=value optimized out, len=value optimized
out, signalhandler_frame=value optimized out)
at ink_stack_trace.cc:68
#1 0x2ba641dccef1 in ink_stack_trace_dump (sighandler_frame=value
optimized out) at ink_stack_trace.cc:114
#2 0x004df020 in signal_handler (sig=value optimized out) at
signals.cc:225
#3 signal handler called
#4 0x006a1ea9 in UnixNetVConnection::do_io_close (this=0x1cc9bd20,
alerrno=value optimized out)
at ../../iocore/eventsystem/I_Lock.h:297
#5 0x0051f1d0 in HttpServerSession::do_io_close
(this=0x2aaab0042c80, alerrno=20600) at HttpServerSession.cc:127
#6 0x0056d1e9 in HttpTunnel::chain_abort_all (this=0x2aabeeffdd70,
p=0x2aabeeffdf68) at HttpTunnel.cc:1300
#7 0x005269ca in HttpSM::tunnel_handler_ua (this=0x2aabeeffc070,
event=104, c=0x2aabeeffdda8) at HttpSM.cc:2987
#8 0x00571dfc in HttpTunnel::consumer_handler (this=0x2aabeeffdd70,
event=104, c=0x2aabeeffdda8) at HttpTunnel.cc:1232
#9 0x00572032 in HttpTunnel::main_handler (this=0x2aabeeffdd70,
event=1088608784, data=value optimized out)
at HttpTunnel.cc:1456
#10 0x006a6307 in write_to_net_io (nh=0x2b12d688, vc=0x1cc876e0,
thread=value optimized out)
at ../../iocore/eventsystem/I_Continuation.h:146
#11 0x0069ce97 in NetHandler::mainNetEvent (this=0x2b12d688,
event=value optimized out, e=0x171c1ed0) at UnixNet.cc:405
#12 0x006cddaf in EThread::process_event (this=0x2b12c010,
e=0x171c1ed0, calling_code=5) at I_Continuation.h:146
#13
