[
https://issues.apache.org/jira/browse/TS-4902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535992#comment-15535992
]
Susan Hinrichs commented on TS-4902:
------------------------------------
Saw another crash in this area while testing fix for TS-4813.
{code}
#0 0x0000003231032625 in raise () from /lib64/libc.so.6
#1 0x0000003231033d8d in abort () from /lib64/libc.so.6
#2 0x00002b49eb4b9d8d in ink_abort (message_format=0x2b49eb4ccbdc "%s:%d:
failed assertion `%s`") at ink_error.cc:79
#3 0x00002b49eb4b7586 in _ink_assert (expression=0x805adf
"client_streams_in_count > 0", file=0x805688 "Http2ConnectionState.cc",
line=1005) at ink_assert.cc:37
#4 0x000000000065adeb in Http2ConnectionState::delete_stream
(this=0x2b4a34a5c190, stream=0x2aac54a69c40) at Http2ConnectionState.cc:1005
#5 0x000000000065be03 in Http2ConnectionState::send_data_frames
(this=0x2b4a34a5c190, stream=0x2aac54a69c40)
at Http2ConnectionState.cc:1191
#6 0x000000000064b57c in Http2Stream::do_io_close (this=0x2aac54a69c40) at
Http2Stream.cc:330
#7 0x00000000005edb61 in HttpSM::tunnel_handler_ua (this=0x2b4a442dcb40,
event=103, c=0x2b4a442ddeb8) at HttpSM.cc:3320
#8 0x00000000006439d5 in HttpTunnel::consumer_handler (this=0x2b4a442dde70,
event=103, c=0x2b4a442ddeb8) at HttpTunnel.cc:1392
#9 0x00000000006442d1 in HttpTunnel::main_handler (this=0x2b4a442dde70,
event=103, data=0x2aac54a69fd0) at HttpTunnel.cc:1647
#10 0x00000000005145dc in Continuation::handleEvent (this=0x2b4a442dde70,
event=103, data=0x2aac54a69fd0)
at ../iocore/eventsystem/I_Continuation.h:153
#11 0x000000000064a8ea in Http2Stream::main_event_handler (this=0x2aac54a69c40,
event=103, edata=0x2b4a07168100) at Http2Stream.cc:88
#12 0x00000000005145dc in Continuation::handleEvent (this=0x2aac54a69c40,
event=103, data=0x2b4a07168100)
at ../iocore/eventsystem/I_Continuation.h:153
#13 0x00000000007bc9be in EThread::process_event (this=0x2b49ed3ee010,
e=0x2b4a07168100, calling_code=103) at UnixEThread.cc:143
#14 0x00000000007bcc2d in EThread::execute (this=0x2b49ed3ee010) at
UnixEThread.cc:197
#15 0x00000000007bbfd2 in spawn_thread_internal (a=0x15efb60) at Thread.cc:84
#16 0x00002b49ec20caa1 in start_thread () from /lib64/libpthread.so.0
#17 0x00000032310e893d in clone () from /lib64/libc.so.6
{code}
Digging in at frame 4, stream_list.head is null, but stream->link.prev is
non-NULL. stream->link.prev->link.next == stream. But they seem to have been
lost from the Http2ConnectionState.stream_list
> Http2ConnectionState::stream_list gets in bad state
> ---------------------------------------------------
>
> Key: TS-4902
> URL: https://issues.apache.org/jira/browse/TS-4902
> Project: Traffic Server
> Issue Type: Bug
> Components: HTTP/2
> Reporter: Susan Hinrichs
>
> Saw this in one of my runs while debugging TS-4900 and TS-4813. It might
> have been due to my attempted fix, but I don't think so.
> I left current master running in production for an hour. When I returned,
> CPU utilization had jumped from 70% to 400%. perf top showed that the
> majority of time was being spent in Http2ConnectionState::restart_streams.
> Connected with debugger and two threads were spinning in that method. In
> each case this->stream_list.head->link.next ==
> this->stream_list.head->link.prev and this->thread_list.head->link.next !=
> NULL.
> I assume that was are missing some locking and the stream_list is being
> manipulated by two threads in parallel.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
