[
https://issues.apache.org/jira/browse/PROTON-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467486#comment-16467486
]
Cliff Jansen commented on PROTON-1842:
--------------------------------------
This test case is quite devilish. Thank-you.
The above fix is necessary but the test case exposes more problems.
The epoll callbacks for socket IO are "mushy". The epoll proactor handles this
quite well (except at tear down as this JIRA has pointed out).
The proactor regularly flips between "wake me when there is socket data to
read" and "wake me when I can read OR write". On any transition, even with
EPOLLONESHOT, it is not possible to know if one or two threads might be awoken,
and if two are, which will get the context lock first.
I am seeing the following calls to pconnection process where the first two are
sequential and the latter two obviously overlap:
event = RW .. rearm(RW) .. wake self
inbound_wake .. rearm(R)
event = R .. segfault on NULL or assert fail on closed fd
event = RW .. begin close .. cleanup .. self delete
> [c] Dispatch/Proton crashes when opening/closing connections
> ------------------------------------------------------------
>
> Key: PROTON-1842
> URL: https://issues.apache.org/jira/browse/PROTON-1842
> Project: Qpid Proton
> Issue Type: Bug
> Components: proton-c
> Affects Versions: proton-c-0.22.0
> Reporter: Chuck Rolke
> Priority: Major
> Attachments: helloworld.cpp
>
>
> Using proton cpp example code that is modified to open and close connections
> by the thousands in the main loop and having the event loop short circuit any
> messaging with:
> {{ void on_connection_open(proton::connection& c) {}}
> {{ c.close();}}
> {{ }}}
> and then directing this client example to a dispatch router 1.1.0. Eventually
> (after 100,000 to 1,000,000 connection open/closes) the router crashes with:
> {{qdrouterd: /home/chug/git/qpid-proton/c/src/proactor/epoll.c:466:
> wake_pop_front: Assertion `p->wakes_in_progress' failed.}}
> and with:
> {{qdrouterd: /home/chug/git/qpid-proton/c/src/proactor/epoll.c:2014:
> proactor_do_epoll: Assertion `ee->type == PCONNECTION_TIMER' failed.}}
> This issue seems to happen only with qpid-dispatch accepting the open/close
> event stream. Proton cpp example _server_direct_ and c example _direct_ work
> properly with the same open/close event stream mounting into the 10s of
> millions of connections.
> A core dump backtrace with the PCONNECTION_TIMER failure reads as:
> {{(gdb) bt}}
> {{#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51}}
> {{#1 0x00007f795c712c41 in __GI_abort () at abort.c:79}}
> {{#2 0x00007f795c709f7a in __assert_fail_base (fmt=0x7f795c85a260
> "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
> assertion=assertion@entry=0x7f795d72e15a "ee->type == PCONNECTION_TIMER", }}
> {{ file=file@entry=0x7f795d72de98
> "/home/chug/git/qpid-proton/c/src/proactor/epoll.c", line=line@entry=2014, }}
> {{ function=function@entry=0x7f795d72e320 <__PRETTY_FUNCTION__.6307>
> "proactor_do_epoll") at assert.c:92}}
> {{#3 0x00007f795c709ff2 in __GI___assert_fail (assertion=0x7f795d72e15a
> "ee->type == PCONNECTION_TIMER", file=0x7f795d72de98
> "/home/chug/git/qpid-proton/c/src/proactor/epoll.c", line=2014, }}
> {{ function=0x7f795d72e320 <__PRETTY_FUNCTION__.6307> "proactor_do_epoll")
> at assert.c:101}}
> {{#4 0x00007f795d72d29f in proactor_do_epoll (p=0x26b7310, can_block=true)
> at /home/chug/git/qpid-proton/c/src/proactor/epoll.c:2014}}
> {{#5 0x00007f795d72d30e in pn_proactor_wait (p=0x26b7310) at
> /home/chug/git/qpid-proton/c/src/proactor/epoll.c:2030}}
> {{#6 0x00007f795dbe89ad in thread_run (arg=0x26be750) at
> /home/chug/git/qpid-dispatch/src/server.c:946}}
> {{#7 0x00007f795d50e50b in start_thread (arg=0x7f794f486700) at
> pthread_create.c:465}}
> {{#8 0x00007f795c7d216f in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:95}}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
