Closing of a federation link done on management thread, not links own io thread
-------------------------------------------------------------------------------
Key: QPID-2046
URL: https://issues.apache.org/jira/browse/QPID-2046
Project: Qpid
Issue Type: Bug
Components: C++ Broker
Affects Versions: 0.5
Reporter: Gordon Sim
Priority: Blocker
Fix For: 0.6
This allows concurrent closingof a connection that may be processing incoming
or outgoing frames on another thread. The Link::destory() method should not
call connection->close() on the management connections thread but should
request the io thread for the link itself and do the close on that.
E.g. The following is from a core dump running reliable_replication_test (which
is currently failing reliably):
Core was generated by `/var/lib/ptolemy/sources/qpid/cpp/src/.libs/lt-qpidd
--daemon --port 0 --no-dat'.
Program terminated with signal 11, Segmentation fault.
[New process 2138]
[New process 2139]
[New process 2137]
[New process 2136]
[New process 2135]
[New process 2134]
#0 0x00002b81d22cb229 in main_arena () from /lib64/libc.so.6
(gdb) thread apply all bt
Thread 6 (process 2134):
#0 0x00002b81d22db5b5 in pthread_sigmask () from /lib64/libpthread.so.0
#1 0x00002b81d1575a8c in qpid::sys::Poller::wait () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#2 0x00002b81d15763f7 in qpid::sys::Poller::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#3 0x00002b81d110cc36 in qpid::broker::Broker::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#4 0x000000000040916b in QpiddDaemon::child ()
#5 0x00002b81d1125e0e in qpid::broker::Daemon::fork () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#6 0x0000000000406a73 in QpiddBroker::execute ()
#7 0x0000000000405488 in main ()
Thread 5 (process 2135):
#0 0x00002b81d22dab00 in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1 0x00002b81d16393e2 in qpid::sys::Timer::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#2 0x00002b81d156cc5a in qpid::sys::(anonymous namespace)::runRunnable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#3 0x00002b81d22d6367 in start_thread () from /lib64/libpthread.so.0
#4 0x00002b81d204c09d in clone () from /lib64/libc.so.6
Thread 4 (process 2136):
#0 0x00002b81d2057e5e in __lll_lock_wait_private () from /lib64/libc.so.6
#1 0x00002b81d1fef934 in _L_lock_15421 () from /lib64/libc.so.6
#2 0x00002b81d1fee901 in free () from /lib64/libc.so.6
#3 0x00002b81d1119b15 in qpid::broker::Connection::close ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#4 0x00002b81d1152715 in qpid::broker::Link::destroy () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#5 0x00002b81d1152fa1 in qpid::broker::Link::ManagementMethod ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#6 0x00002b81d10dca04 in qmf::org::apache::qpid::broker::Link::doMethod ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#7 0x00002b81d11e15ee in
qpid::management::ManagementAgent::handleMethodRequestLH ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#8 0x00002b81d11e5183 in
qpid::management::ManagementAgent::dispatchAgentCommandLH ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#9 0x00002b81d11e54bb in qpid::management::ManagementAgent::dispatchCommand ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#10 0x00002b81d11ea3c8 in qpid::broker::ManagementExchange::route ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
---Type <return> to continue, or q <return> to quit---
#11 0x00002b81d11a200d in qpid::broker::SemanticState::route ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#12 0x00002b81d11a2de1 in qpid::broker::SemanticState::handle ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#13 0x00002b81d11c746e in qpid::broker::SessionState::handleContent ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#14 0x00002b81d11c7932 in qpid::broker::SessionState::handleIn ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#15 0x00002b81d160256a in qpid::amqp_0_10::SessionHandler::handleIn ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#16 0x00002b81d111a172 in qpid::broker::Connection::received ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#17 0x00002b81d10fba06 in qpid::amqp_0_10::Connection::decode ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#18 0x00002b81d162db67 in qpid::sys::AsynchIOHandler::readbuff ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#19 0x00002b81d11ec478 in
boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,
boost::_mfi::mf2<bool, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&,
qpid::sys::AsynchIOBufferBase*>,
boost::_bi::list3<boost::_bi::value<qpid::sys::AsynchIOHandler*>,
boost::arg<1>, boost::arg<2> > >, bool, qpid::sys::AsynchIO&,
qpid::sys::AsynchIOBufferBase*>::invoke ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#20 0x00002b81d156aa8f in boost::function2<bool, qpid::sys::AsynchIO&,
qpid::sys::AsynchIOBufferBase*, std::allocator<boost::function_base>
>::operator() () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#21 0x00002b81d1568c7c in qpid::sys::posix::AsynchIO::readable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#22 0x00002b81d1632def in boost::function1<void, qpid::sys::DispatchHandle&,
std::allocator<boost::function_base> >::operator() () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#23 0x00002b81d1631a8b in qpid::sys::DispatchHandle::processEvent ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#24 0x00002b81d15763e8 in qpid::sys::Poller::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#25 0x00002b81d156cc5a in qpid::sys::(anonymous namespace)::runRunnable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#26 0x00002b81d22d6367 in start_thread () from /lib64/libpthread.so.0
#27 0x00002b81d204c09d in clone () from /lib64/libc.so.6
Thread 3 (process 2137):
#0 0x00002b81d204c488 in epoll_wait () from /lib64/libc.so.6
#1 0x00002b81d1575aa3 in qpid::sys::Poller::wait () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#2 0x00002b81d15763f7 in qpid::sys::Poller::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
---Type <return> to continue, or q <return> to quit---
#3 0x00002b81d156cc5a in qpid::sys::(anonymous namespace)::runRunnable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#4 0x00002b81d22d6367 in start_thread () from /lib64/libpthread.so.0
#5 0x00002b81d204c09d in clone () from /lib64/libc.so.6
Thread 2 (process 2139):
#0 0x00002b81d204c488 in epoll_wait () from /lib64/libc.so.6
#1 0x00002b81d1575aa3 in qpid::sys::Poller::wait () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#2 0x00002b81d15763f7 in qpid::sys::Poller::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#3 0x00002b81d156cc5a in qpid::sys::(anonymous namespace)::runRunnable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#4 0x00002b81d22d6367 in start_thread () from /lib64/libpthread.so.0
#5 0x00002b81d204c09d in clone () from /lib64/libc.so.6
Thread 1 (process 2138):
#0 0x00002b81d22cb229 in main_arena () from /lib64/libc.so.6
#1 0x00002b81d11c36c5 in qpid::broker::SessionState::enqueued ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#2 0x00002b81d11c74cf in qpid::broker::SessionState::handleContent ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#3 0x00002b81d11c7932 in qpid::broker::SessionState::handleIn ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#4 0x00002b81d160256a in qpid::amqp_0_10::SessionHandler::handleIn ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#5 0x00002b81d111a172 in qpid::broker::Connection::received ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#6 0x00002b81d10fba06 in qpid::amqp_0_10::Connection::decode ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#7 0x00002b81d162db67 in qpid::sys::AsynchIOHandler::readbuff ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#8 0x00002b81d11ec478 in
boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,
boost::_mfi::mf2<bool, qpid::sys::AsynchIOHandler, qpid::sys::AsynchIO&,
qpid::sys::AsynchIOBufferBase*>,
boost::_bi::list3<boost::_bi::value<qpid::sys::AsynchIOHandler*>,
boost::arg<1>, boost::arg<2> > >, bool, qpid::sys::AsynchIO&,
qpid::sys::AsynchIOBufferBase*>::invoke ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidbroker.so.2
#9 0x00002b81d156aa8f in boost::function2<bool, qpid::sys::AsynchIO&,
qpid::sys::AsynchIOBufferBase*, std::allocator<boost::function_base>
>::operator() () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#10 0x00002b81d1568c7c in qpid::sys::posix::AsynchIO::readable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
---Type <return> to continue, or q <return> to quit---
#11 0x00002b81d1632def in boost::function1<void, qpid::sys::DispatchHandle&,
std::allocator<boost::function_base> >::operator() () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#12 0x00002b81d1631a8b in qpid::sys::DispatchHandle::processEvent ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#13 0x00002b81d15763e8 in qpid::sys::Poller::run () from
/var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#14 0x00002b81d156cc5a in qpid::sys::(anonymous namespace)::runRunnable ()
from /var/lib/ptolemy/sources/qpid/cpp/src/.libs/libqpidcommon.so.2
#15 0x00002b81d22d6367 in start_thread () from /lib64/libpthread.so.0
#16 0x00002b81d204c09d in clone () from /lib64/libc.so.6
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
