c++ broker dies if federation link is flaky and errors out
----------------------------------------------------------
Key: QPID-3618
URL: https://issues.apache.org/jira/browse/QPID-3618
Project: Qpid
Issue Type: Bug
Components: C++ Broker
Affects Versions: 0.12
Environment: Linux, ARM platform. Limited hardware resources.
Federation link over a cellular data connection.
Reporter: Brandon Pedersen
PROBLEM STATEMENT:
I have a durable queue push route configured to send messages to a central
broker. The link between the source broker and the destination broker is a
flaky cellular connection. When the link goes on and off it will very
frequently cause the broker to seg fault and die.
STEPS TO REPRODUCE:
This is tricky to reproduce but it most often occurs when the physical data
connection is just being brought up. i.e. the cell connection went down, qpid
releases the connection....after a while the connection comes back up and qpid
tries to reconnect *possibly* (not certain) when the connection has not been
completely set up.
RESULTS:
The broker receives some sort of data frame before the connection is fully
established and therefore throws and exception. During this process it appears
there is another thread that accesses some pointer that is invalid and causes
the program to die.
LOGS:
The log shows the following messages just before dieing:
Nov 14 15:43:35 mtcdp daemon.err qpidd[6790]: 2011-11-14 15:43:35
error Connection local:55764-remote:5672 closed by error: Connection
not yet open, invalid frame received.(501)
Nov 14 15:43:35 mtcdp daemon.err qpidd[6790]: 2011-11-14 15:43:35
error Connection local:55764-remote:5672 closed by error: Connection
not yet open, invalid frame received.(501)
Notice that the connection error message shows up twice, this seems suspicious
to me. This occurs every time the broker dies.
I got a core dump from the seg fault (a couple of times) and this is the
backtrace
Core was generated by `qpidd'.
Program terminated with signal 11, Segmentation fault.
#0 0x403fdc98 in qpid::SessionState::disableReceiverTracking() ()
from /usr/lib/libqpidcommon.so.2
(gdb) backtrace
#0 0x403fdc98 in qpid::SessionState::disableReceiverTracking() ()
from /usr/lib/libqpidcommon.so.2
#1 0x4010e8a8 in
qpid::broker::Bridge::create(qpid::broker::Connection&) () from
/usr/lib/libqpidbroker.so.2
#2 0x4017ed18 in qpid::broker::Link::ioThreadProcessing() () from
/usr/lib/libqpidbroker.so.2
#3 0x40180920 in ?? () from /usr/lib/libqpidbroker.so.2
Cannot access memory at address 0x2d74c0f8
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
