** Also affects: neutron
Importance: Undecided
Status: New
** Also affects: neutron/icehouse
Importance: Undecided
Status: New
** Changed in: neutron/icehouse
Status: New => In Progress
** Changed in: neutron
Status: New => Fix Committed
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1303890
Title:
Uncaught qpid error can break a consumer
Status in OpenStack Neutron (virtual network service):
Fix Committed
Status in neutron icehouse series:
In Progress
Status in Oslo - a Library of Common OpenStack Code:
Fix Committed
Status in oslo havana series:
Fix Committed
Status in oslo icehouse series:
Fix Committed
Status in Messaging API for OpenStack:
Fix Committed
Bug description:
The following exception was originally observed against the old rpc
code, but the same problem exists in oslo.messaging.
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/nova/openstack/common/excutils.py",
line 78, in inner_func
return infunc(*args, **kwargs)
File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line
698, in _consumer_thread
self.consume()
File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line
689, in consume
it.next()
File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line
606, in iterconsume
yield self.ensure(_error_callback, _consume)
File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line
540, in ensure
return method(*args, **kwargs)
File
"/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line
597, in _consume
nxt_receiver = self.session.next_receiver(timeout=timeout)
File "<string>", line 6, in next_receiver
File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line
665, in next_receiver
if self._ecwait(lambda: self.incoming, timeout):
File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line
50, in _ecwait
result = self._ewait(lambda: self.closed or predicate(), timeout)
File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line
571, in _ewait
result = self.connection._ewait(lambda: self.error or predicate(),
timeout)
File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line
214, in _ewait
self.check_error()
File "/usr/lib/python2.6/site-packages/qpid/messaging/endpoints.py", line
207, in check_error
raise self.error
InternalError: Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/qpid/messaging/driver.py", line
667, in write
self._op_dec.write(*self._seg_dec.read())
File "/usr/lib/python2.6/site-packages/qpid/framing.py", line 269, in write
if self.op.headers is None:
AttributeError: 'NoneType' object has no attribute 'headers'
It's possible for something to put the qpid client into a bad state.
In particular, I have observed a case that will cause
session.next_receiver() to immediately raise an InternalError. This
exception makes it all the way out. If the eventlet executor is used,
the forever_retry_uncaught_exceptions() decorator will get hit. It
will go back into this code and get the same error, stuck in an
infinite loop of retrying.
The connection needs to be reset in this case to recover.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1303890/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
