Public bug reported:
I'm seeing an error in the l2pop code where it's failing to add a flow
for the ARP entry responder.
This is sometimes leading to DHCP failures for VMs, although a soft
reboot typically fixes that problem.
Here is the trace:
2014-09-10 15:10:36.954 9351 ERROR neutron.agent.linux.ovs_lib
[req-de0c2985-1fac-46a8-a42b-f0bad5a43805 None] OVS flows could not be applied
on bridge br-tun
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib Traceback (most
recent call last):
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 407, in _fdb_chg_ip
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib
self.local_ip, self.local_vlan_map)
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib return
method(*args, **kwargs)
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 250, in fdb_chg_ip_tun
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib for mac, ip
in after:
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib TypeError:
'NoneType' object is not iterable
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib
2014-09-10 15:10:36.955 9351 ERROR oslo.messaging.rpc.dispatcher
[req-de0c2985-1fac-46a8-a42b-f0bad5a43805 ] Exception during message handling:
'NoneType' object is not iterable
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher Traceback
(most recent call last):
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py",
line 134, in _dispatch_and_reply
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
incoming.message))
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py",
line 177, in _dispatch
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py",
line 123, in _do_dispatch
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher result =
getattr(endpoint, method)(ctxt, **new_args)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
method(*args, **kwargs)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 55, in update_fdb_entries
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
self.fdb_update(context, fdb_entries)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
method(*args, **kwargs)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 212, in fdb_update
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
getattr(self, method)(context, values)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 407, in _fdb_chg_ip
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
self.local_ip, self.local_vlan_map)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
method(*args, **kwargs)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 250, in fdb_chg_ip_tun
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher for mac,
ip in after:
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher TypeError:
'NoneType' object is not iterable
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
2014-09-10 15:10:36.957 9351 ERROR oslo.messaging._drivers.common
[req-de0c2985-1fac-46a8-a42b-f0bad5a43805 ] Returning exception 'NoneType'
object is not iterable to caller
I don't know this code well enough to suggest a fix - whether it's
checking the return from agent_ports.items() better, or that there is a
bug elsewhere, so any help would be appreciated.
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1367881
Title:
l2pop RPC code throwing an exception in fdb_chg_ip_tun()
Status in OpenStack Neutron (virtual network service):
New
Bug description:
I'm seeing an error in the l2pop code where it's failing to add a flow
for the ARP entry responder.
This is sometimes leading to DHCP failures for VMs, although a soft
reboot typically fixes that problem.
Here is the trace:
2014-09-10 15:10:36.954 9351 ERROR neutron.agent.linux.ovs_lib
[req-de0c2985-1fac-46a8-a42b-f0bad5a43805 None] OVS flows could not be applied
on bridge br-tun
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib Traceback
(most recent call last):
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 407, in _fdb_chg_ip
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib
self.local_ip, self.local_vlan_map)
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib return
method(*args, **kwargs)
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 250, in fdb_chg_ip_tun
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib for mac,
ip in after:
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib TypeError:
'NoneType' object is not iterable
2014-09-10 15:10:36.954 9351 TRACE neutron.agent.linux.ovs_lib
2014-09-10 15:10:36.955 9351 ERROR oslo.messaging.rpc.dispatcher
[req-de0c2985-1fac-46a8-a42b-f0bad5a43805 ] Exception during message handling:
'NoneType' object is not iterable
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher Traceback
(most recent call last):
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py",
line 134, in _dispatch_and_reply
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
incoming.message))
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py",
line 177, in _dispatch
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
self._do_dispatch(endpoint, method, ctxt, args)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py",
line 123, in _do_dispatch
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher result =
getattr(endpoint, method)(ctxt, **new_args)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
method(*args, **kwargs)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 55, in update_fdb_entries
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
self.fdb_update(context, fdb_entries)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
method(*args, **kwargs)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 212, in fdb_update
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
getattr(self, method)(context, values)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
line 407, in _fdb_chg_ip
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
self.local_ip, self.local_vlan_map)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/common/log.py",
line 36, in wrapper
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher return
method(*args, **kwargs)
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher File
"/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/neutron/agent/l2population_rpc.py",
line 250, in fdb_chg_ip_tun
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher for mac,
ip in after:
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher TypeError:
'NoneType' object is not iterable
2014-09-10 15:10:36.955 9351 TRACE oslo.messaging.rpc.dispatcher
2014-09-10 15:10:36.957 9351 ERROR oslo.messaging._drivers.common
[req-de0c2985-1fac-46a8-a42b-f0bad5a43805 ] Returning exception 'NoneType'
object is not iterable to caller
I don't know this code well enough to suggest a fix - whether it's
checking the return from agent_ports.items() better, or that there is
a bug elsewhere, so any help would be appreciated.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1367881/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
