Public bug reported:
In case when router is migrated from HA to non-HA one, it has to be disabled
and then migrated.
If that is done fast enough, and some error will occur in router_remove method
(https://github.com/openstack/neutron/blob/master/neutron/agent/l3/agent.py#L547)
old router_info will be added to the self.router_info dict.
In such case it will be still HA router added to self.router_info but as this
router is already switched to be non-HA, it will use wrong RouterInfo class and
errors like:
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info [-]
'NoneType' object has no attribute 'remove_vip_by_ip_address': AttributeError:
'NoneType' object has no attribute 'remove_vip_by_ip_address'
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info Traceback
(most recent call last):
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info return
func(*args, **kwargs)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1186,
in process
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self._process_internal_ports()
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 594,
in _process_internal_ports
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self.internal_network_added(p)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 356, in
internal_network_added
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info port,
self.get_internal_device_name, router.INTERNAL_DEV_PREFIX)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 351, in
_plug_ha_router_port
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self._disable_ipv6_addressing_on_interface(interface_name)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 312, in
_disable_ipv6_addressing_on_interface
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self._remove_vip(ipv6_lladdr)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 236, in
_remove_vip
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
instance.remove_vip_by_ip_address(ip_cidr)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent [-] Failed to
process compatible router: 8a8e034d-0c53-4d06-a936-9bb53b064f02:
AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent Traceback (most
recent call last):
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 694, in
_process_routers_if_compatible
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._process_router_if_compatible(router)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 540, in
_process_router_if_compatible
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._process_updated_router(router)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 581, in
_process_updated_router
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent ri.process()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 485, in
process
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent super(HaRouter,
self).process()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent self.logger(e)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self.force_reraise()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
six.reraise(self.type_, self.value, self.tb)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/six.py", line 675, in reraise
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent raise value
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent return
func(*args, **kwargs)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1186,
in process
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._process_internal_ports()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 594,
in _process_internal_ports
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self.internal_network_added(p)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 356, in
internal_network_added
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent port,
self.get_internal_device_name, router.INTERNAL_DEV_PREFIX)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 351, in
_plug_ha_router_port
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._disable_ipv6_addressing_on_interface(interface_name)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 312, in
_disable_ipv6_addressing_on_interface
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._remove_vip(ipv6_lladdr)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 236, in
_remove_vip
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
instance.remove_vip_by_ip_address(ip_cidr)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent AttributeError:
'NoneType' object has no attribute 'remove_vip_by_ip_address'
will occur in L3 agent.
** Affects: neutron
Importance: Medium
Assignee: Slawek Kaplonski (slaweq)
Status: Confirmed
** Tags: l3-ha
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1892846
Title:
AttributeError in l3-agent changing HA router to non-HA
Status in neutron:
Confirmed
Bug description:
In case when router is migrated from HA to non-HA one, it has to be disabled
and then migrated.
If that is done fast enough, and some error will occur in router_remove
method
(https://github.com/openstack/neutron/blob/master/neutron/agent/l3/agent.py#L547)
old router_info will be added to the self.router_info dict.
In such case it will be still HA router added to self.router_info but as this
router is already switched to be non-HA, it will use wrong RouterInfo class and
errors like:
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info [-]
'NoneType' object has no attribute 'remove_vip_by_ip_address': AttributeError:
'NoneType' object has no attribute 'remove_vip_by_ip_address'
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info Traceback
(most recent call last):
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info return
func(*args, **kwargs)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1186,
in process
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self._process_internal_ports()
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 594,
in _process_internal_ports
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self.internal_network_added(p)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 356, in
internal_network_added
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info port,
self.get_internal_device_name, router.INTERNAL_DEV_PREFIX)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 351, in
_plug_ha_router_port
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self._disable_ipv6_addressing_on_interface(interface_name)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 312, in
_disable_ipv6_addressing_on_interface
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
self._remove_vip(ipv6_lladdr)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 236, in
_remove_vip
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
instance.remove_vip_by_ip_address(ip_cidr)
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
2020-08-05 10:17:11.323 142395 ERROR neutron.agent.l3.router_info
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent [-] Failed to
process compatible router: 8a8e034d-0c53-4d06-a936-9bb53b064f02:
AttributeError: 'NoneType' object has no attribute 'remove_vip_by_ip_address'
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent Traceback (most
recent call last):
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 694, in
_process_routers_if_compatible
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._process_router_if_compatible(router)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 540, in
_process_router_if_compatible
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._process_updated_router(router)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 581, in
_process_updated_router
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent ri.process()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 485, in
process
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
super(HaRouter, self).process()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent self.logger(e)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self.force_reraise()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
six.reraise(self.type_, self.value, self.tb)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/six.py", line 675, in reraise
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent raise value
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent return
func(*args, **kwargs)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1186,
in process
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._process_internal_ports()
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 594,
in _process_internal_ports
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self.internal_network_added(p)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 356, in
internal_network_added
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent port,
self.get_internal_device_name, router.INTERNAL_DEV_PREFIX)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 351, in
_plug_ha_router_port
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._disable_ipv6_addressing_on_interface(interface_name)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 312, in
_disable_ipv6_addressing_on_interface
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
self._remove_vip(ipv6_lladdr)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent File
"/usr/lib/python3.6/site-packages/neutron/agent/l3/ha_router.py", line 236, in
_remove_vip
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent
instance.remove_vip_by_ip_address(ip_cidr)
2020-08-05 10:17:11.324 142395 ERROR neutron.agent.l3.agent AttributeError:
'NoneType' object has no attribute 'remove_vip_by_ip_address'
will occur in L3 agent.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1892846/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
