Public bug reported: I use tempest test my company's openstack environment.
# tox -eall -- tempest.api.network When tox finished. Log in L3-agent always show: 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent [-] Error while deleting router da4b28ce-33b1-4000-8609-a41a2ab8c982 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 344, in _safe_router_removed 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self._router_removed(router_id) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 362, in _router_removed 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent ri.delete(self) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 364, in delete 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent super(HaRouter, self).delete(agent) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 273, in delete 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self.process(agent) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 370, in process 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent super(HaRouter, self).process(agent) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/common/utils.py", line 359, in call 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self.logger(e) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__ 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/common/utils.py", line 356, in call 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent return func(*args, **kwargs) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 695, in process 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self.routes_updated() 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 181, in routes_updated 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent instance = self._get_keepalived_instance() 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 131, in _get_keepalived_instance 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent return self.keepalived_manager.config.get_instance(self.ha_vr_id) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'config' I think the reason is tempest create and delete router too fast. when l3-agent create ha-router,tempest delete the router,and neutron-server delete ha-interface https://github.com/openstack/neutron/blob/master/neutron/agent/l3/ha_router.py#L79 class HaRouter can't initialized without ha-interface information,it just return without _init_keepalived_manager When l3-agent delete router,it report AttributeError: 'NoneType'. When l3 agent can't delete the router,l3 agent always fullsync with neutron-server every 30 seconds. In controller, the neutron-server cpu always 70%..^-^ L3 agent should add a check before create HArouter.If it find ha- interface is none ,means router has been deleted in neutron-server. ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1510757 Title: tempest test api.network;l3 agent can't delete HA-router Status in neutron: New Bug description: I use tempest test my company's openstack environment. # tox -eall -- tempest.api.network When tox finished. Log in L3-agent always show: 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent [-] Error while deleting router da4b28ce-33b1-4000-8609-a41a2ab8c982 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 344, in _safe_router_removed 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self._router_removed(router_id) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/agent.py", line 362, in _router_removed 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent ri.delete(self) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 364, in delete 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent super(HaRouter, self).delete(agent) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 273, in delete 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self.process(agent) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 370, in process 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent super(HaRouter, self).process(agent) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/common/utils.py", line 359, in call 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self.logger(e) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 197, in __exit__ 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/common/utils.py", line 356, in call 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent return func(*args, **kwargs) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/router_info.py", line 695, in process 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent self.routes_updated() 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 181, in routes_updated 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent instance = self._get_keepalived_instance() 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent File "/usr/local/lib/python2.7/dist-packages/neutron/agent/l3/ha_router.py", line 131, in _get_keepalived_instance 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent return self.keepalived_manager.config.get_instance(self.ha_vr_id) 2015-10-27 19:32:03.281 23885 ERROR neutron.agent.l3.agent AttributeError: 'NoneType' object has no attribute 'config' I think the reason is tempest create and delete router too fast. when l3-agent create ha-router,tempest delete the router,and neutron-server delete ha-interface https://github.com/openstack/neutron/blob/master/neutron/agent/l3/ha_router.py#L79 class HaRouter can't initialized without ha-interface information,it just return without _init_keepalived_manager When l3-agent delete router,it report AttributeError: 'NoneType'. When l3 agent can't delete the router,l3 agent always fullsync with neutron-server every 30 seconds. In controller, the neutron-server cpu always 70%..^-^ L3 agent should add a check before create HArouter.If it find ha- interface is none ,means router has been deleted in neutron-server. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1510757/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
