This bug has had a related patch abandoned and has been automatically
un-assigned due to inactivity. Please re-assign yourself if you are
continuing work or adjust the state as appropriate if it is no longer
valid.
** Changed in: neutron
Status: Fix Released => New
** Changed in: neutron
Assignee: Yaohua Yan (sleepdream) => (unassigned)
** Tags added: timeout-abandon
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1658060
Title:
FirewallNotFound exceptions when deleting the firewall in FWaaS-DVR
Status in neutron:
New
Bug description:
We have four nodes, and we deploy both the FWaaS and DVR services.
When deleting the firewall, we always get three FirewallNotFound
exceptions. At present, we believe that, in DVR environment, evey node
would run a L3-agent service. This causes a plugin corresponding to
multiple agents. And each agent will call back the plugin's
firewall_deleted() (neutron_fwaas/services/firewall/fwaas_plugin.py)
to delete the instance in DB, but only the first agent will succeed.
How to reproduce:
- first create a firewall applied to a DVR router
- then delete it
$ neutron router-show test-fwaas
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| distributed | True |
| external_gateway_info | |
| ha | False |
| id | cfa3e65e-d101-4cc7-80e5-39daf72c6572 |
| name | test-fwaas |
| routes | |
| status | ACTIVE |
| tenant_id | fc170b1b8a9a467b9e1a63d85ced5a86 |
+-----------------------+--------------------------------------+
$ neutron firewall-create --name fw --router test-fwaas policy
Created a new firewall:
+--------------------+--------------------------------------+
| Field | Value |
+--------------------+--------------------------------------+
| admin_state_up | True |
| description | |
| firewall_policy_id | 1eb3fff7-240f-4f9d-adf6-766e2cad7f59 |
| id | afd38a9e-cf0a-4667-94e0-853a888fd981 |
| name | fw |
| router_ids | cfa3e65e-d101-4cc7-80e5-39daf72c6572 |
| status | CREATED |
| tenant_id | fc170b1b8a9a467b9e1a63d85ced5a86 |
+--------------------+--------------------------------------+
$ neutron firewall-show fw
+--------------------+--------------------------------------+
| Field | Value |
+--------------------+--------------------------------------+
| admin_state_up | True |
| description | |
| firewall_policy_id | 1eb3fff7-240f-4f9d-adf6-766e2cad7f59 |
| id | afd38a9e-cf0a-4667-94e0-853a888fd981 |
| name | fw |
| router_ids | cfa3e65e-d101-4cc7-80e5-39daf72c6572 |
| status | ACTIVE |
| tenant_id | fc170b1b8a9a467b9e1a63d85ced5a86 |
+--------------------+--------------------------------------+
$ neutron firewall-delete fw
$ less neutron-service_error.log
2017-01-20 19:46:11.593 19338 ERROR oslo_messaging.rpc.dispatcher
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] Exception during message handling:
Firewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found.
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatch_and_reply
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher
executor_callback))
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher
executor_callback)
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 130, in _do_dispatch
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher result
= func(ctxt, **new_args)
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/neutron-fwaas/neutron_fwaas/services/firewall/fwaas_plugin.py",
line 67, in firewall_deleted
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher fw_db =
self.plugin._get_firewall(context, firewall_id)
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/neutron-fwaas/neutron_fwaas/db/firewall/firewall_db.py", line
101, in _get_firewall
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher raise
fw_ext.FirewallNotFound(firewall_id=id)
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher
FirewallNotFound: Firewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be
found.
2017-01-20 19:46:11.593 19338 TRACE oslo_messaging.rpc.dispatcher
2017-01-20 19:46:11.594 19338 ERROR oslo_messaging._drivers.common
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] Returning exception Firewall
afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found. to caller
2017-01-20 19:46:11.594 19338 ERROR oslo_messaging._drivers.common
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] ['Traceback (most recent call
last):\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatc
h_and_reply\n executor_callback))\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch\n executor_callback)\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rp
c/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt,
**new_args)\n', ' File
"/opt/openstack/neutron-fwaas/neutron_fwaas/services/firewall/fwaas_plugin.py",
line 67, in firewall_deleted\n fw_db = self.plugin._get_firewall(context,
firewall_id)\n', ' Fi
le "/opt/openstack/neutron-fwaas/neutron_fwaas/db/firewall/firewall_db.py",
line 101, in _get_firewall\n raise
fw_ext.FirewallNotFound(firewall_id=id)\n', 'FirewallNotFound: Firewall
afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found.\n']
2017-01-20 19:46:11.595 19338 ERROR oslo_messaging.rpc.dispatcher
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] Exception during message handling:
Firewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found.
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatch_and_reply
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher
executor_callback))
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher
executor_callback)
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 130, in _do_dispatch
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher result
= func(ctxt, **new_args)
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/neutron-fwaas/neutron_fwaas/services/firewall/fwaas_plugin.py",
line 67, in firewall_deleted
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher fw_db =
self.plugin._get_firewall(context, firewall_id)
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/neutron-fwaas/neutron_fwaas/db/firewall/firewall_db.py", line
101, in _get_firewall
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher raise
fw_ext.FirewallNotFound(firewall_id=id)
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher
FirewallNotFound: Firewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be
found.
2017-01-20 19:46:11.595 19338 TRACE oslo_messaging.rpc.dispatcher
2017-01-20 19:46:11.596 19338 ERROR oslo_messaging._drivers.common
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] Returning exception Firewall
afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found. to caller
2017-01-20 19:46:11.596 19338 ERROR oslo_messaging._drivers.common
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] ['Traceback (most recent call
last):\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatc
h_and_reply\n executor_callback))\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch\n executor_callback)\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rp
c/dispatcher.py", line 130, in _do_dispatch\n result = func(ctxt,
**new_args)\n', ' File
"/opt/openstack/neutron-fwaas/neutron_fwaas/services/firewall/fwaas_plugin.py",
line 67, in firewall_deleted\n fw_db = self.plugin._get_firewall(context,
firewall_id)\n', ' Fi
le "/opt/openstack/neutron-fwaas/neutron_fwaas/db/firewall/firewall_db.py",
line 101, in _get_firewall\n raise
fw_ext.FirewallNotFound(firewall_id=id)\n', 'FirewallNotFound: Firewall
afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found.\n']
2017-01-20 19:46:11.599 19338 ERROR oslo_messaging.rpc.dispatcher
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] Exception during message handling:
Firewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found.
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatch_and_reply
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher
executor_callback))
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher
executor_callback)
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 130, in _do_dispatch
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher result
= func(ctxt, **new_args)
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/neutron-fwaas/neutron_fwaas/services/firewall/fwaas_plugin.py",
line 67, in firewall_deleted
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher fw_db =
self.plugin._get_firewall(context, firewall_id)
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher File
"/opt/openstack/neutron-fwaas/neutron_fwaas/db/firewall/firewall_db.py", line
101, in _get_firewall
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher raise
fw_ext.FirewallNotFound(firewall_id=id)
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher
FirewallNotFound: Firewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be
found.
2017-01-20 19:46:11.599 19338 TRACE oslo_messaging.rpc.dispatcher
2017-01-20 19:46:11.600 19338 ERROR oslo_messaging._drivers.common
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] Returning exception Firewall
afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found. to caller
2017-01-20 19:46:11.600 19338 ERROR oslo_messaging._drivers.common
[req-c4af8425-b05a-4c4e-98e0-4dabe0057df7 ] ['Traceback (most recent call
last):\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatch_and_reply\n executor_callback))\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch\n executor_callback)\n', ' File
"/opt/openstack/.venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py",
line 130, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File
"/opt/openstack/neutron-fwaas/neutron_fwaas/services/firewall/fwaas_plugin.py",
line 67, in firewall_deleted\n fw_db = self.plugin._get_firewall(context,
firewall_id)\n', ' File
"/opt/openstack/neutron-fwaas/neutron_fwaas/db/firewall/firewall_db.py", line
101, in _get_firewall\n raise fw_ext.FirewallNotFound(firewall_id=id)\n',
'FirewallNotFound: Fir
ewall afd38a9e-cf0a-4667-94e0-853a888fd981 could not be found.\n']
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1658060/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
