** Changed in: charm-ovn-central
Status: Fix Committed => Fix Released
** Changed in: charm-neutron-api-plugin-ovn
Status: Fix Committed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1864640
Title:
[Ussuri] Neutron API writes to the Southbound DB, does not play well
with OVN RBAC
Status in charm-neutron-api-plugin-ovn:
Fix Released
Status in charm-ovn-central:
Fix Released
Status in neutron:
Confirmed
Bug description:
At Ussuri Neutron API has begun doing writes directly to the
Southbound DB, there does not appear to be a accompanying RBAC role
for this, so do we need to give it access to the private port
currently reserved for ovn-northd?
The offending change in upstream Neutron arrived here:
https://github.com/openstack/networking-
ovn/commit/70c3d06656e15e11a0daf9c3732a21c8ce601c4d
Example of an failed transaction:
2020-02-25 11:04:33.420 1520231 ERROR ovsdbapp.backend.ovs_idl.transaction
[req-8315d356-f92f-4447-a47b-f724374cfc36 - - - - -] OVSDB Error:
{"details":"RBAC rules for client \"juju-ef641e-1-lxd-2.maas\" role
\"ovn-controller\" prohibit modification of table
\"Chassis\".","error":"permission error"}
2020-02-25 11:04:33.420 1520231 ERROR ovsdbapp.backend.ovs_idl.transaction
[req-fbf878ca-f0bc-465c-b173-882d695cb4aa 3ff519473176440bb9678c95051ed627
dd8f9f301d1e436d8d3a9b695537c897 - cb4c93ee9c98459c8cde54c2c8b0a829
cb4c93ee9c98459c8cde54c2c8b0a829] Traceback (most recent call last):
File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line
122, in run
txn.results.put(txn.do_commit())
File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line
115, in do_commit
raise RuntimeError(msg)
RuntimeError: OVSDB Error: {"details":"RBAC rules for client
\"juju-ef641e-1-lxd-2.maas\" role \"ovn-controller\" prohibit modification of
table \"Chassis\".","error":"permission error"}
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
[req-fbf878ca-f0bc-465c-b173-882d695cb4aa 3ff519473176440bb9678c95051ed627
dd8f9f301d1e436d8d3a9b695537c897 - cb4c93ee9c98459c8cde54c2c8b0a829
cb4c93ee9c98459c8cde54c2c8b0a829] Error executing command: RuntimeError: OVSDB
Error: {"details":"RBAC rules for client \"juju-ef641e-1-lxd-2.maas\" role
\"ovn-controller\" prohibit modification of table
\"Chassis\".","error":"permission error"}
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
Traceback (most recent call last):
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 40,
in execute
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
t.add(self)
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3.6/contextlib.py", line 88, in __exit__
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
next(self.gen)
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 119, in transaction
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
del self._nested_txns_map[cur_thread_id]
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/api.py", line 69, in __exit__
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
self.result = self.commit()
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line
62, in commit
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
raise result.ex
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line
122, in run
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
txn.results.put(txn.do_commit())
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/transaction.py", line
115, in do_commit
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
raise RuntimeError(msg)
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
RuntimeError: OVSDB Error: {"details":"RBAC rules for client
\"juju-ef641e-1-lxd-2.maas\" role \"ovn-controller\" prohibit modification of
table \"Chassis\".","error":"permission error"}
2020-02-25 11:04:33.421 1520231 ERROR ovsdbapp.backend.ovs_idl.command
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
[req-fbf878ca-f0bc-465c-b173-882d695cb4aa 3ff519473176440bb9678c95051ed627
dd8f9f301d1e436d8d3a9b695537c897 - cb4c93ee9c98459c8cde54c2c8b0a829
cb4c93ee9c98459c8cde54c2c8b0a829] POST failed.:
neutron_lib.callbacks.exceptions.CallbackFailure: Callback
neutron.services.segments.db._add_segment_host_mapping_for_segment--9223363248144570382
failed with "OVSDB Error: {"details":"RBAC rules for client
\"juju-ef641e-1-lxd-2.maas\" role \"ovn-controller\" prohibit modification of
table \"Chassis\".","error":"permission error"}"
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
Traceback (most recent call last):
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/pecan/core.py", line 683, in __call__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.invoke_controller(controller, args, kwargs, state)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/pecan/core.py", line 574, in
invoke_controller
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
result = controller(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 139, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
setattr(e, '_RETRY_EXCEEDED', True)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 135, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 154, in wrapper
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
ectxt.value = e.inner_exc
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 142, in wrapper
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 183, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
LOG.debug("Retry wrapper got retriable exception: %s", e)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 179, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*dup_args, **dup_kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/pecan_wsgi/controllers/utils.py",
line 76, in wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File
"/usr/lib/python3/dist-packages/neutron/pecan_wsgi/controllers/resource.py",
line 163, in post
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return self.create(resources)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File
"/usr/lib/python3/dist-packages/neutron/pecan_wsgi/controllers/resource.py",
line 181, in create
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return {key: creator(*creator_args, **creator_kwargs)}
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 685, in
inner
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(self, context, *args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 233, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return method(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 139, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
setattr(e, '_RETRY_EXCEEDED', True)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 135, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 154, in wrapper
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
ectxt.value = e.inner_exc
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_db/api.py", line 142, in wrapper
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 183, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
LOG.debug("Retry wrapper got retriable exception: %s", e)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/api.py", line 179, in
wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return f(*dup_args, **dup_kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/plugin.py", line 1047,
in create_network
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
result, mech_context = self._create_network_db(context, network)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/plugin.py", line 1006,
in _create_network_db
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
tenant_id)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/managers.py", line
218, in create_network_segments
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
segment_index)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/managers.py", line
201, in _add_network_segment
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
context, network_id, segment, segment_index)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron/db/segments_db.py", line 57, in
add_network_segment
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
segment=netseg_obj)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/registry.py", line
56, in notify
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
_get_callback_manager().notify(resource, event, trigger, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 108, in
_wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise db_exc.RetryRequest(e)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 220, in
__exit__
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
self.force_reraise()
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 196, in
force_reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
six.reraise(self.type_, self.value, self.tb)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise value
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 103, in
_wrapped
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
return function(*args, **kwargs)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line
177, in notify
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
raise exceptions.CallbackFailure(errors=errors)
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
neutron_lib.callbacks.exceptions.CallbackFailure: Callback
neutron.services.segments.db._add_segment_host_mapping_for_segment--9223363248144570382
failed with "OVSDB Error: {"details":"RBAC rules for client
\"juju-ef641e-1-lxd-2.maas\" role \"ovn-controller\" prohibit modification of
table \"Chassis\".","error":"permission error"}"
2020-02-25 11:04:33.486 1520231 ERROR neutron.pecan_wsgi.hooks.translation
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-neutron-api-plugin-ovn/+bug/1864640/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
