Public bug reported:
It happend in fullstack test:
neutron.tests.fullstack.test_qos.TestMinBwQoSOvs.test_bw_limit_qos_port_removed(egress).
Test failed and in neutron-openvswitch-agent logs we found error like:
2019-09-17 05:35:50.732 14622 DEBUG neutron.agent.common.ovs_lib
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Processing 1 OpenFlow
rules. do_action_flows
/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/common/ovs_lib.py:432
2019-09-17 05:35:50.733 14622 DEBUG neutron.agent.linux.utils
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Running command (rootwrap
daemon): ['ovs-ofctl', 'del-flows', '-O', 'OpenFlow10', 'br-intad6ecf25c', '-']
execute_rootwrap_daemon
/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py:103
2019-09-17 05:35:50.750 14622 DEBUG ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Running txn n=1
command(idx=0): DbClearCommand(table=QoS,
record=0e930eb1-49eb-4f9e-8639-267abba5f4aa, column=queues) do_commit
/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2019-09-17 05:35:50.757 14622 DEBUG ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Running txn n=1
command(idx=0): DbDestroyCommand(table=Queue,
record=a1470780-1834-48d9-afd0-6fe41fcbb027) do_commit
/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2019-09-17 05:35:50.762 14622 ERROR ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] OVSDB Error:
{"details":"cannot delete Queue row a1470780-1834-48d9-afd0-6fe41fcbb027
because of 1 remaining reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.764 14622 ERROR ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Traceback (most recent
call last):
File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 122, in run
txn.results.put(txn.do_commit())
File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 115, in do_commit
raise RuntimeError(msg)
RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Error executing command:
RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command Traceback
(most recent call last):
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 111, in transaction
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command yield
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command KeyError:
140632266952480
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command During
handling of the above exception, another exception occurred:
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command Traceback
(most recent call last):
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py",
line 40, in execute
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
t.add(self)
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3.6/contextlib.py", line 88, in __exit__
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
next(self.gen)
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 119, in transaction
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command del
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 69, in __exit__
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
self.result = self.commit()
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 62, in commit
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command raise
result.ex
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 122, in run
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
txn.results.put(txn.do_commit())
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 115, in do_commit
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command raise
RuntimeError(msg)
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Error while processing VIF
ports: RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 111, in transaction
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent yield
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError:
140632266952480
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2435, in rpc_loop
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info,
provisioning_needed)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 1968, in process_network_ports
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
devices_added_updated, provisioning_needed))
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 1834, in treat_devices_added_or_updated
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.ext_manager.delete_port(self.context, {'port_id': device})
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/l2_agent_extensions_manager.py",
line 54, in delete_port
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
extension.obj.delete_port(context, data)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 279, in delete_port
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._process_reset_port(port)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 299, in _process_reset_port
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.qos_driver.delete(port)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 94, in delete
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._handle_rule_delete(port, rule_type)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 119, in _handle_rule_delete
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
handler(port)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py",
line 218, in delete_minimum_bandwidth
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.br_int.delete_minimum_bandwidth_queue(port['port_id'])
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/common/ovs_lib.py",
line 985, in delete_minimum_bandwidth_queue
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._delete_queue(queue['_uuid'])
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/common/ovs_lib.py",
line 1057, in _delete_queue
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.ovsdb.db_destroy('Queue', queue_id).execute(check_error=True)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py",
line 40, in execute
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent t.add(self)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python3.6/contextlib.py", line 88, in __exit__
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
next(self.gen)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 119, in transaction
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent del
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 69, in __exit__
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.result
= self.commit()
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 62, in commit
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent raise
result.ex
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 122, in run
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
txn.results.put(txn.do_commit())
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 115, in do_commit
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent raise
RuntimeError(msg)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent RuntimeError:
OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
Full log can be found at
https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_847/681893/1/gate/neutron-fullstack/847f4d9/controller/logs/dsvm-fullstack-logs/TestMinBwQoSOvs.test_bw_limit_qos_port_removed_egress_/neutron-openvswitch-agent--2019-09-17--05-35-29-575569_log.txt.gz
** Affects: neutron
Importance: Medium
Status: Confirmed
** Tags: gate-failure qos
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1845176
Title:
Removing of QoS queue in neutron-ovs-agent fails due to existing
references
Status in neutron:
Confirmed
Bug description:
It happend in fullstack test:
neutron.tests.fullstack.test_qos.TestMinBwQoSOvs.test_bw_limit_qos_port_removed(egress).
Test failed and in neutron-openvswitch-agent logs we found error like:
2019-09-17 05:35:50.732 14622 DEBUG neutron.agent.common.ovs_lib
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Processing 1 OpenFlow
rules. do_action_flows
/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/common/ovs_lib.py:432
2019-09-17 05:35:50.733 14622 DEBUG neutron.agent.linux.utils
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Running command (rootwrap
daemon): ['ovs-ofctl', 'del-flows', '-O', 'OpenFlow10', 'br-intad6ecf25c', '-']
execute_rootwrap_daemon
/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/linux/utils.py:103
2019-09-17 05:35:50.750 14622 DEBUG ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Running txn n=1
command(idx=0): DbClearCommand(table=QoS,
record=0e930eb1-49eb-4f9e-8639-267abba5f4aa, column=queues) do_commit
/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2019-09-17 05:35:50.757 14622 DEBUG ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Running txn n=1
command(idx=0): DbDestroyCommand(table=Queue,
record=a1470780-1834-48d9-afd0-6fe41fcbb027) do_commit
/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:84
2019-09-17 05:35:50.762 14622 ERROR ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] OVSDB Error:
{"details":"cannot delete Queue row a1470780-1834-48d9-afd0-6fe41fcbb027
because of 1 remaining reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.764 14622 ERROR ovsdbapp.backend.ovs_idl.transaction
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Traceback (most recent
call last):
File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 122, in run
txn.results.put(txn.do_commit())
File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 115, in do_commit
raise RuntimeError(msg)
RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Error executing command:
RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
Traceback (most recent call last):
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 111, in transaction
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
yield self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
KeyError: 140632266952480
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command During
handling of the above exception, another exception occurred:
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
Traceback (most recent call last):
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py",
line 40, in execute
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
t.add(self)
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3.6/contextlib.py", line 88, in __exit__
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
next(self.gen)
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 119, in transaction
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command del
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 69, in __exit__
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
self.result = self.commit()
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 62, in commit
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
raise result.ex
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 122, in run
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
txn.results.put(txn.do_commit())
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 115, in do_commit
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
raise RuntimeError(msg)
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.765 14622 ERROR ovsdbapp.backend.ovs_idl.command
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
[req-afe98974-4696-4632-9f6c-a5f19fee65cd - - - - -] Error while processing VIF
ports: RuntimeError: OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 111, in transaction
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent yield
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent KeyError:
140632266952480
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent During handling
of the above exception, another exception occurred:
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent Traceback (most
recent call last):
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 2435, in rpc_loop
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent port_info,
provisioning_needed)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 1968, in process_network_ports
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
devices_added_updated, provisioning_needed))
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/ovs_neutron_agent.py",
line 1834, in treat_devices_added_or_updated
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.ext_manager.delete_port(self.context, {'port_id': device})
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/l2_agent_extensions_manager.py",
line 54, in delete_port
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
extension.obj.delete_port(context, data)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 279, in delete_port
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._process_reset_port(port)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 299, in _process_reset_port
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.qos_driver.delete(port)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 94, in delete
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._handle_rule_delete(port, rule_type)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/l2/extensions/qos.py",
line 119, in _handle_rule_delete
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
handler(port)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/extension_drivers/qos_driver.py",
line 218, in delete_minimum_bandwidth
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.br_int.delete_minimum_bandwidth_queue(port['port_id'])
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/common/ovs_lib.py",
line 985, in delete_minimum_bandwidth_queue
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self._delete_queue(queue['_uuid'])
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/neutron/agent/common/ovs_lib.py",
line 1057, in _delete_queue
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
self.ovsdb.db_destroy('Queue', queue_id).execute(check_error=True)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py",
line 40, in execute
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent t.add(self)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/usr/lib/python3.6/contextlib.py", line 88, in __exit__
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
next(self.gen)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 119, in transaction
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent del
self._nested_txns_map[cur_thread_id]
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/api.py",
line 69, in __exit__
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent self.result
= self.commit()
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 62, in commit
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent raise
result.ex
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
line 122, in run
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent
txn.results.put(txn.do_commit())
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent File
"/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
line 115, in do_commit
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent raise
RuntimeError(msg)
2019-09-17 05:35:50.769 14622 ERROR
neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent RuntimeError:
OVSDB Error: {"details":"cannot delete Queue row
a1470780-1834-48d9-afd0-6fe41fcbb027 because of 1 remaining
reference(s)","error":"referential integrity violation"}
Full log can be found at
https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_847/681893/1/gate/neutron-fullstack/847f4d9/controller/logs/dsvm-fullstack-logs/TestMinBwQoSOvs.test_bw_limit_qos_port_removed_egress_/neutron-openvswitch-agent--2019-09-17--05-35-29-575569_log.txt.gz
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1845176/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
