** Changed in: openvswitch (Ubuntu)
Status: New => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1928031
Title:
neutron-ovn-metadata-agent AttributeError: 'MetadataProxyHandler'
object has no attribute 'sb_idl'
Status in charm-ovn-chassis:
Invalid
Status in Ubuntu Cloud Archive:
New
Status in Ubuntu Cloud Archive ussuri series:
New
Status in Ubuntu Cloud Archive wallaby series:
New
Status in neutron:
Fix Released
Status in neutron package in Ubuntu:
Fix Released
Status in openvswitch package in Ubuntu:
Fix Released
Status in neutron source package in Focal:
New
Status in openvswitch source package in Focal:
New
Bug description:
neutron-ovn-metadata-agent not able to handle any metadata requests
from the instances.
Scenario:
* Initially there is some intermittent connectivity issues that are descirbed
in LP #1907686
https://bugs.launchpad.net/charm-ovn-chassis/+bug/1907686/comments/9
* The fix for the above is available in python3-openvswitch package in
ussuri-proposed pocket
Installed the fix on all neutron-server and compute and restarted
neutron-ovn-metadata-agent one by one
* neutron-ovn-metadata-agent on one of the compute nodes not able to
handle any metadata requests after restart. ( Please note the problem
happened with only one ovn-metadata agent and rest of the agents are
good on other compute nodes, so this is some race condition in IDL)
Stacktrace shows both the workers 69188/69189 timed out on OVNSB IDL
connection and hence sb_idl is never initialized.
Stacktrace of Attribute error:
------------------------------
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server
Traceback (most recent call last):
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 67,
in __call__
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server
instance_id, project_id = self._get_instance_and_project_id(req)
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 84,
in _get_instance_and_project_id
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server
ports = self.sb_idl.get_network_port_bindings_by_ip(network_id,
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server
AttributeError: 'MetadataProxyHandler' object has no attribute 'sb_idl'
2021-04-27 08:51:01.340 69188 ERROR neutron.agent.ovn.metadata.server
Stacktrace at the restart of neutron-ovn-metadata-agent process:
----------------------------------------------------------------
2021-04-15 22:27:03.803 69124 INFO neutron.common.config [-]
/usr/bin/neutron-ovn-metadata-agent version 16.2.0
2021-04-15 22:27:03.832 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-]
tcp:127.0.0.1:6640: connecting...
2021-04-15 22:27:03.833 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-]
tcp:127.0.0.1:6640: connected
2021-04-15 22:27:03.949 69124 WARNING neutron.agent.ovn.metadata.agent [-]
Can't read ovn-bridge external-id from OVSDB. Using br-int instead.
2021-04-15 22:27:03.950 69124 INFO oslo_service.service [-] Starting 2 workers
2021-04-15 22:27:03.985 69188 INFO ovsdbapp.backend.ovs_idl.vlog [-]
ssl:10.216.241.118:6642: connecting...
2021-04-15 22:27:03.986 69189 INFO ovsdbapp.backend.ovs_idl.vlog [-]
ssl:10.216.241.118:6642: connecting...
2021-04-15 22:27:04.005 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-]
ssl:10.216.241.118:6642: connecting...
2021-04-15 22:27:04.006 69188 INFO ovsdbapp.backend.ovs_idl.vlog [-]
ssl:10.216.241.118:6642: connected
2021-04-15 22:27:04.033 69189 INFO ovsdbapp.backend.ovs_idl.vlog [-]
ssl:10.216.241.118:6642: connected
2021-04-15 22:27:04.061 69124 INFO ovsdbapp.backend.ovs_idl.vlog [-]
ssl:10.216.241.118:6642: connected
2021-04-15 22:27:06.129 69124 INFO oslo.privsep.daemon [-] Running privsep
helper: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf',
'privsep-helper', '--config-file', '/etc/neutron/neutron.conf',
'--config-file', '/etc/neutron/neutron_ovn_metadata_agent.ini',
'--privsep_context', 'neutron.privileged.default', '--privsep_sock_path',
'/tmp/tmpgncr2rq7/privsep.sock']
2021-04-15 22:27:06.757 69124 INFO oslo.privsep.daemon [-] Spawned new
privsep daemon via rootwrap
2021-04-15 22:27:06.676 69211 INFO oslo.privsep.daemon [-] privsep daemon
starting
2021-04-15 22:27:06.678 69211 INFO oslo.privsep.daemon [-] privsep process
running with uid/gid: 0/0
2021-04-15 22:27:06.680 69211 INFO oslo.privsep.daemon [-] privsep process
running with capabilities (eff/prm/inh):
CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_NET_ADMIN|CAP_SYS_ADMIN|CAP_SYS_PTRACE/none
2021-04-15 22:27:06.680 69211 INFO oslo.privsep.daemon [-] privsep daemon
running as pid 69211
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] OVS database
connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS
and OVN services are available and that the 'ovn_nb_connection' and
'ovn_sb_connection' configuration options are correct.: Exception: Timeout
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most
recent call last):
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 67, in start_connection
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
self.ovsdb_connection.start()
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line
79, in start
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
173, in wait_for_change
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn raise
Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Exception:
Timeout
2021-04-15 22:30:08.542 69188 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager [-] Error
during notification for
neutron.agent.ovn.metadata.server.MetadataProxyHandler.post_fork_initialize-476074
process, after_init:
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable:
OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify
that the OVS and OVN services are available and that the 'ovn_nb_connection'
and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 67, in start_connection
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
self.ovsdb_connection.start()
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line
79, in start
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
173, in wait_for_change
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise
Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Exception:
Timeout
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager During
handling of the above exception, another exception occurred:
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager During
handling of the above exception, another exception occurred:
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in
_notify_loop
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
callback(resource, event, trigger, **kwargs)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 60,
in post_fork_initialize
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
tables=('Port_Binding', 'Datapath_Binding')).start()
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/ovsdb.py", line 57,
in start
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager return
impl_idl_ovn.OvsdbSbOvnIdl(conn)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 724, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py",
line 26, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 63, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
super(Backend, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 32,
in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
self.start_connection(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 72, in start_connection
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise
connection_exception
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable:
OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify
that the OVS and OVN services are available and that the 'ovn_nb_connection'
and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise
Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Exception:
Timeout
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager During
handling of the above exception, another exception occurred:
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in
_notify_loop
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
callback(resource, event, trigger, **kwargs)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 60,
in post_fork_initialize
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
tables=('Port_Binding', 'Datapath_Binding')).start()
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/ovsdb.py", line 57,
in start
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager return
impl_idl_ovn.OvsdbSbOvnIdl(conn)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 724, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py",
line 26, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 63, in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
super(Backend, self).__init__(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 32,
in __init__
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
self.start_connection(connection)
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 72, in start_connection
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager raise
connection_exception
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable:
OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify
that the OVS and OVN services are available and that the 'ovn_nb_connection'
and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.544 69188 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.546 69188 INFO eventlet.wsgi.server [-] (69188) wsgi
starting up on http:/var/lib/neutron/metadata_proxy
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] OVS database
connection to OVN_Southbound failed with error: 'Timeout'. Verify that the OVS
and OVN services are available and that the 'ovn_nb_connection' and
'ovn_sb_connection' configuration options are correct.: Exception: Timeout
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Traceback (most
recent call last):
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 67, in start_connection
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
self.ovsdb_connection.start()
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line
79, in start
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
173, in wait_for_change
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn raise
Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn Exception:
Timeout
2021-04-15 22:30:08.550 69189 ERROR
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager [-] Error
during notification for
neutron.agent.ovn.metadata.server.MetadataProxyHandler.post_fork_initialize-476074
process, after_init:
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable:
OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify
that the OVS and OVN services are available and that the 'ovn_nb_connection'
and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 67, in start_connection
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
self.ovsdb_connection.start()
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/connection.py", line
79, in start
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
idlutils.wait_for_change(self.idl, self.timeout)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
173, in wait_for_change
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager raise
Exception("Timeout") # TODO(twilson) use TimeoutException?
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager Exception:
Timeout
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager During
handling of the above exception, another exception occurred:
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in
_notify_loop
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
callback(resource, event, trigger, **kwargs)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/server.py", line 60,
in post_fork_initialize
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
tables=('Port_Binding', 'Datapath_Binding')).start()
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/agent/ovn/metadata/ovsdb.py", line 57,
in start
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager return
impl_idl_ovn.OvsdbSbOvnIdl(conn)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 724, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py",
line 26, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 63, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
super(OvsdbSbOvnIdl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/schema/ovn_southbound/impl_idl.py",
line 26, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
super(OvnSbApiIdlImpl, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 63, in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
super(Backend, self).__init__(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 32,
in __init__
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
self.start_connection(connection)
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py",
line 72, in start_connection
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager raise
connection_exception
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn.OvsdbConnectionUnavailable:
OVS database connection to OVN_Southbound failed with error: 'Timeout'. Verify
that the OVS and OVN services are available and that the 'ovn_nb_connection'
and 'ovn_sb_connection' configuration options are correct.
2021-04-15 22:30:08.552 69189 ERROR neutron_lib.callbacks.manager
2021-04-15 22:30:08.555 69189 INFO eventlet.wsgi.server [-] (69189) wsgi
starting up on http:/var/lib/neutron/metadata_proxy
* However netstat shows the TCP connection to SB IDL is established
for both the workers. So this could be some problem in data at IDL.
$ grep 10.216.241.118
sosreport-*-2021-04-27-edidnbl/sos_commands/networking/netstat_-W_-neopa
tcp 0 0 10.216.241.244:53284 10.216.241.118:6642 ESTABLISHED 0 407222779
69124/neutron-ovn-m off (0.00/0/0)
tcp 1490260 0 10.216.241.244:53282 10.216.241.118:6642 ESTABLISHED 0
407288937 69189/neutron-ovn-m off (0.00/0/0)
tcp 1490260 0 10.216.241.244:53280 10.216.241.118:6642 ESTABLISHED 0
407156544 69188/neutron-ovn-m off (0.00/0/0)
It will be good if sb_idl is not initialised, the worker thread can be
closed or neutron-ovn-metadata-agent should be stopped so that the
problem will be notified to operator at the earliest.
To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-ovn-chassis/+bug/1928031/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
