*** This bug is a duplicate of bug 1912359 *** https://bugs.launchpad.net/bugs/1912359
Public bug reported: Deploying neutron with httpd with ovn driver fails with the 500 with the below traceback[1] when creating networks. 'openstack network agent list' does not list any controller agents or metadata agents. Looks like ovn driver pre_fork_initialize/post_fork_initialize are not triggered that initializes ovn db connections. neutron.conf ------------- .... auth_strategy=keystone core_plugin=ml2 host=oc0-controller-0.mydomain.tld dns_domain=openstacklocal dhcp_agent_notification=True allow_overlapping_ips=True global_physnet_mtu=1500 vlan_transparent=False service_plugins=qos,ovn-router,trunk,segments,port_forwarding,log l3_ha=False max_l3_agents_per_router=3 api_workers=2 rpc_workers=1 router_scheduler_driver=neutron.scheduler.l3_agent_scheduler.ChanceScheduler router_distributed=False enable_dvr=False allow_automatic_l3agent_failover=True ml2_conf.ini ------------ [ml2] type_drivers=geneve,vxlan,vlan,flat tenant_network_types=geneve,vlan mechanism_drivers=ovn path_mtu=0 extension_drivers=qos,port_security,dns overlay_ip_version=4 [ml2_type_geneve] max_header_size=38 vni_ranges=1:65536 [ml2_type_vxlan] vxlan_group=224.0.0.1 vni_ranges=1:65536 [ml2_type_vlan] network_vlan_ranges=datacentre:1:1000 [ml2_type_flat] flat_networks=datacentre [ovn] ovn_nb_connection=tcp:172.16.13.9:6641 ovn_sb_connection=tcp:172.16.13.9:6642 ovsdb_connection_timeout=180 neutron_sync_mode=log ovn_metadata_enabled=True additional_worker_classes_with_ovn_idl=[MaintenanceWorker, RpcWorker] enable_distributed_floating_ip=True dns_servers= ovn_emit_need_to_frag=False [1] 2021-07-26 10:52:26.530 19 DEBUG neutron.api.v2.base [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Request body: {'network': {'name': 'test_net', 'admin_state_up': True}} prepare_request_body /usr/lib/python3.6/site-packages/neutron/api/v2/base.py:729 2021-07-26 10:52:26.532 19 INFO neutron.quota [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7f8daf3696d8>. 2021-07-26 10:52:26.560 19 DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Made reservation on behalf of 20ad786fb60d459a9ac43bea8623d8b3 for: {'network': 1} before /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/quota_enforcement.py:55 2021-07-26 10:52:26.615 19 DEBUG neutron.policy [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Loaded default policies from ['neutron'] under neutron.policies entry points register_rules /usr/lib/python3.6/site-packages/neutron/policy.py:75 2021-07-26 10:52:26.671 19 DEBUG neutron_lib.callbacks.manager [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036846591125'] for network, before_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193 2021-07-26 10:52:26.732 19 DEBUG neutron.plugins.ml2.drivers.helpers [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] geneve segment allocate from pool success with {'geneve_vni': 32248} allocate_partially_specified_segment /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/helpers.py:155 2021-07-26 10:52:26.747 19 DEBUG neutron_lib.callbacks.manager [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment--9223363271446231241', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change--9223372036848473802'] for segment, precommit_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193 2021-07-26 10:52:26.747 19 INFO neutron.db.segments_db [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Added segment 71185329-c337-40d3-b4b6-42797d1d76d7 of type geneve for network 8313f893-f5ca-4dfd-8a3a-b87a305b10ef 2021-07-26 10:52:26.813 19 DEBUG neutron_lib.callbacks.manager [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Notify callbacks ['neutron.services.qos.qos_plugin.QoSPlugin._validate_create_network_callback--9223372036854650062', 'neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8765406609216'] for network, precommit_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Mechanism driver 'ovn' failed in create_network_precommit: AttributeError: 'NoneType' object has no attribute 'nb_global' 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers Traceback (most recent call last): 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 477, in _call_on_drivers 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers getattr(driver.obj, method_name)(context) 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 507, in create_network_precommit 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers self._validate_network_segments(context.network_segments) 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 459, in _validate_network_segments 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers max_tunid = self._get_max_tunid() 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 453, in _get_max_tunid 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers return int(self._nb_ovn.nb_global.options.get('max_tunid')) 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers AttributeError: 'NoneType' object has no attribute 'nb_global' 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] POST failed.: neutron.plugins.ml2.common.exceptions.MechanismDriverError 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last): 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/pecan/core.py", line 683, in __call__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.invoke_controller(controller, args, kwargs, state) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/pecan/core.py", line 574, in invoke_controller 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation result = controller(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation setattr(e, '_RETRY_EXCEEDED', True) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation ectxt.value = e.inner_exc 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 163, in post 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return self.create(resources) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 181, in create 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return {key: creator(*creator_args, **creator_kwargs)} 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 685, in inner 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(self, context, *args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 218, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return method(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation setattr(e, '_RETRY_EXCEEDED', True) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation ectxt.value = e.inner_exc 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1102, in create_network 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation result, mech_context = self._create_network_db(context, network) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1095, in _create_network_db 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.mechanism_manager.create_network_precommit(mech_context) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 513, in create_network_precommit 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise_db_retriable=True) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 495, in _call_on_drivers 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation errors=errors 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation neutron.plugins.ml2.common.exceptions.MechanismDriverError 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation 2021-07-26 10:52:26.913 19 DEBUG neutron.pecan_wsgi.hooks.notifier [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] No notification will be sent due to unsuccessful status code: 500 after /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79 ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1938030 Title: neutron deployed with httpd does not work with ovn mech driver Status in neutron: New Bug description: Deploying neutron with httpd with ovn driver fails with the 500 with the below traceback[1] when creating networks. 'openstack network agent list' does not list any controller agents or metadata agents. Looks like ovn driver pre_fork_initialize/post_fork_initialize are not triggered that initializes ovn db connections. neutron.conf ------------- .... auth_strategy=keystone core_plugin=ml2 host=oc0-controller-0.mydomain.tld dns_domain=openstacklocal dhcp_agent_notification=True allow_overlapping_ips=True global_physnet_mtu=1500 vlan_transparent=False service_plugins=qos,ovn-router,trunk,segments,port_forwarding,log l3_ha=False max_l3_agents_per_router=3 api_workers=2 rpc_workers=1 router_scheduler_driver=neutron.scheduler.l3_agent_scheduler.ChanceScheduler router_distributed=False enable_dvr=False allow_automatic_l3agent_failover=True ml2_conf.ini ------------ [ml2] type_drivers=geneve,vxlan,vlan,flat tenant_network_types=geneve,vlan mechanism_drivers=ovn path_mtu=0 extension_drivers=qos,port_security,dns overlay_ip_version=4 [ml2_type_geneve] max_header_size=38 vni_ranges=1:65536 [ml2_type_vxlan] vxlan_group=224.0.0.1 vni_ranges=1:65536 [ml2_type_vlan] network_vlan_ranges=datacentre:1:1000 [ml2_type_flat] flat_networks=datacentre [ovn] ovn_nb_connection=tcp:172.16.13.9:6641 ovn_sb_connection=tcp:172.16.13.9:6642 ovsdb_connection_timeout=180 neutron_sync_mode=log ovn_metadata_enabled=True additional_worker_classes_with_ovn_idl=[MaintenanceWorker, RpcWorker] enable_distributed_floating_ip=True dns_servers= ovn_emit_need_to_frag=False [1] 2021-07-26 10:52:26.530 19 DEBUG neutron.api.v2.base [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Request body: {'network': {'name': 'test_net', 'admin_state_up': True}} prepare_request_body /usr/lib/python3.6/site-packages/neutron/api/v2/base.py:729 2021-07-26 10:52:26.532 19 INFO neutron.quota [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7f8daf3696d8>. 2021-07-26 10:52:26.560 19 DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Made reservation on behalf of 20ad786fb60d459a9ac43bea8623d8b3 for: {'network': 1} before /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/quota_enforcement.py:55 2021-07-26 10:52:26.615 19 DEBUG neutron.policy [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Loaded default policies from ['neutron'] under neutron.policies entry points register_rules /usr/lib/python3.6/site-packages/neutron/policy.py:75 2021-07-26 10:52:26.671 19 DEBUG neutron_lib.callbacks.manager [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler--9223372036846591125'] for network, before_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193 2021-07-26 10:52:26.732 19 DEBUG neutron.plugins.ml2.drivers.helpers [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] geneve segment allocate from pool success with {'geneve_vni': 32248} allocate_partially_specified_segment /usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/helpers.py:155 2021-07-26 10:52:26.747 19 DEBUG neutron_lib.callbacks.manager [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment--9223363271446231241', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change--9223372036848473802'] for segment, precommit_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193 2021-07-26 10:52:26.747 19 INFO neutron.db.segments_db [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Added segment 71185329-c337-40d3-b4b6-42797d1d76d7 of type geneve for network 8313f893-f5ca-4dfd-8a3a-b87a305b10ef 2021-07-26 10:52:26.813 19 DEBUG neutron_lib.callbacks.manager [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Notify callbacks ['neutron.services.qos.qos_plugin.QoSPlugin._validate_create_network_callback--9223372036854650062', 'neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8765406609216'] for network, precommit_create _notify_loop /usr/lib/python3.6/site-packages/neutron_lib/callbacks/manager.py:193 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] Mechanism driver 'ovn' failed in create_network_precommit: AttributeError: 'NoneType' object has no attribute 'nb_global' 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers Traceback (most recent call last): 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 477, in _call_on_drivers 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers getattr(driver.obj, method_name)(context) 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 507, in create_network_precommit 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers self._validate_network_segments(context.network_segments) 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 459, in _validate_network_segments 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers max_tunid = self._get_max_tunid() 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 453, in _get_max_tunid 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers return int(self._nb_ovn.nb_global.options.get('max_tunid')) 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers AttributeError: 'NoneType' object has no attribute 'nb_global' 2021-07-26 10:52:26.904 19 ERROR neutron.plugins.ml2.managers 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] POST failed.: neutron.plugins.ml2.common.exceptions.MechanismDriverError 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last): 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/pecan/core.py", line 683, in __call__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.invoke_controller(controller, args, kwargs, state) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/pecan/core.py", line 574, in invoke_controller 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation result = controller(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation setattr(e, '_RETRY_EXCEEDED', True) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation ectxt.value = e.inner_exc 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/utils.py", line 76, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 163, in post 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return self.create(resources) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/pecan_wsgi/controllers/resource.py", line 181, in create 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return {key: creator(*creator_args, **creator_kwargs)} 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 685, in inner 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(self, context, *args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 218, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return method(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation setattr(e, '_RETRY_EXCEEDED', True) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation ectxt.value = e.inner_exc 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise() 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise self.value 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1102, in create_network 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation result, mech_context = self._create_network_db(context, network) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/plugin.py", line 1095, in _create_network_db 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation self.mechanism_manager.create_network_precommit(mech_context) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 513, in create_network_precommit 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation raise_db_retriable=True) 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/managers.py", line 495, in _call_on_drivers 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation errors=errors 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation neutron.plugins.ml2.common.exceptions.MechanismDriverError 2021-07-26 10:52:26.910 19 ERROR neutron.pecan_wsgi.hooks.translation 2021-07-26 10:52:26.913 19 DEBUG neutron.pecan_wsgi.hooks.notifier [req-67d890d1-889e-40c7-a380-99396f88cd29 1f65b02b06844d59aa18031903301a79 20ad786fb60d459a9ac43bea8623d8b3 - default default] No notification will be sent due to unsuccessful status code: 500 after /usr/lib/python3.6/site-packages/neutron/pecan_wsgi/hooks/notifier.py:79 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1938030/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp