Public bug reported:
Running Openstack 2024.1 "Caracal" on Ubuntu Jammy deployed with FCE
(MaaS 3.6, Juju 3.6) and OVN 24.03, we're facing an issue when using
private network+FIP based tenants.
When creating such tenant, we need to create a router with a gateway subnet.
When this object is created we see these error messages in neutron logs :
==============================================================================================
2025-09-17 12:04:57.099 4411 INFO neutron.db.ovn_revision_numbers_db [None
req-dc2f485e-6938-4c4c-a776-91cb1578809d 3d1d518e2e534286a94a3ad73e9ec00a
900e2c26f77041d19163704cbd8243f9 - - 6b4f95d250fe4dd08b9a4c23e61cce88
6b4f95d250fe4dd08b9a4c23e61cce88] Successfully bumped revision number for
resource 0626d085-bfb4-4019-8ff5-b8dd8259bf0d (type: ports) to 2
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command [None
req-541af44e-4c9f-4e19-88ff-b777bf201710 - - - - - -] Error executing command
(DbGetCommand): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find
Logical_Router_Port with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command Traceback
(most recent call last):
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 47,
in execute
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
self.run_idl(None)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 235,
in run_idl
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command record
= self.api.lookup(self.table, self.record)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
184, in lookup
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command return
self._lookup(table, record)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
235, in _lookup
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command row =
idlutils.row_by_value(self, rl.table, rl.column, record)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
114, in row_by_value
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command raise
RowNotFound(table=table, col=column, match=match)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port
with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager [None
req-541af44e-4c9f-4e19-88ff-b777bf201710 - - - - - -] Error during notification
for neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._port_update-8663864275972
port, after_update: ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find
Logical_Router_Port with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 189, in
_notify_loop
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
callback.method(resource, event, trigger, payload=payload)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/services/ovn_l3/plugin.py", line 391,
in _port_update
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
l3plugin._ovn_client.update_router_port(context,
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py",
line 1866, in update_router_port
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
self._update_lrouter_port(context, port, if_exists=if_exists,
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py",
line 1843, in _update_lrouter_port
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
external_ids = self._nb_idl.db_get(
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 47,
in execute
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
self.run_idl(None)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 235,
in run_idl
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager record =
self.api.lookup(self.table, self.record)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
184, in lookup
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager return
self._lookup(table, record)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
235, in _lookup
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager row =
idlutils.row_by_value(self, rl.table, rl.column, record)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
114, in row_by_value
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager raise
RowNotFound(table=table, col=column, match=match)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port
with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
2025-09-17 12:04:57.142 4412 WARNING neutron_lib.context [None
req-142f99a3-4935-48fc-8286-6d8d19d1fba8 6e3294ba50944dd7b489d144d2f652ec
8933ba1c7be94c51b89d44e2843592c8 - - 6b4f95d250fe4dd08b9a4c23e61cce88
6b4f95d250fe4dd08b9a4c23e61cce88] Method 'is_advsvc' is deprecated since 2023.2
release (neutron-lib 3.8.0) and will be removed once support for the old RBAC
API policies will be removed from Neutron. Please use method 'is_service_role'
instead.
2025-09-17 12:04:57.206 4411 INFO neutron.db.ovn_revision_numbers_db [None
req-dc2f485e-6938-4c4c-a776-91cb1578809d 3d1d518e2e534286a94a3ad73e9ec00a
900e2c26f77041d19163704cbd8243f9 - - 6b4f95d250fe4dd08b9a4c23e61cce88
6b4f95d250fe4dd08b9a4c23e61cce88] Successfully bum
==============================================================================================
VMs are created sucessfully and private network works as expected, VMs talk to
each other on the private network, private gateway is also reachable.
However, traffic don't go through FIP gateway, VMs are not accessible
through their FIP and they can't talk outside the private network.
We have a test project with a private network and we had the same issue
and we found out that we were missing a HA_Chassis_Group in the OVN NB
database. As a manual workaround, we needed to apply those commands :
==============================================================================================
# 4cfa62ce-ba4b-4d9d-a1db-4133e962b542 is the port associated to the router
gateway
sudo ovn-nbctl lrp-set-ha-chassis-group lrp-4cfa62ce-
ba4b-4d9d-a1db-4133e962b542 hcg-fip
HCG=$(sudo ovn-nbctl --db=unix:/var/run/ovn/ovnnb_db.sock --bare
--columns=_uuid find HA_Chassis_Group name=hcg-fip)
sudo ovn-nbctl --db=unix:/var/run/ovn/ovnnb_db.sock set Logical_Router_Port
lrp-4cfa62ce-ba4b-4d9d-a1db-4133e962b542 ha_chassis_group=$HCG
==============================================================================================
Once this was applied, our test VMs were reachable through their FIP.
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2124269
Title:
Can't bind an HA router_gateway port to chassis
Status in neutron:
New
Bug description:
Running Openstack 2024.1 "Caracal" on Ubuntu Jammy deployed with FCE
(MaaS 3.6, Juju 3.6) and OVN 24.03, we're facing an issue when using
private network+FIP based tenants.
When creating such tenant, we need to create a router with a gateway subnet.
When this object is created we see these error messages in neutron logs :
==============================================================================================
2025-09-17 12:04:57.099 4411 INFO neutron.db.ovn_revision_numbers_db [None
req-dc2f485e-6938-4c4c-a776-91cb1578809d 3d1d518e2e534286a94a3ad73e9ec00a
900e2c26f77041d19163704cbd8243f9 - - 6b4f95d250fe4dd08b9a4c23e61cce88
6b4f95d250fe4dd08b9a4c23e61cce88] Successfully bumped revision number for
resource 0626d085-bfb4-4019-8ff5-b8dd8259bf0d (type: ports) to 2
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command [None
req-541af44e-4c9f-4e19-88ff-b777bf201710 - - - - - -] Error executing command
(DbGetCommand): ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find
Logical_Router_Port with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command Traceback
(most recent call last):
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 47,
in execute
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
self.run_idl(None)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 235,
in run_idl
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
record = self.api.lookup(self.table, self.record)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
184, in lookup
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
return self._lookup(table, record)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
235, in _lookup
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command row =
idlutils.row_by_value(self, rl.table, rl.column, record)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
114, in row_by_value
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command raise
RowNotFound(table=table, col=column, match=match)
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port
with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.110 4413 ERROR ovsdbapp.backend.ovs_idl.command
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager [None
req-541af44e-4c9f-4e19-88ff-b777bf201710 - - - - - -] Error during notification
for neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._port_update-8663864275972
port, after_update: ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find
Logical_Router_Port with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager Traceback
(most recent call last):
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 189, in
_notify_loop
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
callback.method(resource, event, trigger, payload=payload)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/services/ovn_l3/plugin.py", line 391,
in _port_update
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
l3plugin._ovn_client.update_router_port(context,
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py",
line 1866, in update_router_port
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
self._update_lrouter_port(context, port, if_exists=if_exists,
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py",
line 1843, in _update_lrouter_port
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
external_ids = self._nb_idl.db_get(
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 47,
in execute
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
self.run_idl(None)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/command.py", line 235,
in run_idl
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager record =
self.api.lookup(self.table, self.record)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
184, in lookup
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager return
self._lookup(table, record)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/__init__.py", line
235, in _lookup
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager row =
idlutils.row_by_value(self, rl.table, rl.column, record)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager File
"/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line
114, in row_by_value
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager raise
RowNotFound(table=table, col=column, match=match)
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router_Port
with name=lrp-e9c93a54-539e-4f1e-97b5-be16943c7c6f
2025-09-17 12:04:57.111 4413 ERROR neutron_lib.callbacks.manager
2025-09-17 12:04:57.142 4412 WARNING neutron_lib.context [None
req-142f99a3-4935-48fc-8286-6d8d19d1fba8 6e3294ba50944dd7b489d144d2f652ec
8933ba1c7be94c51b89d44e2843592c8 - - 6b4f95d250fe4dd08b9a4c23e61cce88
6b4f95d250fe4dd08b9a4c23e61cce88] Method 'is_advsvc' is deprecated since 2023.2
release (neutron-lib 3.8.0) and will be removed once support for the old RBAC
API policies will be removed from Neutron. Please use method 'is_service_role'
instead.
2025-09-17 12:04:57.206 4411 INFO neutron.db.ovn_revision_numbers_db [None
req-dc2f485e-6938-4c4c-a776-91cb1578809d 3d1d518e2e534286a94a3ad73e9ec00a
900e2c26f77041d19163704cbd8243f9 - - 6b4f95d250fe4dd08b9a4c23e61cce88
6b4f95d250fe4dd08b9a4c23e61cce88] Successfully bum
==============================================================================================
VMs are created sucessfully and private network works as expected, VMs talk
to each other on the private network, private gateway is also reachable.
However, traffic don't go through FIP gateway, VMs are not accessible
through their FIP and they can't talk outside the private network.
We have a test project with a private network and we had the same
issue and we found out that we were missing a HA_Chassis_Group in the
OVN NB database. As a manual workaround, we needed to apply those
commands :
==============================================================================================
# 4cfa62ce-ba4b-4d9d-a1db-4133e962b542 is the port associated to the router
gateway
sudo ovn-nbctl lrp-set-ha-chassis-group lrp-4cfa62ce-
ba4b-4d9d-a1db-4133e962b542 hcg-fip
HCG=$(sudo ovn-nbctl --db=unix:/var/run/ovn/ovnnb_db.sock --bare
--columns=_uuid find HA_Chassis_Group name=hcg-fip)
sudo ovn-nbctl --db=unix:/var/run/ovn/ovnnb_db.sock set Logical_Router_Port
lrp-4cfa62ce-ba4b-4d9d-a1db-4133e962b542 ha_chassis_group=$HCG
==============================================================================================
Once this was applied, our test VMs were reachable through their FIP.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2124269/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
