Public bug reported: Originally reported in the Octavia launchpad: https://bugs.launchpad.net/octavia/+bug/2028524
The commit https://review.opendev.org/c/openstack/neutron/+/882588 introduced a regression in Octavia It adds a validate_port_binding_and_virtual_port function that raises an exception when a port: - has non-empty binding:host_id - has fixed_ips/subnets - has VIRTUAL type (in ovn) When we create a load balancer in Octavia (with an IPv6 VIP) $ openstack loadbalancer create --vip-subnet ipv6-public-subnet --name lb1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | admin_state_up | True | | availability_zone | None | | created_at | 2023-07-25T07:11:25 | | description | | | flavor_id | None | | id | 75cf51d2-4576-4878-8bfe-ad55584a7d76 | | listeners | | | name | lb1 | | operating_status | OFFLINE | | pools | | | project_id | 86f57e2e56874381a0d586263fc8d900 | | provider | amphora | | provisioning_status | PENDING_CREATE | | updated_at | None | | vip_address | 2001:db8::b1 | | vip_network_id | 2d16ac53-8438-435d-a787-e5ceb4b783be | | vip_port_id | 83e51017-8f02-4916-bcd2-ebe0475b1ce6 | | vip_qos_policy_id | None | | vip_subnet_id | 813adce0-21de-44c9-958a-6967441b8623 | | tags | | | additional_vips | [] | +---------------------+--------------------------------------+ The VIP port contains: $ openstack port show 83e51017-8f02-4916-bcd2-ebe0475b1ce6 +-------------------------+--------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+--------------------------------------------------------------------------------------------------------+ | admin_state_up | DOWN | | allowed_address_pairs | | | binding_host_id | gthiemon-devstack | | binding_profile | | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | normal | | created_at | 2023-07-25T07:11:25Z | | data_plane_status | None | | description | | | device_id | lb-75cf51d2-4576-4878-8bfe-ad55584a7d76 | | device_owner | Octavia | | device_profile | None | | dns_assignment | fqdn='host-2001-db8--b1.openstackgate.local.', hostname='host-2001-db8--b1', ip_address='2001:db8::b1' | | dns_domain | | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='2001:db8::b1', subnet_id='813adce0-21de-44c9-958a-6967441b8623' | | id | 83e51017-8f02-4916-bcd2-ebe0475b1ce6 | | ip_allocation | None | | mac_address | fa:16:3e:c9:4f:7e | | name | octavia-lb-75cf51d2-4576-4878-8bfe-ad55584a7d76 | | network_id | 2d16ac53-8438-435d-a787-e5ceb4b783be | | numa_affinity_policy | None | | port_security_enabled | True | | project_id | 86f57e2e56874381a0d586263fc8d900 | | propagate_uplink_status | None | | qos_network_policy_id | None | | qos_policy_id | None | | resource_request | None | | revision_number | 10 | | security_group_ids | 7c8d8935-9445-4e74-a815-a24246af757a | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2023-07-25T07:12:14Z | +-------------------------+--------------------------------------------------------------------------------------------------------+ The port is not bound and has a binding_host_id, has a fixed_ips with a subnet and there's another port that has an allowed_address_pair with the VIP port's allocated address (so the port is a virtual port in OVN) Any updates of this port result in a BadRequest Exception: Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow Traceback (most recent call last): Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/octavia/octavia/network/drivers/neutron/base.py", line 129, in _add_security_group_to_port Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow self.network_proxy.update_port( Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/network/v2/_proxy.py", line 2979, in update_port Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return self._update(_port.Port, port, if_revision=if_revision, **attrs) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/proxy.py", line 64, in check Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return method(self, expected, actual, *args, **kwargs) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/network/v2/_proxy.py", line 189, in _update Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return res.commit(self, base_path=base_path, if_revision=if_revision) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/resource.py", line 1794, in commit Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return self._commit( Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/resource.py", line 1839, in _commit Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow self._translate_response(response, has_body=has_body) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/resource.py", line 1278, in _translate_response Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow exceptions.raise_from_response(response, error_message=error_message) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/exceptions.py", line 263, in raise_from_response Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow raise cls( Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: http://192.168.1.101:9696/networking/v2.0/ports/618567c4-78c7-4398-b889-b567f6fd6aeb, Bad port request: A virtual logical switch port cannot be bound to a host. The goal of this validation function seems to raise an exception when the binding_host_id is not empty, but the PortBindingUpdateVirtualPortsEvent class sets the binding_host_id of virtual ports. https://opendev.org/openstack/neutron/src/commit/58c8493ff9defbb4544803ec3fc0432c0685c592/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L532-L537 Interestingly, it's not 100% reproducible (around 90%) and it is not reproducible with IPv4 VIPs, with IPv4 ports, the binding_host_id is always empty. I have a couple of questions: - is the validation function correct? should the binding_host_id be empty for VIP ports? - why is binding_host_id set for IPv6 VIPs but not for IPv4 VIPs? I can provide more logs if needed ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2028651 Title: IPv6 VIPs broken with ML2/OVN Status in neutron: New Bug description: Originally reported in the Octavia launchpad: https://bugs.launchpad.net/octavia/+bug/2028524 The commit https://review.opendev.org/c/openstack/neutron/+/882588 introduced a regression in Octavia It adds a validate_port_binding_and_virtual_port function that raises an exception when a port: - has non-empty binding:host_id - has fixed_ips/subnets - has VIRTUAL type (in ovn) When we create a load balancer in Octavia (with an IPv6 VIP) $ openstack loadbalancer create --vip-subnet ipv6-public-subnet --name lb1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | admin_state_up | True | | availability_zone | None | | created_at | 2023-07-25T07:11:25 | | description | | | flavor_id | None | | id | 75cf51d2-4576-4878-8bfe-ad55584a7d76 | | listeners | | | name | lb1 | | operating_status | OFFLINE | | pools | | | project_id | 86f57e2e56874381a0d586263fc8d900 | | provider | amphora | | provisioning_status | PENDING_CREATE | | updated_at | None | | vip_address | 2001:db8::b1 | | vip_network_id | 2d16ac53-8438-435d-a787-e5ceb4b783be | | vip_port_id | 83e51017-8f02-4916-bcd2-ebe0475b1ce6 | | vip_qos_policy_id | None | | vip_subnet_id | 813adce0-21de-44c9-958a-6967441b8623 | | tags | | | additional_vips | [] | +---------------------+--------------------------------------+ The VIP port contains: $ openstack port show 83e51017-8f02-4916-bcd2-ebe0475b1ce6 +-------------------------+--------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+--------------------------------------------------------------------------------------------------------+ | admin_state_up | DOWN | | allowed_address_pairs | | | binding_host_id | gthiemon-devstack | | binding_profile | | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | normal | | created_at | 2023-07-25T07:11:25Z | | data_plane_status | None | | description | | | device_id | lb-75cf51d2-4576-4878-8bfe-ad55584a7d76 | | device_owner | Octavia | | device_profile | None | | dns_assignment | fqdn='host-2001-db8--b1.openstackgate.local.', hostname='host-2001-db8--b1', ip_address='2001:db8::b1' | | dns_domain | | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='2001:db8::b1', subnet_id='813adce0-21de-44c9-958a-6967441b8623' | | id | 83e51017-8f02-4916-bcd2-ebe0475b1ce6 | | ip_allocation | None | | mac_address | fa:16:3e:c9:4f:7e | | name | octavia-lb-75cf51d2-4576-4878-8bfe-ad55584a7d76 | | network_id | 2d16ac53-8438-435d-a787-e5ceb4b783be | | numa_affinity_policy | None | | port_security_enabled | True | | project_id | 86f57e2e56874381a0d586263fc8d900 | | propagate_uplink_status | None | | qos_network_policy_id | None | | qos_policy_id | None | | resource_request | None | | revision_number | 10 | | security_group_ids | 7c8d8935-9445-4e74-a815-a24246af757a | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2023-07-25T07:12:14Z | +-------------------------+--------------------------------------------------------------------------------------------------------+ The port is not bound and has a binding_host_id, has a fixed_ips with a subnet and there's another port that has an allowed_address_pair with the VIP port's allocated address (so the port is a virtual port in OVN) Any updates of this port result in a BadRequest Exception: Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow Traceback (most recent call last): Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/octavia/octavia/network/drivers/neutron/base.py", line 129, in _add_security_group_to_port Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow self.network_proxy.update_port( Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/network/v2/_proxy.py", line 2979, in update_port Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return self._update(_port.Port, port, if_revision=if_revision, **attrs) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/proxy.py", line 64, in check Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return method(self, expected, actual, *args, **kwargs) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/network/v2/_proxy.py", line 189, in _update Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return res.commit(self, base_path=base_path, if_revision=if_revision) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/resource.py", line 1794, in commit Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow return self._commit( Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/resource.py", line 1839, in _commit Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow self._translate_response(response, has_body=has_body) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/resource.py", line 1278, in _translate_response Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow exceptions.raise_from_response(response, error_message=error_message) Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow File "/opt/stack/openstacksdk/openstack/exceptions.py", line 263, in raise_from_response Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow raise cls( Jul 24 03:08:07 gthiemon-devstack octavia-worker[97901]: ERROR octavia.common.base_taskflow openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: http://192.168.1.101:9696/networking/v2.0/ports/618567c4-78c7-4398-b889-b567f6fd6aeb, Bad port request: A virtual logical switch port cannot be bound to a host. The goal of this validation function seems to raise an exception when the binding_host_id is not empty, but the PortBindingUpdateVirtualPortsEvent class sets the binding_host_id of virtual ports. https://opendev.org/openstack/neutron/src/commit/58c8493ff9defbb4544803ec3fc0432c0685c592/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#L532-L537 Interestingly, it's not 100% reproducible (around 90%) and it is not reproducible with IPv4 VIPs, with IPv4 ports, the binding_host_id is always empty. I have a couple of questions: - is the validation function correct? should the binding_host_id be empty for VIP ports? - why is binding_host_id set for IPv6 VIPs but not for IPv4 VIPs? I can provide more logs if needed To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2028651/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
