Public bug reported: Recently I ran into an issue that instances are unable to get metadata, after a few debugs I found out the issue to be the LB health monitor is replying the arp request for OVN metadata agent's IP causing the request going to metadata agent actually being sent to another MAC address.
Seems to related to https://bugs.launchpad.net/bugs/1956034, but that bug is closed by mistake. Form sb_db logical flow dump I found the bellow flow _uuid : df93e919-c7f5-4d04-be6f-796e175e266f actions : "eth.dst = eth.src; eth.src = 5e:69:e7:47:38:7b; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 5e:69:e7:47:38:7b; arp.tpa = arp.spa; arp.spa = 10.0.0.2; outport = inport; flags.loopback = 1; output;" controller_meter : [] external_ids : {source="northd.c:7971", stage-hint="5b11468a", stage-name=ls_in_arp_rsp} logical_datapath : f714c506-3043-4366-8065-d6af4d2a08de logical_dp_group : [] match : "arp.tpa == 10.0.0.2 && arp.op == 1" pipeline : ingress priority : 110 table_id : 18 tags : {} hash : 0 Priority 110 flow with this action should be from the LB health monitor according to ovn-northd doc [1] Where as ovn-metadata agent also uses the same IP (10.0.0.2) +-------------------------+---------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | | | binding_profile | | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | normal | | created_at | 2023-01-17T08:04:35Z | | data_plane_status | None | | description | | | device_id | ovnmeta-5640d643-e490-46b3-a724-0478aef3722e | | device_owner | network:distributed | | device_profile | None | | dns_assignment | fqdn='host-10-0-0-2.infra.cloudnative.tw.', hostname='host-10-0-0-2', ip_address='10.0.0.2' | | dns_domain | | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.0.2', subnet_id='3000b8bf-f687-406d-b000-55d38f365e33' | | id | c1ba8ceb-6f02-48e6-9da3-3cb9b628b771 | | ip_allocation | None | | mac_address | fa:16:3e:7c:b8:1c | | name | | | network_id | 5640d643-e490-46b3-a724-0478aef3722e | | numa_affinity_policy | None | | port_security_enabled | False | | project_id | 2fbc86f895ed4ef1ad036b7e4a068b50 | | propagate_uplink_status | None | | qos_network_policy_id | None | | qos_policy_id | None | | resource_request | None | | revision_number | 2 | | security_group_ids | | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2023-01-17T08:04:36Z | +-------------------------+---------------------------------------------------------------------------------------------+ This causes all packet that should be going to 10.0.0.2 (metadata agent) actually being sent to 5e:69:e7:47:38:7b MAC address which is incorrect. Expected result: OVN LB health monitor sending arp replies to an unused IP, or reserved a port with an IP specifically for health monitor. [1] https://www.ovn.org/support/dist-docs/ovn-northd.8.html ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2004238 Title: OVN metadata use the same IP as LB health monitor Status in neutron: New Bug description: Recently I ran into an issue that instances are unable to get metadata, after a few debugs I found out the issue to be the LB health monitor is replying the arp request for OVN metadata agent's IP causing the request going to metadata agent actually being sent to another MAC address. Seems to related to https://bugs.launchpad.net/bugs/1956034, but that bug is closed by mistake. Form sb_db logical flow dump I found the bellow flow _uuid : df93e919-c7f5-4d04-be6f-796e175e266f actions : "eth.dst = eth.src; eth.src = 5e:69:e7:47:38:7b; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 5e:69:e7:47:38:7b; arp.tpa = arp.spa; arp.spa = 10.0.0.2; outport = inport; flags.loopback = 1; output;" controller_meter : [] external_ids : {source="northd.c:7971", stage-hint="5b11468a", stage-name=ls_in_arp_rsp} logical_datapath : f714c506-3043-4366-8065-d6af4d2a08de logical_dp_group : [] match : "arp.tpa == 10.0.0.2 && arp.op == 1" pipeline : ingress priority : 110 table_id : 18 tags : {} hash : 0 Priority 110 flow with this action should be from the LB health monitor according to ovn-northd doc [1] Where as ovn-metadata agent also uses the same IP (10.0.0.2) +-------------------------+---------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+---------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | | | binding_profile | | | binding_vif_details | | | binding_vif_type | unbound | | binding_vnic_type | normal | | created_at | 2023-01-17T08:04:35Z | | data_plane_status | None | | description | | | device_id | ovnmeta-5640d643-e490-46b3-a724-0478aef3722e | | device_owner | network:distributed | | device_profile | None | | dns_assignment | fqdn='host-10-0-0-2.infra.cloudnative.tw.', hostname='host-10-0-0-2', ip_address='10.0.0.2' | | dns_domain | | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.0.2', subnet_id='3000b8bf-f687-406d-b000-55d38f365e33' | | id | c1ba8ceb-6f02-48e6-9da3-3cb9b628b771 | | ip_allocation | None | | mac_address | fa:16:3e:7c:b8:1c | | name | | | network_id | 5640d643-e490-46b3-a724-0478aef3722e | | numa_affinity_policy | None | | port_security_enabled | False | | project_id | 2fbc86f895ed4ef1ad036b7e4a068b50 | | propagate_uplink_status | None | | qos_network_policy_id | None | | qos_policy_id | None | | resource_request | None | | revision_number | 2 | | security_group_ids | | | status | DOWN | | tags | | | trunk_details | None | | updated_at | 2023-01-17T08:04:36Z | +-------------------------+---------------------------------------------------------------------------------------------+ This causes all packet that should be going to 10.0.0.2 (metadata agent) actually being sent to 5e:69:e7:47:38:7b MAC address which is incorrect. Expected result: OVN LB health monitor sending arp replies to an unused IP, or reserved a port with an IP specifically for health monitor. [1] https://www.ovn.org/support/dist-docs/ovn-northd.8.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2004238/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : [email protected] Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
