On Fri, Jul 10, 2020 at 4:41 PM Numan Siddique <[email protected]> wrote:
> > > On Fri, Jul 10, 2020 at 12:45 AM Ankur Sharma <[email protected]> > wrote: > >> Hi Numan, Daniel, >> >> I have not looked at the patch yet. But replacing arp.sha with chassis >> mac is not the correct approach from networking perspective. >> Chassic mac is NOT meant to replace the IP-MAC binding of router port, it >> is ONLY meant to ensure that for EW traffic a distributed router port mac >> does not show on multiple TOR ports. >> Both for NS and EW, ARP resolution for router port ip should be responded >> with router port mac ONLY. >> >> I am trying to understand the use case and we can discuss an alternative >> in this thread. >> Can you share the repro steps, i can try the same and will try to come up >> with an alternative. >> >> > Hi Ankur, > > In this particular case, the originator of the traffic is from a logical > port of type 'external'. > > One example of using external ports is for SRIOV VMs. The traffic from > these VMs are not seen > by the local ovn-controller. And we want to provide E-W routing and other > OVN services like DHCP, DNS etc > to these VMS. > > So one of the controller nodes (which can receive the traffic sent by > these SRIOV VMs) binds these external ports > and it responds to the ARP requests and does the routing for it. > > To reproduce the issue, can you please use own-fake-multi node setup from > here ? - > https://github.com/numansiddique/ovn-fake-multinode/tree/vlan_chassis_mac_issue > > The steps are: > 1. Build OVN containers. > ./ovn_cluster.sh build > > Please note, before the 'start', you need to start openvswitch on the host. Thanks Numan > 2. ./ovn_cluster.sh start > > Run > 3. sudo ip netns exec sw0-ext1 ping -c3 20.0.0.3 > PING 20.0.0.3 (20.0.0.3) 56(84) bytes of data. > 64 bytes from 20.0.0.3: icmp_seq=1 ttl=63 time=0.074 ms > 64 bytes from 20.0.0.3: icmp_seq=1 ttl=63 time=0.086 ms (DUP!) > 64 bytes from 20.0.0.3: icmp_seq=1 ttl=63 time=0.089 ms (DUP!) > 64 bytes from 20.0.0.3: icmp_seq=2 ttl=63 time=0.105 ms > 64 bytes from 20.0.0.3: icmp_seq=2 ttl=63 time=0.120 ms (DUP!) > 64 bytes from 20.0.0.3: icmp_seq=2 ttl=63 time=0.124 ms (DUP!) > 64 bytes from 20.0.0.3: icmp_seq=3 ttl=63 time=0.145 ms > > --- 20.0.0.3 ping statistics --- > 3 packets transmitted, 3 received, +4 duplicates, 0% packet loss, time > 2036ms > rtt min/avg/max/mdev = 0.074/0.106/0.145/0.023 ms > > You will see a few DUP packets. > > $sudo ip netns exec sw0-ext1 ping -c3 10.0.0.1 > PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. > 64 bytes from 10.0.0.1: icmp_seq=1 ttl=254 time=0.298 ms > 64 bytes from 10.0.0.1: icmp_seq=1 ttl=254 time=0.358 ms (DUP!) > 64 bytes from 10.0.0.1: icmp_seq=1 ttl=254 time=0.384 ms (DUP!) > 64 bytes from 10.0.0.1: icmp_seq=2 ttl=254 time=0.598 ms > 64 bytes from 10.0.0.1: icmp_seq=2 ttl=254 time=0.594 ms (DUP!) > 64 bytes from 10.0.0.1: icmp_seq=2 ttl=254 time=0.656 ms (DUP!) > 64 bytes from 10.0.0.1: icmp_seq=3 ttl=254 time=0.715 ms > > --- 10.0.0.1 ping statistics --- > 3 packets transmitted, 3 received, +4 duplicates, 0% packet loss, time > 2088ms > rtt min/avg/max/mdev = 0.298/0.514/0.715/0.152 ms > > In the setup, sw0-ext1 represents an external logical switch port. If you > see the script here [1], > sw0-ext1 is claimed by ovn-chassis-1 node. > > And when sw0-ext1 sends ARP request to 10.0.0.1, the arp request is > handled by ovn-chassis-1 > and the reply has - arp.sha = router mac and eth.src = chassis mac of > ovn-chassis-1. > > And hence sw0-ext1 sends ping packets with the destination mac of router > port IP - 10.0.0.1. > And all the 3 nodes reply - ovn-chassis-1, ovn-chassis-2 and ovn-gw-1. > > I'm not sure if you have played with ovn-fake-multinode before. If you run > "docker ps", you will see a docker > container representing each chassis. > > Please do "docker exec -it ovn-central bash" and run a few > ovn-nbctl/ovn-sbctl commands to know more. > > You can also see the script in [1] and reproduce the issue in your setup. > > I didn't find any other way to solve this issue. Also in normal situations > where external ports are not used, > any arp request to the router IP from bridge logical switch ports don't > leave the chassis since the local > ovn-controller itself replies. This is for tenant bridged VLAN logical > switches. I guess for provider VLAN networks > (which provide the N/S traffic, I guess the arp request for the router > port can come from the physical network). > > > [1] - > https://github.com/numansiddique/ovn-fake-multinode/blob/vlan_chassis_mac_issue/ovn_cluster.sh#L501 > > > Thanks > Numan > > > > Regards, >> Ankur >> ________________________________ >> From: [email protected] <[email protected]> >> Sent: Thursday, July 9, 2020 2:11 AM >> To: [email protected] <[email protected]> >> Cc: Numan Siddique <[email protected]>; Daniel Alvarez <[email protected]>; >> Ankur Sharma <[email protected]> >> Subject: [PATCH ovn v2] Fix the routing for external logical ports of >> bridged logical switches. >> >> From: Numan Siddique <[email protected]> >> >> Routing for external logical ports is broken if these ports belonged >> to bridged logical switches (with localnet port) and >> 'ovn-chassis-mac-mappings' >> is configured. External logical ports are those which are external to OVN, >> but there is a logical port for it and it is claimed by one of the HA >> chassis. >> The claimed chassis provides routing and other native OVN serices like >> dhcp and dns. >> >> When the external port sends ARP request for the router IP, the claimed >> chassis >> replies for the ARP request, but the arp.sha is set to the actual router >> mac instead >> of the chassis mac. This causes the traffic from external port >> VM/container to be handled >> incorrectly. A ping to the router ip, is replied by all the chassis which >> can see this >> packet instead of just the claimed HA chassis. >> >> To fix this, this patch does 2 things. >> >> 1. In the table - OFTABLE_LOG_TO_PHY (65), it adds a 160 priority flow to >> modify the ARP packets arp.sha to store the chassis mac. >> >> 2. And when the packet destined to the chassis mac is received, it >> replaces the >> chassis mac with the actual router mac in table 0. >> >> Reported-at: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1829762&d=DwIDAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=u_maNAEOYzfy4_tzUirBX0TdPn35ePuIddtQDl4B8fs&s=T6SxlTDjkPxA6_Lsv_KjWkOSUSfesz0LIVnovPxBXlc&e= >> Reported-by >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__bugzilla.redhat.com_show-5Fbug.cgi-3Fid-3D1829762&d=DwIDAg&c=s883GpUCOChKOHiocYtGcg&r=mZwX9gFQgeJHzTg-68aCJgsODyUEVsHGFOfL90J6MJY&m=u_maNAEOYzfy4_tzUirBX0TdPn35ePuIddtQDl4B8fs&s=T6SxlTDjkPxA6_Lsv_KjWkOSUSfesz0LIVnovPxBXlc&e=Reported-by>: >> Daniel Alvarez <[email protected]> >> CC: Ankur Sharma <[email protected]> >> Signed-off-by: Numan Siddique <[email protected]> >> --- >> >> v1 -> v2 >> ---- >> * Rebased. >> >> controller/chassis.c | 48 ++++++++------ >> controller/chassis.h | 2 + >> controller/physical.c | 145 +++++++++++++++++++++++++++++++++++++++--- >> tests/ovn.at | 131 ++++++++++++++++++++++++++++++++++++++ >> 4 files changed, 299 insertions(+), 27 deletions(-) >> >> diff --git a/controller/chassis.c b/controller/chassis.c >> index eec270ea39..25146d75f2 100644 >> --- a/controller/chassis.c >> +++ b/controller/chassis.c >> @@ -645,10 +645,11 @@ chassis_run(struct ovsdb_idl_txn *ovnsb_idl_txn, >> } >> >> bool >> -chassis_get_mac(const struct sbrec_chassis *chassis_rec, >> - const char *bridge_mapping, >> - struct eth_addr *chassis_mac) >> +chassis_get_mac_mappings(const struct sbrec_chassis *chassis_rec, >> + struct smap *chassis_mappings) >> { >> + smap_init(chassis_mappings); >> + >> const char *tokens >> = get_chassis_mac_mappings(&chassis_rec->other_config); >> if (!tokens[0]) { >> @@ -656,7 +657,6 @@ chassis_get_mac(const struct sbrec_chassis >> *chassis_rec, >> } >> >> char *save_ptr = NULL; >> - bool ret = false; >> char *tokstr = xstrdup(tokens); >> >> /* Format for a chassis mac configuration is: >> @@ -669,24 +669,36 @@ chassis_get_mac(const struct sbrec_chassis >> *chassis_rec, >> char *chassis_mac_bridge = strtok_r(token, ":", &save_ptr2); >> char *chassis_mac_str = strtok_r(NULL, "", &save_ptr2); >> >> - if (!strcmp(chassis_mac_bridge, bridge_mapping)) { >> - struct eth_addr temp_mac; >> + smap_replace(chassis_mappings, chassis_mac_bridge, >> chassis_mac_str); >> + } >> >> - /* Return the first chassis mac. */ >> - char *err_str = str_to_mac(chassis_mac_str, &temp_mac); >> - if (err_str) { >> - free(err_str); >> - continue; >> - } >> + free(tokstr); >> + return true; >> +} >> >> - ret = true; >> - *chassis_mac = temp_mac; >> - break; >> - } >> +bool >> +chassis_get_mac(const struct sbrec_chassis *chassis_rec, >> + const char *bridge_mapping, >> + struct eth_addr *chassis_mac) >> +{ >> + struct smap chassis_mappings; >> + >> + if (!chassis_get_mac_mappings(chassis_rec, &chassis_mappings)) { >> + return false; >> } >> >> - free(tokstr); >> - return ret; >> + const char *chassis_mac_str = smap_get_def(&chassis_mappings, >> + bridge_mapping, ""); >> + struct eth_addr temp_mac; >> + >> + char *err_str = str_to_mac(chassis_mac_str, &temp_mac); >> + if (err_str) { >> + free(err_str); >> + return false; >> + } >> + >> + *chassis_mac = temp_mac; >> + return true; >> } >> >> /* Returns true if the database is all cleaned up, false if more work is >> diff --git a/controller/chassis.h b/controller/chassis.h >> index 178d2957e8..dae761312d 100644 >> --- a/controller/chassis.h >> +++ b/controller/chassis.h >> @@ -42,6 +42,8 @@ bool chassis_cleanup(struct ovsdb_idl_txn >> *ovnsb_idl_txn, >> bool chassis_get_mac(const struct sbrec_chassis *chassis, >> const char *bridge_mapping, >> struct eth_addr *chassis_mac); >> +bool chassis_get_mac_mappings(const struct sbrec_chassis *, >> + struct smap *chassis_mappings); >> const char *chassis_get_id(void); >> const char * get_chassis_mac_mappings(const struct smap *ext_ids); >> >> diff --git a/controller/physical.c b/controller/physical.c >> index 6d7d8e93bc..b43a157b94 100644 >> --- a/controller/physical.c >> +++ b/controller/physical.c >> @@ -62,7 +62,8 @@ load_logical_ingress_metadata(const struct >> sbrec_port_binding *binding, >> /* UUID to identify OF flows not associated with ovsdb rows. */ >> static struct uuid *hc_uuid = NULL; >> >> -#define CHASSIS_MAC_TO_ROUTER_MAC_CONJID 100 >> +#define CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID 100 >> +#define CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID 101 >> >> void >> physical_register_ovs_idl(struct ovsdb_idl *ovs_idl) >> @@ -148,6 +149,18 @@ put_move(enum mf_field_id src, int src_ofs, >> move->dst.n_bits = n_bits; >> } >> >> +static void >> +put_value(const uint8_t *data, size_t len, >> + enum mf_field_id dst, int ofs, int n_bits, >> + struct ofpbuf *ofpacts) >> +{ >> + struct ofpact_set_field *sf = ofpact_put_set_field(ofpacts, >> + mf_from_id(dst), >> NULL, >> + NULL); >> + bitwise_copy(data, len, 0, sf->value, sf->field->n_bytes, ofs, >> n_bits); >> + bitwise_one(ofpact_set_field_mask(sf), sf->field->n_bytes, ofs, >> n_bits); >> +} >> + >> static void >> put_resubmit(uint8_t table_id, struct ofpbuf *ofpacts) >> { >> @@ -494,11 +507,10 @@ put_chassis_mac_conj_id_flow(const struct >> sbrec_chassis_table *chassis_table, >> ofpbuf_clear(ofpacts_p); >> match_init_catchall(&match); >> >> - >> match_set_dl_src(&match, chassis_mac); >> >> conj = ofpact_put_CONJUNCTION(ofpacts_p); >> - conj->id = CHASSIS_MAC_TO_ROUTER_MAC_CONJID; >> + conj->id = CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID; >> conj->n_clauses = 2; >> conj->clause = 0; >> ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180, >> @@ -507,6 +519,51 @@ put_chassis_mac_conj_id_flow(const struct >> sbrec_chassis_table *chassis_table, >> } >> >> free_remote_chassis_macs(); >> + >> + /* We need to replace the packet destined to the chassis mac >> (eth.dst) >> + * with the router mac. This is required to support external ports. >> + * These ports don't see the router mac at all since we send the >> + * chassis MAC in the ARP reply for any ARP requests to the router >> IPs. >> + * Without these flows, the packets will not enter the router >> pipeline >> + * if they need to be routed. >> + * Please see put_replace_chassis_mac_flows() for the 2nd clause of >> + * conj id - CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID. >> + * */ >> + struct smap chassis_mac_mappings = >> SMAP_INITIALIZER(&chassis_mac_mappings); >> + if (chassis_get_mac_mappings(chassis, &chassis_mac_mappings)) { >> + struct smap_node *node; >> + struct sset macs = SSET_INITIALIZER(&macs); >> + SMAP_FOR_EACH (node, &chassis_mac_mappings) { >> + struct eth_addr chassis_mac; >> + >> + char *err_str = str_to_mac(node->value, &chassis_mac); >> + if (err_str) { >> + free(err_str); >> + continue; >> + } >> + >> + if (!sset_add(&macs, node->value)) { >> + /* The OF flow for the mac is already added. */ >> + continue; >> + } >> + >> + ofpbuf_clear(ofpacts_p); >> + match_init_catchall(&match); >> + >> + match_set_dl_dst(&match, chassis_mac); >> + >> + struct ofpact_conjunction *conj; >> + conj = ofpact_put_CONJUNCTION(ofpacts_p); >> + conj->id = CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID; >> + conj->n_clauses = 2; >> + conj->clause = 0; >> + ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180, >> + 0, &match, ofpacts_p, hc_uuid); >> + } >> + sset_destroy(&macs); >> + } >> + >> + smap_destroy(&chassis_mac_mappings); >> } >> >> static void >> @@ -555,7 +612,7 @@ put_replace_chassis_mac_flows(const struct simap >> *ct_zones, >> >> /* Match on ingress port, vlan_id and conjunction id */ >> match_set_in_port(&match, ofport); >> - match_set_conj_id(&match, CHASSIS_MAC_TO_ROUTER_MAC_CONJID); >> + match_set_conj_id(&match, CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID); >> >> if (tag) { >> match_set_dl_vlan(&match, htons(tag), 0); >> @@ -572,6 +629,37 @@ put_replace_chassis_mac_flows(const struct simap >> *ct_zones, >> replace_mac = ofpact_put_SET_ETH_SRC(ofpacts_p); >> replace_mac->mac = router_port_mac; >> >> + /* Resubmit to first logical ingress pipeline table. */ >> + put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p); >> + ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180, >> + rport_binding->header_.uuid.parts[0], >> + &match, ofpacts_p, hc_uuid); >> + >> + ofpbuf_clear(ofpacts_p); >> + match_init_catchall(&match); >> + >> + /* Add flow, which will match on conjunction id and will >> + * replace destination mac with router port mac */ >> + >> + /* Match on ingress port, vlan_id and conjunction id */ >> + match_set_in_port(&match, ofport); >> + match_set_conj_id(&match, CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID); >> + >> + if (tag) { >> + match_set_dl_vlan(&match, htons(tag), 0); >> + } else { >> + match_set_dl_tci_masked(&match, 0, htons(VLAN_CFI)); >> + } >> + >> + /* Actions */ >> + >> + if (tag) { >> + ofpact_put_STRIP_VLAN(ofpacts_p); >> + } >> + load_logical_ingress_metadata(localnet_port, &zone_ids, >> ofpacts_p); >> + replace_mac = ofpact_put_SET_ETH_DST(ofpacts_p); >> + replace_mac->mac = router_port_mac; >> + >> /* Resubmit to first logical ingress pipeline table. */ >> put_resubmit(OFTABLE_LOG_INGRESS_PIPELINE, ofpacts_p); >> ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180, >> @@ -579,7 +667,7 @@ put_replace_chassis_mac_flows(const struct simap >> *ct_zones, >> &match, ofpacts_p, hc_uuid); >> >> /* Provide second search criteria, i.e localnet port's >> - * vlan ID for conjunction flow */ >> + * vlan ID for conjunction flows. */ >> struct ofpact_conjunction *conj; >> ofpbuf_clear(ofpacts_p); >> match_init_catchall(&match); >> @@ -591,12 +679,19 @@ put_replace_chassis_mac_flows(const struct simap >> *ct_zones, >> } >> >> conj = ofpact_put_CONJUNCTION(ofpacts_p); >> - conj->id = CHASSIS_MAC_TO_ROUTER_MAC_CONJID; >> + conj->id = CHASSIS_MAC_TO_ROUTER_SRC_MAC_CONJID; >> + conj->n_clauses = 2; >> + conj->clause = 1; >> + >> + conj = ofpact_put_CONJUNCTION(ofpacts_p); >> + conj->id = CHASSIS_MAC_TO_ROUTER_DST_MAC_CONJID; >> conj->n_clauses = 2; >> conj->clause = 1; >> + >> ofctrl_add_flow(flow_table, OFTABLE_PHY_TO_LOG, 180, >> rport_binding->header_.uuid.parts[0], >> &match, ofpacts_p, hc_uuid); >> + >> } >> } >> >> @@ -665,9 +760,6 @@ put_replace_router_port_mac_flows(struct >> ovsdb_idl_index >> * a. Flow replaces ingress router port mac with a chassis mac. >> * b. Flow appends the vlan id localnet port is configured with. >> */ >> - match_init_catchall(&match); >> - ofpbuf_clear(ofpacts_p); >> - >> ovs_assert(rport_binding->n_mac == 1); >> char *err_str = str_to_mac(rport_binding->mac[0], >> &router_port_mac); >> if (err_str) { >> @@ -679,6 +771,9 @@ put_replace_router_port_mac_flows(struct >> ovsdb_idl_index >> } >> >> /* Replace Router mac flow */ >> + match_init_catchall(&match); >> + ofpbuf_clear(ofpacts_p); >> + >> match_set_metadata(&match, htonll(dp_key)); >> match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key); >> match_set_dl_src(&match, router_port_mac); >> @@ -698,6 +793,38 @@ put_replace_router_port_mac_flows(struct >> ovsdb_idl_index >> ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 150, >> localnet_port->header_.uuid.parts[0], >> &match, ofpacts_p, &localnet_port->header_.uuid); >> + >> + /* Replace Router mac in the ARP packets (arp.sha) to the >> chassis MAC. >> + * This is very important and required for external logical >> ports and >> + * when these ports send ARP for their router IPs, the chassis >> mac >> + * should be sent which has claimed these external ports. */ >> + match_init_catchall(&match); >> + ofpbuf_clear(ofpacts_p); >> + >> + match_set_metadata(&match, htonll(dp_key)); >> + match_set_reg(&match, MFF_LOG_OUTPORT - MFF_REG0, port_key); >> + match_set_dl_src(&match, router_port_mac); >> + match_set_dl_type(&match, htons(ETH_TYPE_ARP)); >> + match_set_arp_sha(&match, router_port_mac); >> + >> + replace_mac = ofpact_put_SET_ETH_SRC(ofpacts_p); >> + replace_mac->mac = chassis_mac; >> + >> + if (tag) { >> + struct ofpact_vlan_vid *vlan_vid; >> + vlan_vid = ofpact_put_SET_VLAN_VID(ofpacts_p); >> + vlan_vid->vlan_vid = tag; >> + vlan_vid->push_vlan_if_needed = true; >> + } >> + >> + put_value(chassis_mac.ea, sizeof chassis_mac.ea, MFF_ARP_SHA, >> + 0, 48, ofpacts_p); >> + >> + ofpact_put_OUTPUT(ofpacts_p)->port = ofport; >> + >> + ofctrl_add_flow(flow_table, OFTABLE_LOG_TO_PHY, 160, >> + localnet_port->header_.uuid.parts[0], >> + &match, ofpacts_p, &localnet_port->header_.uuid); >> } >> } >> >> diff --git a/tests/ovn.at b/tests/ovn.at >> index 24d93bc245..f033401410 100644 >> --- a/tests/ovn.at >> +++ b/tests/ovn.at >> @@ -14748,6 +14748,137 @@ AT_CHECK([cat ext1_v6.packets | cut -c -120], >> [0], [expout]) >> cat ext1_v6.expected | cut -c 125- > expout >> AT_CHECK([cat ext1_v6.packets | cut -c 125-], [0], [expout]) >> >> +# Configure ovn-chassis-mac-mappings on all the hypervisors. >> +as hv1 >> +ovs-vsctl set open . >> external_ids:ovn-chassis-mac-mappings=phys:1e:02:ad:aa:bb:01 >> + >> +as hv2 >> +ovs-vsctl set open . >> external_ids:ovn-chassis-mac-mappings=phys:1e:02:ad:aa:bb:02 >> + >> +as hv3 >> +ovs-vsctl set open . >> external_ids:ovn-chassis-mac-mappings=phys:1e:02:ad:aa:bb:03 >> + >> +OVS_WAIT_UNTIL([test 6 = $(as hv1 ovs-ofctl dump-flows br-int table=0 | >> grep conj -c)]) >> +OVS_WAIT_UNTIL([test 6 = $(as hv2 ovs-ofctl dump-flows br-int table=0 | >> grep conj -c)]) >> +OVS_WAIT_UNTIL([test 6 = $(as hv3 ovs-ofctl dump-flows br-int table=0 | >> grep conj -c)]) >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv1 ovs-ofctl dump-flows br-int table=0 | \ >> +grep conj | grep "dl_dst=1e:02:ad:aa:bb:01" -c)]) >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int table=0 | \ >> +grep conj | grep "dl_dst=1e:02:ad:aa:bb:02" -c)]) >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int table=0 | \ >> +grep conj | grep "dl_dst=1e:02:ad:aa:bb:03" -c)]) >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv1 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb01->NXM_NX_ARP_SHA" -c)]) >> + >> +OVS_WAIT_UNTIL([test 0 = $(as hv2 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb01->NXM_NX_ARP_SHA" -c)]) >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb02->NXM_NX_ARP_SHA" -c)]) >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb03->NXM_NX_ARP_SHA" -c)]) >> + >> +as hv1 >> +reset_pcap_file hv1-ext1 hv1/ext1 >> + >> +send_arp_request() { >> + local inport=$1 eth_src=$2 eth_dst=$3 spa=$4 tpa=$5 >> + local reply_src_mac=$6 reply_dst_mac=$7 >> + local reply_sha=$8 reply_tha=$9 >> + >> + local eth_type=0806 >> + local eth=${eth_dst}${eth_src}${eth_type} >> + >> + local arp=0001080006040001${eth_src}${spa}${eth_dst}${tpa} >> + >> + local request=${eth}${arp} >> + as hv1 ovs-appctl netdev-dummy/receive hv${inport}-ext${inport} >> $request >> + >> + local reply=${reply_dst_mac}${reply_src_mac}${eth_type} >> + reply=${reply}0001080006040002${reply_sha}${tpa}${reply_tha}${spa} >> + echo $reply > hv1-ext${inport}.expected >> +} >> + >> +src_mac=f00000000003 >> +dst_mac=ffffffffffff >> +reply_src_mac=1e02adaabb03 >> +repl_dst_mac=f00000000003 >> +# Send ARP request to router ip - 10.0.0.1 >> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6) >> $(ip_to_hex 10 0 0 1) \ >> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac} >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb03->NXM_NX_ARP_SHA" | grep "n_packets=1" -c)]) >> + >> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected]) >> + >> +as hv1 >> +reset_pcap_file hv1-ext1 hv1/ext1 >> + >> +# Send unicast ARP request destined to the chassis mac of hv3. >> +src_mac=f00000000003 >> +dst_mac=1e02adaabb03 >> +reply_src_mac=1e02adaabb03 >> +repl_dst_mac=f00000000003 >> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6) >> $(ip_to_hex 10 0 0 1) \ >> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac} >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv3 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb03->NXM_NX_ARP_SHA" | grep "n_packets=2" -c)]) >> + >> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected]) >> + >> +# Make hv2 active. >> +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv2 60 >> + >> +OVS_WAIT_UNTIL( >> + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ >> +logical_port=ls1-lp_ext1` >> + test "$chassis" = "$hv2_uuid"]) >> + >> +reset_pcap_file hv1-ext1 hv1/ext1 >> + >> +src_mac=f00000000003 >> +dst_mac=ffffffffffff >> +reply_src_mac=1e02adaabb02 >> +repl_dst_mac=f00000000003 >> +# Send ARP request to router ip - 10.0.0.1. Should be replied by hv2. >> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6) >> $(ip_to_hex 10 0 0 1) \ >> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac} >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb02->NXM_NX_ARP_SHA" | grep "n_packets=1" -c)]) >> + >> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected]) >> + >> +as hv1 >> +reset_pcap_file hv1-ext1 hv1/ext1 >> + >> +# Send unicast ARP request destined to the chassis mac of hv2. >> +src_mac=f00000000003 >> +dst_mac=1e02adaabb02 >> +reply_src_mac=1e02adaabb02 >> +repl_dst_mac=f00000000003 >> +send_arp_request 1 ${src_mac} ${dst_mac} $(ip_to_hex 10 0 0 6) >> $(ip_to_hex 10 0 0 1) \ >> +${reply_src_mac} ${repl_dst_mac} ${reply_src_mac} ${repl_dst_mac} >> + >> +OVS_WAIT_UNTIL([test 1 = $(as hv2 ovs-ofctl dump-flows br-int >> table=65,arp | \ >> +grep "load:0x1e02adaabb02->NXM_NX_ARP_SHA" | grep "n_packets=2" -c)]) >> + >> +OVN_CHECK_PACKETS([hv1/ext1-tx.pcap], [hv1-ext1.expected]) >> + >> +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv3 70 >> +ovn-nbctl ha-chassis-group-add-chassis hagrp1 hv2 10 >> +OVS_WAIT_UNTIL( >> + [chassis=`ovn-sbctl --bare --columns chassis find port_binding \ >> +logical_port=ls1-lp_ext1` >> + test "$chassis" = "$hv3_uuid"]) >> + >> # disconnect hv3 from the network, hv1 should take over >> as hv3 >> port=${sandbox}_br-phys >> -- >> 2.26.2 >> >> _______________________________________________ >> dev mailing list >> [email protected] >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> >> _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
