Re: [ovs-discuss] UDP stateful ACL not working when pkt passing through two switches ?
On Thu, Jul 28, 2022 at 11:53 AM Brendan Doyle wrote: > > UDP stateful ACL not working? The logical representation of My network is shown bellow > ('ovn-nbctl show' shown towards the end). I have a Port Group (pg_vcn3_net1_sl3) that has > two ports in it, the VM port on switch(ls_vcn3_net1) and lsb_vcn4_stgw-lr_vcn3_stgw switch > (ls_vcn3_backbone) asshown below ((o)). > > I do a 'showmount -e 192.16.1.106' in the VM, I see the pkt go out from the VM get to the NFS > server on the underlay, see the reply on the underlay and then I see my PG ACL drop the pkt. > > The ACLs are: > > Egress From VM - Ingress to switch > --- > from-lport 32767 (inport == @pg_vcn3_net1_sl3 && (arp || udp.dst == 67 || udp.dst == 68)) allow-related > from-lport 27000 (inport == @pg_vcn3_net1_sl3 && ip4.dst == 192.16.1.0/24 && udp.dst == 111) allow-related > from-lport 0 (inport == @pg_vcn3_net1_sl3) drop log(name=fss-8,severity=debug) <--- Drops the return pkt According to your description, the ACL here not only applies to the VM port but also the router port (lsb_vcn4_stgw-lr_vcn3_stgw) on the ls_vcn3_backbone switch. So the return packet is in fact dropped at the backbone switch, which is expected because we don't support conntrack for router ports, so the "to-lport" ACL below wouldn't create the conntrack entry. OVN ACL is primarily to apply rules for VIFs (VMs/containers). I remember @Numan Siddique worked on some patches related to ACL on router port recently, so maybe he could provide more details or correct me if I am wrong. Thanks, Han > > Ingress TO VM - Egress from switch > > to-lport 32767 (outport == @pg_vcn3_net1_sl3 && (arp || udp.dst == 67 || udp.dst == 68)) allow-related > to-lport 27000 (outport == @pg_vcn3_net1_sl3 && ip4.src == 192.16.1.0/24 && tcp.dst == 111) allow-related > to-lport 27000 (outport == @pg_vcn3_net1_sl3 && ip4.src == 192.16.1.0/24 && tcp.dst == 20048) allow-related > to-lport 27000 (outport == @pg_vcn3_net1_sl3 && ip4.src == 192.16.1.0/24 && udp.dst == 111) allow-related <--- But this should > to-lport 0 (outport == @pg_vcn3_net1_sl3) drop log(name=fss-17,severity=debug) have allowed the > return pkt > > > ++ > | VM | > | 192.16.1.6 | > +-((O))--+ > | 284195d2-9280-4334-900e-571ecd00327a in PG pg_vcn3_net1_sl3 > +-+ > |ls_vcn3_net1 | > +-+ > | ls_vcn3_net1-lr_vcn3_net1 (proxy ARP for 192.16.1.106) > | > | > | lr_vcn3_net1-ls_vcn3_net1 (192.16.1.1/24) > /\ > ( lr_vcn3_net1 ) > \/ > | lr_vcn3_net1-lsb_vcn3_net1 (253.255.25.1/25) > | > | > | lsb_vcn3_net1-lr_vcn3_net1 > ++ > | ls_vcn3_backbone | > +((O))---+ > | lsb_vcn4_stgw-lr_vcn3_stgw in PG pg_vcn3_net1_sl3 > | > | > | lr_vcn3_stgw-lsb_vcn3_stgw (253.255.25.10/25) > /\ > ( lr_vcn3_stgw ) SNAT 192.16.1.6 to 253.255.80.8 > \/ > | lr_vcn3_stgw-ls_vcn3_external_stgw (253.255.80.20/16) > | > | > | ls_vcn3_external_stgw-lr_vcn3_stgw > +---+ > | ls_vcn3_external_stgw | > +---+ > | ln-ls_vcn3_external_stgw > | (localnet) > | >+-+ >| br-ext | Physical OVS on chassis >+-+ > | Egress : Change dst 192.16.1.106 to dst 253.255.0.2 > | Ingress: Change src 253.255.0.2 to 192.16.1.106 > 253.255.0.0/16 | > | > +---+ > | NFS server | > | 253.255.0.2 | > +---+ > > When I do a trace of the out going pkt, it looks like to me that there is no conntrack > established in the ls_vcn3_backbone so it does not recognize the return pkt as a return > but the 'allow-related' should have established that. See Below > > > ovn-trace --detailed ls_vcn3_net1 'inport == "284195d2-9280-4334-900e-571ecd00327a" && eth.dst == 40:44:00:00:00:90 && eth.src == 52:54:00:02:55:96 && ip4.src == 192.16.1.6 && ip4.dst == 192.16.1.106 && ip.ttl == 64 && udp.dst == 111' > # udp,reg14=0x1,vlan_tci=0x,dl_src=52:54:00:02:55:96,dl_dst=40:44:00:00:00:90,nw_src=192.16.1.6,nw_dst=192.16.1.106,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=0,tp_dst=111 > > ingress(dp="ls_vcn3_net1", inport="284195") > --- > 0. ls_
[ovs-discuss] UDP stateful ACL not working when pkt passing through two switches ?
UDP stateful ACL not working? The logical representation of My network is shown bellow ('ovn-nbctl show' shown towards the end). I have a Port Group (pg_vcn3_net1_sl3) that has two ports in it, the VM port on switch(ls_vcn3_net1) and lsb_vcn4_stgw-lr_vcn3_stgw switch (ls_vcn3_backbone) asshown below ((o)). I do a 'showmount -e 192.16.1.106' in the VM, I see the pkt go out from the VM get to the NFS server on the underlay, see the reply on the underlay and then I see my PG ACL drop the pkt. The ACLs are: Egress From VM - Ingress to switch --- from-lport 32767 (inport == @pg_vcn3_net1_sl3 && (arp || udp.dst == 67 || udp.dst == 68)) allow-related from-lport 27000 (inport == @pg_vcn3_net1_sl3 && ip4.dst == 192.16.1.0/24 && udp.dst == 111) allow-related from-lport 0 (inport == @pg_vcn3_net1_sl3) drop log(name=fss-8,severity=debug) <--- Drops the return pkt Ingress TO VM - Egress from switch to-lport 32767 (outport == @pg_vcn3_net1_sl3 && (arp || udp.dst == 67 || udp.dst == 68)) allow-related to-lport 27000 (outport == @pg_vcn3_net1_sl3 && ip4.src == 192.16.1.0/24 && tcp.dst == 111) allow-related to-lport 27000 (outport == @pg_vcn3_net1_sl3 && ip4.src == 192.16.1.0/24 && tcp.dst == 20048) allow-related * to-lport 27000 (outport == @pg_vcn3_net1_sl3 && ip4.src == 192.16.1.0/24 && udp.dst == 111) allow-related <--- But this should * to-lport 0 (outport == @pg_vcn3_net1_sl3) drop log(name=fss-17,severity=debug) have allowed the return pkt ++ | VM | | 192.16.1.6 | +-((O))--+ | *284195d2-9280-4334-900e-571ecd00327a *in PG*pg_vcn3_net1_sl3* +-+ | ls_vcn3_net1 | +-+ | ls_vcn3_net1-lr_vcn3_net1 (proxy ARP for 192.16.1.106) | | | lr_vcn3_net1-ls_vcn3_net1 (192.16.1.1/24) /\ ( lr_vcn3_net1 ) \/ | lr_vcn3_net1-lsb_vcn3_net1 (253.255.25.1/25) | | | lsb_vcn3_net1-lr_vcn3_net1 ++ | ls_vcn3_backbone | +((O))---+ | *lsb_vcn4_stgw-lr_vcn3_stgw *in PG*pg_vcn3_net1_sl3* | | | lr_vcn3_stgw-lsb_vcn3_stgw (253.255.25.10/25) /\ ( lr_vcn3_stgw ) SNAT 192.16.1.6 to 253.255.80.8 \/ | lr_vcn3_stgw-ls_vcn3_external_stgw (253.255.80.20/16) | | | ls_vcn3_external_stgw-lr_vcn3_stgw +---+ | ls_vcn3_external_stgw | +---+ | ln-ls_vcn3_external_stgw | (localnet) | +-+ | br-ext | Physical OVS on chassis +-+ | Egress : Change dst 192.16.1.106 to dst 253.255.0.2 | Ingress: Change src 253.255.0.2 to 192.16.1.106 253.255.0.0/16 | | +---+ | NFS server | | 253.255.0.2 | +---+ When I do a trace of the out going pkt, it looks like to me that there is no conntrack established in the ls_vcn3_backbone so it does not recognize the return pkt as a return but the 'allow-related' should have established that. See Below ovn-trace --detailed ls_vcn3_net1 'inport == "284195d2-9280-4334-900e-571ecd00327a" && eth.dst == 40:44:00:00:00:90 && eth.src == 52:54:00:02:55:96 && ip4.src == 192.16.1.6 && ip4.dst == 192.16.1.106 && ip.ttl == 64 && udp.dst == 111' # udp,reg14=0x1,vlan_tci=0x,dl_src=52:54:00:02:55:96,dl_dst=40:44:00:00:00:90,nw_src=192.16.1.6,nw_dst=192.16.1.106,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=0,tp_dst=111 ingress(dp="ls_vcn3_net1", inport="284195") --- 0. ls_in_port_sec_l2 (northd.c:5493): inport == "284195", priority 50, uuid 0b9563aa next; 5. ls_in_pre_acl (northd.c:5753): ip, priority 100, uuid fad1d4d2 reg0[0] = 1; next; 7. ls_in_pre_stateful (northd.c:5937): reg0[0] == 1, priority 100, uuid 0bb84a55 ct_next; ct_next(ct_state=est|trk /* default (use --ct to customize) */) *<- Looks like we set up conntrack on the ls_vcn3_net1* --- 8. ls_in_acl_hint (northd.c:6019): !ct.new && ct.est && !ct.rpl && ct_label.blocked == 0, priority 4, uuid 2712c6a2 reg0[8] = 1; reg0[10] = 1; next; 9. ls_in_acl (northd.c:6244): reg0[8] == 1 && (inport == @pg_vcn3_net1_sl3 && ip4.dst == 192.16.1.0/24 && udp.dst == 111), priority 28000, uuid aff2df9c next; 22. ls_in_l2_lkup (northd.c:8218): eth.ds