Hi, lic121 and Ilya, Ilya, it looks like I did not get the first reply from you.
But it's ok, I now understand the solution, which is to restrict the termination condition rather than specifying the mirror context, and yes, this is a more general fix. Thanks for the explanation. I have some further questions. lic121 <lic...@chinatelecom.cn> 于2022年5月4日周三 11:23写道： > On Tue, May 03, 2022 at 10:18:18PM +0200, Ilya Maximets wrote: > > On 5/3/22 15:31, Peng He wrote: > > > Hi， > > > this issue has been found and resolved by two independent patches: > > > > > > > http://patchwork.ozlabs.org/project/openvswitch/patch/20190726121839.92497-1-wang...@139.com/ > < > http://patchwork.ozlabs.org/project/openvswitch/patch/20190726121839.92497-1-wang...@139.com/ > > > > > > http://patchwork.ozlabs.org/project/openvswitch/patch/20201009121503.43893-1-hepeng.0...@bytedance.com/ > < > http://patchwork.ozlabs.org/project/openvswitch/patch/20201009121503.43893-1-hepeng.0...@bytedance.com/ > > > > > > > > but I recently found my patch is still not fixing all the cases. > > > > > > would like to have a discussion about this. > > > > Hrm. It seems to me that the problem is more genric then mirrors. > > The reason is that tnl_port_map_lookup() logic doesn't work for > > a bridge with more than one tunnel termination port. > > Before commit dc0bd12f5b04 ("userspace: Enable non-bridge port > > as tunnel endpoint."), only the bridge port was allowed to terminate > > the tunnel. MAC+IP lookup performed by the classifier inside the > > tnl_port_map_lookup() was enough to determine if termination is > > needed or if the packet has to be forwarded without tunnel_pop. > > Now we have multiple ports that may or may not terminate the tunnel. > > > > Assuming we have 2 different tunnel ports and two different normal > > ports with different IP addresses and we expect the first tunnel > > to be terminated on the first port and the second tunnel to be > > terminated on the second port. At the same time, the packet from > > the first tunnel should be forwarded to the second port without > > tunnel_pop and the packet from the second tunnel should be > > forwarded to the first port without tunnel_pop. > > > > Since the classifier in the tunnel-port.c will have rules for > > both tunnels and it only matches on MAC+IP (+ some other fileds, > > but we don't care much right now), we will have decapsulation > > triggered in all cases. The same as what happened with the > > mirrored traffic. > > > > Two patcehs mentioned above only focused on mirrors, so they > > will not fix the problem in a common case. The patch below > > is closer to a solution, but it will skip the matches on dl_dst, > > and, possibly, other tunnel header fileds, which we absolutely > > need in a non-mirror case with 2 tunnels. > > > Hi Ilya, you mean we need to check dl_dst as well before tnl classifier > lookup? For the case that two ports have the same ip address? > > > > > The correct solution would be to add an output port match to > > the classifier, but clasifier is not designed to do that and > > 'struct match' doesn't have an appropriate field. It shoudl > > be a separate classifier per ipdev, or something like that. > > > > Thoughts? > did you mean that if the output port is NORMAL or LOCAL, the tunnel filter will work, however, if it's a concrete port like a normal one, we should not decap it but just forward it. Event the IP addresses and port matches? > > > > In any case, every part of the packet that was used to make a > > decision must be unwildcarded. IP on the port is not part of > > the packet, but dst_ip is. > > > > Best regards, Ilya Maximets. > > > > > > > > > > > > > > lic121 <lic...@chinatelecom.cn <mailto:lic...@chinatelecom.cn>> > 于2022年5月3日周二 20:55写道： > > > > > > In compose_output_action__(), terminate_native_tunnel() is called > > > to test if a pkt matches a tnl port. If yes, a TUNNEL_POP action > > > is appended. Instead of outputing the pkt to the origin port, pkt > > > is redirected into the tnl port. > > > > > > pkt metadata(i.e. nw_proto, dst_ip, dst_port) is considered when > > > testing a pkt regardless of the out port until Ilya's patch[1]. > > > Patch[1] adds the condition that terminal ports must have IP > > > address. This patch is to make addtional port ip check that port > > > ip address must match the pkt dst ip. This check covers the mirror > > > port case: > > > To tcpdump underlay pkt, we may create mirror port of dpdk port. > > > When compose_output_action__() is called on the mirror port, > > > pkt is redirected into tnl port again. As a result, ping shows > > > DUP. fast path flow has two tnl_pop actions. > > > > > > ``` > > > [root@ovs1 vagrant]# ping 172.16.100.2 > > > PING 172.16.100.2 (172.16.100.2) 56(84) bytes of data. > > > 64 bytes from 172.16.100.2 <http://172.16.100.2>: icmp_seq=1 > ttl=64 time=0.606 ms > > > 64 bytes from 172.16.100.2 <http://172.16.100.2>: icmp_seq=1 > ttl=64 time=0.617 ms (DUP!) > > > ``` > > > > > > ``` > > > [root@ovs ovs]# ovs-appctl dpctl/dump-flows |grep tnl_pop > > > > > recirc_id(0),in_port(3),packet_type(ns=0,id=0),eth(src=52:54:00:a6:c6:de,dst=ea:b4:09:72:44:4c),eth_type(0x0800),ipv4(dst=192.168.124.33,proto=17,frag=no),udp(dst=8472), > packets:1, bytes:148, used:0.637s, actions:tnl_pop(2),tnl_pop(2) > > > ``` > > > > > > [1] (d584bb2b6a1 "ofproto-dpif-xlate: Terminate native tunnels > only on ports with IP addresses.") > > > > > > Signed-off-by: lic121 <lic...@chinatelecom.cn <mailto: > lic...@chinatelecom.cn>> > > > --- > > > ofproto/ofproto-dpif-xlate.c | 45 > +++++++++++++++++++++++++++------------- > > > tests/tunnel-push-pop.at <http://tunnel-push-pop.at> | 49 > ++++++++++++++++++++++++++++++++++++++++++++ > > > 2 files changed, 80 insertions(+), 14 deletions(-) > > > > > > diff --git a/ofproto/ofproto-dpif-xlate.c > b/ofproto/ofproto-dpif-xlate.c > > > index 17f7e28..30e636e 100644 > > > --- a/ofproto/ofproto-dpif-xlate.c > > > +++ b/ofproto/ofproto-dpif-xlate.c > > > @@ -4107,18 +4107,26 @@ is_neighbor_reply_correct(const struct > xlate_ctx *ctx, const struct flow *flow) > > > } > > > > > > static bool > > > -xport_has_ip(const struct xport *xport) > > > +xport_has_ip(const struct xport *xport, const struct in6_addr ip6) > > > { > > > struct in6_addr *ip_addr, *mask; > > > int n_in6 = 0; > > > + int i; > > > + bool hasip = false; > > > > > > if (netdev_get_addr_list(xport->netdev, &ip_addr, &mask, > &n_in6)) { > > > n_in6 = 0; > > > } else { > > > + for (i = 0; i < n_in6; i++) { > > > + if (IN6_ARE_ADDR_EQUAL(ip_addr + i, &ip6)) { > > > + hasip = true; > > > + break; > > > + } > > > + } > > > free(ip_addr); > > > free(mask); > > > } > > > - return n_in6 ? true : false; > > > + return hasip; > > > } > > > > > > static bool > > > @@ -4127,26 +4135,35 @@ terminate_native_tunnel(struct xlate_ctx > *ctx, const struct xport *xport, > > > odp_port_t *tnl_port) > > > { > > > *tnl_port = ODPP_NONE; > > > + bool ip_pkt = true; > > > + struct in6_addr ip_dst; > > > + > > > + if (flow->dl_type == htons(ETH_TYPE_IP)) { > > > + ip_dst = in6_addr_mapped_ipv4(flow->nw_dst); > > > + } else if (flow->dl_type == htons(ETH_TYPE_IPV6)) { > > > + ip_dst = flow->ipv6_dst; > > > + } else { > > > + ip_pkt = false; > > > + } > > > > > > /* XXX: Write better Filter for tunnel port. We can use > in_port > > > * in tunnel-port flow to avoid these checks completely. > > > * > > > - * Port without an IP address cannot be a tunnel termination > point. > > > - * Not performing a lookup in this case to avoid > unwildcarding extra > > > - * flow fields (dl_dst). */ > > > - if (ovs_native_tunneling_is_on(ctx->xbridge->ofproto) > > > - && xport_has_ip(xport)) { > > > + * Not performing a lookup if dst ip not match, to avoid > unwildcarding > > > + * extra flow fields (dl_dst). Also avoid duplicate tnl_pop. > */ > > > + if ((flow->dl_type == htons(ETH_TYPE_ARP) || > > > + flow->nw_proto == IPPROTO_ICMPV6) && > > > + is_neighbor_reply_correct(ctx, flow)) { > > > + tnl_neigh_snoop(flow, wc, ctx->xbridge->name, > > > + ctx->xin->allow_side_effects); > > > + > > > + } else if (ovs_native_tunneling_is_on(ctx->xbridge->ofproto) > > > + && ip_pkt && xport_has_ip(xport, ip_dst)) { > > > *tnl_port = tnl_port_map_lookup(flow, wc); > > > > > > /* If no tunnel port was found and it's about an ARP or > ICMPv6 packet, > > > * do tunnel neighbor snooping. */ > > > - if (*tnl_port == ODPP_NONE && > > > - (flow->dl_type == htons(ETH_TYPE_ARP) || > > > - flow->nw_proto == IPPROTO_ICMPV6) && > > > - is_neighbor_reply_correct(ctx, flow)) { > > > - tnl_neigh_snoop(flow, wc, ctx->xbridge->name, > > > - ctx->xin->allow_side_effects); > > > - } else if (*tnl_port != ODPP_NONE && > > > + if (*tnl_port != ODPP_NONE && > > > ctx->xin->allow_side_effects && > > > dl_type_is_ip_any(flow->dl_type)) { > > > struct eth_addr mac = flow->dl_src; > > > diff --git a/tests/tunnel-push-pop.at <http://tunnel-push-pop.at> > b/tests/tunnel-push-pop.at <http://tunnel-push-pop.at> > > > index c633441..b3a3bf9 100644 > > > --- a/tests/tunnel-push-pop.at <http://tunnel-push-pop.at> > > > +++ b/tests/tunnel-push-pop.at <http://tunnel-push-pop.at> > > > @@ -808,6 +808,55 @@ NXST_FLOW reply: > > > OVS_VSWITCHD_STOP > > > AT_CLEANUP > > > > > > +AT_SETUP([tunnel_push_pop - multiple output]) > > > + > > > +OVS_VSWITCHD_START([add-port br0 p0 -- set Interface p0 > type=dummy ofport_request=1 other-config:hwaddr=aa:55:aa:55:00:00]) > > > +AT_CHECK([ovs-vsctl add-port br0 m0 -- set Interface m0 > type=dummy ofport_request=2]) > > > +AT_CHECK([ovs-vsctl add-br int-br -- set bridge int-br > datapath_type=dummy], [0]) > > > +AT_CHECK([ovs-vsctl add-port int-br t1 -- set Interface t1 > type=gre \ > > > + options:remote_ip=1.1.2.92 options:key=456 > options:seq=true ofport_request=4], [0]) > > > + > > > +AT_CHECK([ovs-appctl dpif/show], [0], [dnl > > > +dummy@ovs-dummy: hit:0 missed:0 > > > + br0: > > > + br0 65534/100: (dummy-internal) > > > + m0 2/2: (dummy) > > > + p0 1/1: (dummy) > > > + int-br: > > > + int-br 65534/3: (dummy-internal) > > > + t1 4/4: (gre: key=456, remote_ip=1.1.2.92, seq=true) > > > +]) > > > + > > > +AT_CHECK([ovs-appctl netdev-dummy/ip4addr br0 1.1.2.88/24 < > http://1.1.2.88/24>], [0], [OK > > > +]) > > > +AT_CHECK([ovs-appctl netdev-dummy/ip4addr m0 1.1.3.88/24 < > http://1.1.3.88/24>], [0], [OK > > > +]) > > > +AT_CHECK([ovs-appctl ovs/route/add 1.1.2.92/24 < > http://1.1.2.92/24> br0], [0], [OK > > > +]) > > > +AT_CHECK([ovs-appctl ovs/route/add 1.1.3.92/24 < > http://1.1.3.92/24> m0], [0], [OK > > > +]) > > > +AT_CHECK([ovs-ofctl add-flow br0 'arp,priority=1,action=normal']) > > > + > > > +dnl Use arp reply to achieve tunnel next hop mac binding > > > +AT_CHECK([ovs-appctl netdev-dummy/receive p0 > 'recirc_id(0),in_port(1),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0806),arp(sip=1.1.2.92,tip=1.1.2.88,op=2,sha=f8:bc:12:44:34:b6,tha=00:00:00:00:00:00)']) > > > + > > > +dnl Ouput gre pkt to both br0 port and m0 port > > > +dnl m0 port to simulate mirror port, or flood case > > > +AT_CHECK([ovs-ofctl add-flow br0 > 'in_port(1),ip,ip_proto=47,priority=99,action=br0,m0']) > > > + > > > +AT_CHECK([ovs-appctl netdev-dummy/receive p0 > 'recirc_id(0),in_port(1),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(src=1.1.2.92,dst=1.1.2.88,proto=47,tos=0,ttl=64,frag=no)']) > > > + > > > +dnl port br0: has the configured ip address of tnl port t1, so > 'tnl_pop(4)' > > > +dnl port m0: does not have a configured tnl ip address, so > 'output:2' > > > +AT_CHECK([ovs-appctl dpctl/dump-flows], [0], [dnl > > > +flow-dump from the main thread: > > > > > +recirc_id(0),in_port(1),packet_type(ns=0,id=0),eth(src=f8:bc:12:44:34:b6,dst=aa:55:aa:55:00:00),eth_type(0x0806),arp(sip=1.1.2.92,tip=1.1.2.88,op=2,sha=f8:bc:12:44:34:b6), > packets:0, bytes:0, used:never, actions:100,2 > > > > > +recirc_id(0),in_port(1),packet_type(ns=0,id=0),eth(dst=aa:55:aa:55:00:00),eth_type(0x0800),ipv4(dst=1.1.2.88,proto=47,frag=no), > packets:0, bytes:0, used:never, actions:tnl_pop(4),2 > > > +]) > > > + > > > +OVS_VSWITCHD_STOP > > > +AT_CLEANUP > > > + > > > AT_SETUP([tunnel_push_pop - flow translation on unrelated > bridges]) > > > > > > OVS_VSWITCHD_START( > > > -- > > > 1.8.3.1 > > > > > > _______________________________________________ > > > dev mailing list > > > d...@openvswitch.org <mailto:d...@openvswitch.org> > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev < > https://mail.openvswitch.org/mailman/listinfo/ovs-dev> > > > > > > > > > > > > -- > > > hepeng > > > -- hepeng _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
