OK so the simple 1 line change to northd.c in:

 [ovs-dev,v8,1/6] northd: Swap src and dst eth addresses in router egress loop.

fixes the problem, can access all external networks, and the haripin between 10.68.49.185 <-> 10.68.49.185
works. Thumbs up for me on this patch!


On 30/06/2021 10:11, Brendan Doyle wrote:
So If I do :

ovn-nbctl add logical_router_port lr1-ls1_external networks "10.68.49.185/32 10.68.49.184/32"

Then the hairpin works and I have connectivity between 10.68.49.185 <-> 10.68.49.185

But This patch also look promising:
[ovs-dev,v8,1/6] northd: Swap src and dst eth addresses in router egress loop.

I'll try adding this, and incrementally the other patches in the series.

Brendan


On 29/06/2021 22:40, Numan Siddique wrote:
On Tue, Jun 29, 2021 at 5:06 PM Brendan Doyle <[email protected]> wrote:


On 29/06/2021 21:38, Numan Siddique wrote:
On Tue, Jun 29, 2021 at 4:13 PM Brendan Doyle <[email protected]> wrote:
Hi,

With a very simple notwork (two VMs on different chassis), 1 subnet,
single LS and
LR/Gateway. The two VMs can ping each other using their Logical IPs.
Each has an
"External IP", and each can be accessed from an external network on that
external IP.
BUT they can't ping each other using their external IPs. I would have
expected that
either:

a) The packets are sent on the external net then hairpinned back to the OVN
        gateway by the external net router.

b) They are hairpinned by OVN.

It seems that OVN attempts the latter, but does not succeed. The
details, NB network,
and pkt trace are as follows:

ovn-nbctl show
switch 2710eebe-f2b3-49e4-bcd6-dcfa48ed6470 (ls1_external)
       port ln-ls1_external
           type: localnet
           addresses: ["unknown"]
       port ls1_external-lr1
           type: router
           router-port: lr1-ls1_external

switch ff909b16-d863-4e3d-a10b-2f0010f17b23 (ls1)
       port 47433b54-ac10-42f1-ae84-cc6fbb580297
           addresses: ["52:54:00:be:06:16 192.16.1.6"]
       port 00bff7c0-2e2d-41ba-9485-3b5fa9801365
           addresses: ["52:54:00:e6:4f:46 192.16.1.5"]
       port ls1-lr1
           type: router
           router-port: lr1-ls1

router 63e1b6a2-327f-4a24-b0c9-3a0e951beb2b (lr1)
       port lr1-ls1_external
           mac: "40:44:00:00:01:a0"
           networks: ["253.255.80.10/16"]
           gateway chassis: [ca-rain06 ca-rain17 ca-rain05]
       port lr1-ls1
           mac: "40:44:00:00:01:30"
           networks: ["192.16.1.1/24"]
       nat f4675661-f4cc-4f7c-b534-ca75e090ed74
           external ip: "10.68.49.184"
           logical ip: "192.16.1.5"
           type: "dnat_and_snat"
       nat f5592262-5fbd-4cef-8773-903875ba34d6
           external ip: "10.68.49.185"
           logical ip: "192.16.1.6"
           type: "dnat_and_snat"

Why don't the external ips belong to the subnet - 253.255.80.10/16 ?
i.e to the network of ls1_external ?
The 253.255.80.10/16 network is an internal "underlay" Network. An infra
structure network
of the rack product. The "External IPs", are IPs belonging to networks
outside the rack.

So in Normal case traffic  destined for a VM from outside the rack,
would send to the VM
"External IP", that arrives at the rack physical uplink router, and is
sent across the rack
physical network (253.255.0.0/16) to the OVN Gateway, which DNATs and
send to the VM
Logical IP (reverse on traffic from VM to destination outside the rack).


I'm pretty sure if you change the external_ips from 10.68.49.184 and
10.68.49.185 to
the ones belonging to 253.255.80.10/16, it would work.
We can't do that, these are different address spaces in different
physical networks.
I could try adding the 10.68.49.184/185 IPs to the "networks" table in
lr1-ls1_external
I'd suggest trying out with these patches once ? -
https://urldefense.com/v3/__https://patchwork.ozlabs.org/project/ovn/list/?series=247106__;!!ACWV5N9M2RV99hQ!ZKO2z-ifCaUA-TPeLm7ZP9V7hkX8tZSv4HE4-Ogo2BhBcLfSbibLIh4xDsIiqu4xmH8$
Ok, will do, are they in master, as I'm running with a fairly recent
build (maybe two weeks old)
The patches are still under review and may not apply cleanly with the
tip.  You can access it from here too -
https://urldefense.com/v3/__https://github.com/ovsrobot/ovn/commits/series_247106__;!!ACWV5N9M2RV99hQ!e3YISaySCgi6qg3Y-8_gdx0IN_FeVsl5onOgkxhBhhgp_69r8PTAROpeu3yG3eaPN0c$

Thanks
Numan

Thanks

Numan


ovn-nbctl lr-route-list lr1
IPv4 Routes
                   0.0.0.0/0               253.255.0.1 dst-ip lr1-ls1_external

ovn-trace --detailed ls1 'inport ==
"47433b54-ac10-42f1-ae84-cc6fbb580297" && eth.dst == 40:44:00:00:01:30 && eth.src == 52:54:00:be:06:16 && ip4.src == 192.16.1.6 && ip4.dst ==
10.68.49.184 && ip.ttl == 64 && icmp4.type == 8'
#
icmp,reg14=0x1,vlan_tci=0x0000,dl_src=52:54:00:be:06:16,dl_dst=40:44:00:00:01:30,nw_src=192.16.1.6,nw_dst=10.68.49.184,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0

ingress(dp="ls1", inport="47433b")
----------------------------------
    0. ls_in_port_sec_l2 (ovn-northd.c:4834): inport == "47433b", priority
50, uuid ae50c799
       next;
22. ls_in_l2_lkup (ovn-northd.c:7587): eth.dst == 40:44:00:00:01:30,
priority 50, uuid c29dec2a
       outport = "ls1-lr1";
       output;

egress(dp="ls1", inport="47433b", outport="ls1-lr1")
----------------------------------------------------
    0. ls_out_pre_lb (ovn-northd.c:4980): ip && outport == "ls1-lr1",
priority 110, uuid d4d7c7af
       next;
    9. ls_out_port_sec_l2 (ovn-northd.c:4929): outport == "ls1-lr1",
priority 50, uuid 36b335f9
       output;
       /* output to "ls1-lr1", type "patch" */

ingress(dp="lr1", inport="lr1-ls1")
-----------------------------------
    0. lr_in_admission (ovn-northd.c:9575): eth.dst == 40:44:00:00:01:30
&& inport == "lr1-ls1", priority 50, uuid c67387d7
       xreg0[0..47] = 40:44:00:00:01:30;
       next;
    1. lr_in_lookup_neighbor (ovn-northd.c:9654): 1, priority 0, uuid c050ede1
       reg9[2] = 1;
       next;
    2. lr_in_learn_neighbor (ovn-northd.c:9663): reg9[2] == 1, priority
100, uuid e5780577
       next;
10. lr_in_ip_routing (ovn-northd.c:8622): ip4.dst == 0.0.0.0/0, priority
1, uuid 52d001c6
       ip.ttl--;
       reg8[0..15] = 0;
       reg0 = 253.255.0.1;
       reg1 = 253.255.80.10;
       eth.src = 40:44:00:00:01:a0;
       outport = "lr1-ls1_external";
       flags.loopback = 1;
       next;
11. lr_in_ip_routing_ecmp (ovn-northd.c:9921): reg8[0..15] == 0,
priority 150, uuid 920ee40c
       next;
12. lr_in_policy (ovn-northd.c:10046): 1, priority 0, uuid e2014343
       reg8[0..15] = 0;
       next;
13. lr_in_policy_ecmp (ovn-northd.c:10048): reg8[0..15] == 0, priority
150, uuid ed8c4d4d
       next;
14. lr_in_arp_resolve (ovn-northd.c:10082): ip4, priority 0, uuid 2cfde30a
       get_arp(outport, reg0);
       /* MAC binding to 00:00:0c:07:ac:14. */
       next;
17. lr_in_gw_redirect (ovn-northd.c:10598): outport ==
"lr1-ls1_external", priority 50, uuid 521a9223
       outport = "cr-lr1-ls1_external";
       next;
18. lr_in_arp_request (ovn-northd.c:10671): 1, priority 0, uuid e43fdfbd
       output;
       /* Replacing type "chassisredirect" outport "cr-lr1-ls1_external"
with distributed port "lr1-ls1_external". */

egress(dp="lr1", inport="lr1-ls1", outport="lr1-ls1_external")
--------------------------------------------------------------
    0. lr_out_undnat (ovn-northd.c:11459): ip && ip4.src == 192.16.1.6 &&
outport == "lr1-ls1_external" &&
is_chassis_resident("cr-lr1-ls1_external"), priority 100, uuid e8b081df
       ct_dnat;

ct_dnat /* assuming no un-dnat entry, so no change */
-----------------------------------------------------
    1. lr_out_snat (ovn-northd.c:11552): ip && ip4.src == 192.16.1.6 &&
outport == "lr1-ls1_external" &&
is_chassis_resident("cr-lr1-ls1_external"), priority 161, uuid f50e5215
       ct_snat(10.68.49.185);

ct_snat(ip4.src=10.68.49.185)
-----------------------------
    2. lr_out_egr_loop (ovn-northd.c:11846): ip4.dst == 10.68.49.184 &&
outport == "lr1-ls1_external" &&
is_chassis_resident("cr-lr1-ls1_external"), priority 100, uuid a6499050
       clone { ct_clear; inport = outport; outport = ""; flags = 0;
flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0;
reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[0] = 1;
next(pipeline=ingress, table=0); };

clone
-----
       ct_clear;
       inport = outport;
       outport = "";
       flags = 0;
       flags.loopback = 1;
       reg0 = 0;
       reg1 = 0;
       reg2 = 0;
       reg3 = 0;
       reg4 = 0;
       reg5 = 0;
       reg6 = 0;
       reg7 = 0;
       reg8 = 0;
       reg9 = 0;
       reg9[0] = 1;
       next(pipeline=ingress, table=0);

ingress(dp="lr1", inport="lr1-ls1_external")
--------------------------------------------
    0. lr_in_admission: no match (implicit drop)

If we look at the section of code pointed to by ovn-northd.c:11846

          /* Egress Loopback table: For NAT on a distributed router.
            * If packets in the egress pipeline on the distributed
            * gateway port have ip.dst matching a NAT external IP, then
            * loop a clone of the packet back to the beginning of the
            * ingress pipeline with inport = outport. */
           if (od->l3dgw_port) {
               /* Distributed router. */
               ds_clear(match);
               ds_put_format(match, "ip%s.dst == %s && outport == %s",
                             is_v6 ? "6" : "4",
                             nat->external_ip,
od->l3dgw_port->json_key);
               if (!distributed) {
                   ds_put_format(match, " && is_chassis_resident(%s)",
od->l3redirect_port->json_key);
               } else {
                   ds_put_format(match, " && is_chassis_resident(\"%s\")",
                                 nat->logical_port);
               }
              ds_clear(actions);
               ds_put_format(actions,
                             "clone { ct_clear; "
                             "inport = outport; outport = \"\"; "
                             "flags = 0; flags.loopback = 1; ");
               for (int j = 0; j < MFF_N_LOG_REGS; j++) {
                   ds_put_format(actions, "reg%d = 0; ", j);
               }
               ds_put_format(actions, REGBIT_EGRESS_LOOPBACK" = 1; "
                             "next(pipeline=ingress, table=%d); };",
ovn_stage_get_table(S_ROUTER_IN_ADMISSION));
               ovn_lflow_add_with_hint(lflows, od, S_ROUTER_OUT_EGR_LOOP, 100,                                        ds_cstr(match), ds_cstr(actions),
&nat->header_);
           }

It seems clear what the intent is, but the pkt is dropped immediately
when returned to the ingress
pipeline. Am I missing some config?


Thanks Brendan


_______________________________________________
discuss mailing list
[email protected]
https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!ZKO2z-ifCaUA-TPeLm7ZP9V7hkX8tZSv4HE4-Ogo2BhBcLfSbibLIh4xDsIi0Md7RaE$
_______________________________________________
discuss mailing list
[email protected]
https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!e3YISaySCgi6qg3Y-8_gdx0IN_FeVsl5onOgkxhBhhgp_69r8PTAROpeu3yGTHgIUMg$

_______________________________________________
discuss mailing list
[email protected]
https://urldefense.com/v3/__https://mail.openvswitch.org/mailman/listinfo/ovs-discuss__;!!ACWV5N9M2RV99hQ!Z3k6-fDQJqwNn8Agn4ngwEJJy4wHaYT_i3fUlmeofW97TlsXsnMU3UMjNkbzwhW9YBM$


_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Reply via email to