May be I missed something, but when I tried setting logical routers into specific chassis, still the E/W traffic was handled in a distributed way (from original chassis to destination chassis without going through the router chassis), such chassis was only used for N/S, but may be I got something wrong.
On Wed, Sep 20, 2017 at 4:48 PM, Gao Zhenyu <[email protected]> wrote: > " > But, if an ovn port in foo (chassis A) wants to talk to alice1 (chassis > B), > wouldn't all that E/W routing will happen virtually and the end result is > just a tunneled packet between chassis A and chassis B ? " > [ Now the hash function base on dst IP, if foo1 only talks to alice1, and > it is the tunnel packet between chassisA and chassis B ] > > The benifit is if you have two ovn-routers and those router are ONLY > deployed in chassis C and chassis D, the traffics can be sperated in two > paths automatically. Otherwise you need to config static rule one by one to > seperate traffics. > To make a long story short, you also can do same thing by config numerous > static rules to seperate traffic but the multipath can do it > automatically. > > 2017-09-20 22:08 GMT+08:00 Miguel Angel Ajo Pelayo <[email protected]>: > >> I forgot to say thank you very much for the explanation and diagrams. >> >> On Wed, Sep 20, 2017 at 4:07 PM, Miguel Angel Ajo Pelayo < >> [email protected]> wrote: >> >>> But, if an ovn port in foo (chassis A) wants to talk to alice1 (chassis >>> B), >>> wouldn't all that E/W routing will happen virtually and the end result >>> is just a tunneled packet between chassis A and chassis B ? >>> >>> What's the benefit of multipath there if the possible failing link is >>> always the connection between chassis A and chassis B ? >>> >>> I suspect there's something I'm missing on the picture. >>> >>> On Wed, Sep 20, 2017 at 3:49 PM, Gao Zhenyu <[email protected]> >>> wrote: >>> >>>> You can take a look at this patch that implement a testcase : >>>> https://patchwork.ozlabs.org/patch/815475/ >>>> >>>> In the testcase, we have R1, R2, R3. >>>> >>>> R1 and R2 that are connected to each other via LS "join" in >>>> 20.0.0.0/24 network. >>>> R1 and R3 that are connected to each other via LS "join2" in >>>> 20.0.0.0/24 network. >>>> R1 has switchess foo (192.168.1.0/24) connected to it. R2 and R3 has >>>> alice (172.16.1.0/24) connected to it. >>>> R2 and R3 are gateway routers. >>>> >>>> A packet send to alice1/aclie2 from foo have mulitpath to destination: >>>> 1. foo-->R1-->join-->R2-->alice. >>>> 2. foo-->R1-->join2-->R3-->alice. >>>> >>>> In this testcase, it simulates two packet, one's destination is >>>> 172.16.1.2, another is 172.16.1.4. The mulitpath that was configured in R1 >>>> can seperate those traffics to R2/R3. Finally, 172.16.1.2 packet travels >>>> path2, 172.16.1.4 packet travels path1 >>>> >>>> +------+ >>>> | foo | >>>> +------+ >>>> | >>>> | >>>> +------+ >>>> | R1 |---------+ >>>> +------+ | >>>> | | >>>> | | >>>> +------+ +-------+ >>>> | join | | join2 | >>>> +------+ +-------+ >>>> | | >>>> | | >>>> +------+ +-------+ >>>> | R2 | | R3 | >>>> +------+ +-------+ >>>> | | >>>> | | >>>> +-----------------+ >>>> | alice | >>>> +-----------------+ >>>> | | >>>> alice1 alice2 >>>> >>>> Please let me know if you have any question on it. :) >>>> >>>> Thanks >>>> Zhenyu Gao >>>> >>>> 2017-09-20 20:58 GMT+08:00 Miguel Angel Ajo Pelayo <[email protected] >>>> >: >>>> >>>>> Can you share an example of how this would benefit E/W routing. I'm >>>>> just not seeing the specific use case myself out of ignorance. >>>>> >>>>> It'd be great if you could explain how would it work between several >>>>> ports in the networks and routers (may be a diagram?) otherwise I can't be >>>>> really helpful reviewing :) >>>>> >>>>> Cheers, and thanks for the patience. >>>>> >>>>> On Wed, Sep 20, 2017 at 12:25 PM, Gao Zhenyu <[email protected]> >>>>> wrote: >>>>> >>>>>> Thanks for the suggestions! >>>>>> >>>>>> Not all Logical port has a real ofp_port connect with it. And >>>>>> bundle_load/bundle actions need real ovs port. >>>>>> Especially in ovn router port, all router port are virtual port which >>>>>> just a number/reg in our ovs-flows. >>>>>> >>>>>> This implement of multipath can seperate ovn east-west traffic, it >>>>>> helps dispatch traffic to gateways and routers easily. >>>>>> >>>>>> For south-north traffic, we can have bundle/bundle_load action to >>>>>> consider the remote tunnel up/down status. I would like to make it step >>>>>> by >>>>>> step and implement it in my next series patches. >>>>>> >>>>>> Thanks >>>>>> Zhenyu Gao >>>>>> >>>>>> 2017-09-20 17:53 GMT+08:00 Miguel Angel Ajo Pelayo < >>>>>> [email protected]>: >>>>>> >>>>>>> I'm not very familiar with multipath implementations, >>>>>>> >>>>>>> but would it be possible to use bundle( ouput action with hrw >>>>>>> algorithm instead of multipath calculation to a register?. >>>>>>> >>>>>>> I say this, because if you look at lib/multipath.c lib/bundle.c you >>>>>>> will find that bundle.c is going to consider the up/down status >>>>>>> (slave_enabled check) of the links. >>>>>>> >>>>>>> That way the controller doesn't need to modify any flow based on >>>>>>> link status. >>>>>>> >>>>>>> On Wed, Sep 20, 2017 at 5:45 AM, Gao Zhenyu <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Thansk for the questions. >>>>>>>> >>>>>>>> the multipath_port can be set via ovn-nbctl. >>>>>>>> Like : ovn-nbctl -- --id=@lrt create Logical_Router_Static_Route >>>>>>>> ip_prefix=0.0.0.0/0 nexthop=10.88.77.1 multipath_port=[mp1,mp2] -- >>>>>>>> add Logical_Router edge1 static_routes @lrt >>>>>>>> This patch haven't implement a ovn-nbctl command to configure >>>>>>>> multipath routing. Because I am still considering reusing nexthop or >>>>>>>> output_port(make them become array entries), and want to collect >>>>>>>> suggestions on it. >>>>>>>> >>>>>>>> About the status of next -hop, I would like to introduce >>>>>>>> bundle_load and bfd to make it later. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Zhenyu Gao >>>>>>>> >>>>>>>> 2017-09-20 11:13 GMT+08:00 <[email protected]>: >>>>>>>> >>>>>>>>> How to configure multipath_port in static_route? I think the the >>>>>>>>> multipath >>>>>>>>> can be figured out from exist data of static_route, may not need >>>>>>>>> to add >>>>>>>>> this multipath_port column. >>>>>>>>> >>>>>>>>> And I think we should add a status column to indicate the nexthop >>>>>>>>> state. >>>>>>>>> When some of nexthop in multipath is down, ovn should change the >>>>>>>>> correspond flows. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Zhenyu Gao <[email protected]> >>>>>>>>> 发件人： [email protected] >>>>>>>>> 2017/09/19 19:37 >>>>>>>>> >>>>>>>>> 收件人： [email protected], [email protected], >>>>>>>>> [email protected], [email protected], [email protected], >>>>>>>>> 抄送： >>>>>>>>> 主题： [ovs-dev] [PATCH v1 1/3] Add multipath static router >>>>>>>>> in >>>>>>>>> OVN northd and north-db >>>>>>>>> >>>>>>>>> >>>>>>>>> 1. ovn-nb.ovsschema was updated to add new field multipath_port. >>>>>>>>> 2. Add multipath feature in ovn-northd part. northd generates >>>>>>>>> multipath >>>>>>>>> flows to dispatch traffic by using packet's IP dst address if user >>>>>>>>> set >>>>>>>>> Logical_Router_Static_Route's multipath_port with ports. >>>>>>>>> 3. Add new table(lr_in_multipath) in ovn-northd's router ingress >>>>>>>>> stages >>>>>>>>> to dispatch traffic to ports. >>>>>>>>> 4. Add multipath flow in Table 5(lr_in_ip_routing) and store hash >>>>>>>>> result >>>>>>>>> into reg0. reg9[2] was used to indicate packet which need >>>>>>>>> dispatching. >>>>>>>>> 5. Add multipath feature description in ovn/northd/ovn-northd.8.xml >>>>>>>>> and ovn/ovn-nb.xml >>>>>>>>> >>>>>>>>> Signed-off-by: Zhenyu Gao <[email protected]> >>>>>>>>> --- >>>>>>>>> ovn/northd/ovn-northd.8.xml | 67 +++++++++++- >>>>>>>>> ovn/northd/ovn-northd.c | 245 >>>>>>>>> ++++++++++++++++++++++++++++++++++++++------ >>>>>>>>> ovn/ovn-nb.ovsschema | 6 +- >>>>>>>>> ovn/ovn-nb.xml | 9 ++ >>>>>>>>> 4 files changed, 289 insertions(+), 38 deletions(-) >>>>>>>>> >>>>>>>>> diff --git a/ovn/northd/ovn-northd.8.xml >>>>>>>>> b/ovn/northd/ovn-northd.8.xml >>>>>>>>> index 0d85ec0..b1ce9a9 100644 >>>>>>>>> --- a/ovn/northd/ovn-northd.8.xml >>>>>>>>> +++ b/ovn/northd/ovn-northd.8.xml >>>>>>>>> @@ -1598,6 +1598,9 @@ icmp4 { >>>>>>>>> port (ingress table <code>ARP Request</code> will generate >>>>>>>>> an ARP >>>>>>>>> request, if needed, with <code>reg0</code> as the target >>>>>>>>> protocol >>>>>>>>> address and <code>reg1</code> as the source protocol >>>>>>>>> address). >>>>>>>>> + A IP route can be configured that it has multipath to >>>>>>>>> next-hop. >>>>>>>>> + If a packet has multipath to destination, OVN assign the >>>>>>>>> port >>>>>>>>> + index into reg[0] to indicate the packet's output port in >>>>>>>>> table 6. >>>>>>>>> </p> >>>>>>>>> >>>>>>>>> <p> >>>>>>>>> @@ -1617,6 +1620,28 @@ icmp4 { >>>>>>>>> >>>>>>>>> <li> >>>>>>>>> <p> >>>>>>>>> + IPv4/IPV6 multipath routing table. For each route to >>>>>>>>> IPv4/IPv6 >>>>>>>>> + network <var>N</var> with netmask <var>M</var>, on >>>>>>>>> multipath >>>>>>>>> port >>>>>>>>> + <var>P</var> with IP address <var>A</var> and Ethernet >>>>>>>>> + address <var>E</var>, a logical flow with match >>>>>>>>> + <code>ip4.dst ==<var>N</var>/<var>M</var></code>,whose >>>>>>>>> priority >>>>>>>>> + is the number of 1-bits plus 10 in <var>M</var>, >>>>>>>>> + has the following actions: >>>>>>>>> + </p> >>>>>>>>> + >>>>>>>>> + <pre> >>>>>>>>> +ip.ttl--; >>>>>>>>> +multipath (nw_dst, 0, modulo_n, <var>n_links</var>, 0, reg0); >>>>>>>>> +reg9[2] = 1 >>>>>>>>> +next; >>>>>>>>> + </pre> >>>>>>>>> + <p> >>>>>>>>> + <var>n_links</var> is the number of multipath port. >>>>>>>>> + </p> >>>>>>>>> + </li> >>>>>>>>> + >>>>>>>>> + <li> >>>>>>>>> + <p> >>>>>>>>> IPv4 routing table. For each route to IPv4 network >>>>>>>>> <var>N</var> with >>>>>>>>> netmask <var>M</var>, on router port <var>P</var> with >>>>>>>>> IP >>>>>>>>> address >>>>>>>>> <var>A</var> and Ethernet >>>>>>>>> @@ -1686,7 +1711,43 @@ next; >>>>>>>>> </li> >>>>>>>>> </ul> >>>>>>>>> >>>>>>>>> - <h3>Ingress Table 6: ARP/ND Resolution</h3> >>>>>>>>> + <h3>Ingress Table 6: Multipath</h3> >>>>>>>>> + <p> >>>>>>>>> + Any packet taht reaches this table is an IP packet and >>>>>>>>> reg9[2]=1 >>>>>>>>> + using the following flows to route to corresponding port. >>>>>>>>> This >>>>>>>>> table >>>>>>>>> + implement dispatching by consuming reg0. >>>>>>>>> + </p> >>>>>>>>> + >>>>>>>>> + <ul> >>>>>>>>> + <li> >>>>>>>>> + <p> >>>>>>>>> + A packet with netmask <var>M</var>, IP address >>>>>>>>> <var>A</var> and >>>>>>>>> + <code>reg9[2] = 1</code>, whose priority above 1 has >>>>>>>>> following >>>>>>>>> + actions: >>>>>>>>> + </p> >>>>>>>>> + >>>>>>>>> + <pre> >>>>>>>>> +reg0 = <var>G</var>; >>>>>>>>> +reg1 = <var>A</var>; >>>>>>>>> +eth.src = <var>E</var>; >>>>>>>>> +outport = <var>P</var>; >>>>>>>>> +flags.loopback = 1; >>>>>>>>> +next; >>>>>>>>> + </pre> >>>>>>>>> + >>>>>>>>> + <p> >>>>>>>>> + <var>G</var> is the gateway IP address. <var>A</var>, >>>>>>>>> <var>E</var> >>>>>>>>> + and <var>P</var> are the values that were described in >>>>>>>>> multipath >>>>>>>>> + routeing in table 5 >>>>>>>>> + </p> >>>>>>>>> + >>>>>>>>> + <p> >>>>>>>>> + A priority-0 logical flow with match has actions >>>>>>>>> <code>next;</code>. >>>>>>>>> + </p> >>>>>>>>> + </li> >>>>>>>>> + </ul> >>>>>>>>> + >>>>>>>>> + <h3>Ingress Table 7: ARP/ND Resolution</h3> >>>>>>>>> >>>>>>>>> <p> >>>>>>>>> Any packet that reaches this table is an IP packet whose >>>>>>>>> next-hop >>>>>>>>> @@ -1779,7 +1840,7 @@ next; >>>>>>>>> </li> >>>>>>>>> </ul> >>>>>>>>> >>>>>>>>> - <h3>Ingress Table 7: Gateway Redirect</h3> >>>>>>>>> + <h3>Ingress Table 8: Gateway Redirect</h3> >>>>>>>>> >>>>>>>>> <p> >>>>>>>>> For distributed logical routers where one of the logical >>>>>>>>> router >>>>>>>>> @@ -1836,7 +1897,7 @@ next; >>>>>>>>> </li> >>>>>>>>> </ul> >>>>>>>>> >>>>>>>>> - <h3>Ingress Table 8: ARP Request</h3> >>>>>>>>> + <h3>Ingress Table 9: ARP Request</h3> >>>>>>>>> >>>>>>>>> <p> >>>>>>>>> In the common case where the Ethernet destination has been >>>>>>>>> resolved, this >>>>>>>>> diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c >>>>>>>>> index 49e4ac3..44d1fd4 100644 >>>>>>>>> --- a/ovn/northd/ovn-northd.c >>>>>>>>> +++ b/ovn/northd/ovn-northd.c >>>>>>>>> @@ -135,9 +135,10 @@ enum ovn_stage { >>>>>>>>> PIPELINE_STAGE(ROUTER, IN, UNSNAT, 3, "lr_in_unsnat") >>>>>>>>> \ >>>>>>>>> PIPELINE_STAGE(ROUTER, IN, DNAT, 4, "lr_in_dnat") >>>>>>>>> \ >>>>>>>>> PIPELINE_STAGE(ROUTER, IN, IP_ROUTING, 5, >>>>>>>>> "lr_in_ip_routing") \ >>>>>>>>> - PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 6, >>>>>>>>> "lr_in_arp_resolve") \ >>>>>>>>> - PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 7, >>>>>>>>> "lr_in_gw_redirect") \ >>>>>>>>> - PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 8, >>>>>>>>> "lr_in_arp_request") \ >>>>>>>>> + PIPELINE_STAGE(ROUTER, IN, MULTIPATH, 6, >>>>>>>>> "lr_in_multipath") \ >>>>>>>>> + PIPELINE_STAGE(ROUTER, IN, ARP_RESOLVE, 7, >>>>>>>>> "lr_in_arp_resolve") \ >>>>>>>>> + PIPELINE_STAGE(ROUTER, IN, GW_REDIRECT, 8, >>>>>>>>> "lr_in_gw_redirect") \ >>>>>>>>> + PIPELINE_STAGE(ROUTER, IN, ARP_REQUEST, 9, >>>>>>>>> "lr_in_arp_request") \ >>>>>>>>> >>>>>>>>> \ >>>>>>>>> /* Logical router egress stages. */ >>>>>>>>> \ >>>>>>>>> PIPELINE_STAGE(ROUTER, OUT, UNDNAT, 0, "lr_out_undnat") >>>>>>>>> \ >>>>>>>>> @@ -173,6 +174,11 @@ enum ovn_stage { >>>>>>>>> * one of the logical router's own IP addresses. */ >>>>>>>>> #define REGBIT_EGRESS_LOOPBACK "reg9[1]" >>>>>>>>> >>>>>>>>> +/* Indicate multipath action has process this packet and store >>>>>>>>> hash >>>>>>>>> result >>>>>>>>> + * into other regX. Should consume the hash result to determin >>>>>>>>> the right >>>>>>>>> + * output port. */ >>>>>>>>> +#define REGBIT_MULTIPATH "reg9[2]" >>>>>>>>> + >>>>>>>>> /* Returns an "enum ovn_stage" built from the arguments. */ >>>>>>>>> static enum ovn_stage >>>>>>>>> ovn_stage_build(enum ovn_datapath_type dp_type, enum ovn_pipeline >>>>>>>>> pipeline, >>>>>>>>> @@ -4142,72 +4148,165 @@ add_route(struct hmap *lflows, const >>>>>>>>> struct >>>>>>>>> ovn_port *op, >>>>>>>>> } >>>>>>>>> >>>>>>>>> static void >>>>>>>>> -build_static_route_flow(struct hmap *lflows, struct ovn_datapath >>>>>>>>> *od, >>>>>>>>> - struct hmap *ports, >>>>>>>>> - const struct >>>>>>>>> nbrec_logical_router_static_route >>>>>>>>> *route) >>>>>>>>> +add_multipath_route(struct hmap *lflows, uint32_t port_num, >>>>>>>>> + struct ovn_port **out_ports, >>>>>>>>> + const char **lrp_addr_s, >>>>>>>>> + struct ovn_datapath *od, >>>>>>>>> + const char *network_s, int plen, >>>>>>>>> + const char *gateway, const char *policy) >>>>>>>>> +{ >>>>>>>>> + bool is_ipv4 = strchr(network_s, '.') ? true : false; >>>>>>>>> + struct ds match = DS_EMPTY_INITIALIZER; >>>>>>>>> + const char *dir; >>>>>>>>> + uint16_t priority; >>>>>>>>> + >>>>>>>>> + if (policy && !strcmp(policy, "src-ip")) { >>>>>>>>> + dir = "src"; >>>>>>>>> + priority = plen * 2; >>>>>>>>> + } else { >>>>>>>>> + dir = "dst"; >>>>>>>>> + priority = (plen * 2) + 1; >>>>>>>>> + } >>>>>>>>> + >>>>>>>>> + /* Set higer priority than regular route. */ >>>>>>>>> + priority += 10; >>>>>>>>> + >>>>>>>>> + ds_put_format(&match, "ip%s.%s == %s/%d", is_ipv4 ? "4" : >>>>>>>>> "6", dir, >>>>>>>>> + network_s, plen); >>>>>>>>> + >>>>>>>>> + struct ds actions = DS_EMPTY_INITIALIZER; >>>>>>>>> + >>>>>>>>> + ds_put_format(&actions, "ip.ttl--; "); >>>>>>>>> + ds_put_format(&actions, >>>>>>>>> + "multipath (nw_dst, 0, modulo_n, %u, 0, reg0); " >>>>>>>>> + "%s = 1; " >>>>>>>>> + "next;", >>>>>>>>> + port_num, REGBIT_MULTIPATH); >>>>>>>>> + >>>>>>>>> + /* The priority here is calculated to implement >>>>>>>>> longest-prefix-match >>>>>>>>> + * routing. */ >>>>>>>>> + ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, priority, >>>>>>>>> + ds_cstr(&match), ds_cstr(&actions)); >>>>>>>>> + >>>>>>>>> + for (int i = 0; i < port_num; i++) { >>>>>>>>> + struct ds mp_match = DS_EMPTY_INITIALIZER; >>>>>>>>> + struct ds mp_actions = DS_EMPTY_INITIALIZER; >>>>>>>>> + >>>>>>>>> + ds_put_format(&mp_match, "%s == 1 && reg0 == %d && ", >>>>>>>>> + REGBIT_MULTIPATH, i); >>>>>>>>> + ds_put_format(&mp_match, "ip%s.%s == %s/%d", >>>>>>>>> + is_ipv4 ? "4" : "6", dir, >>>>>>>>> + network_s, plen); >>>>>>>>> + >>>>>>>>> + ds_put_format(&mp_actions, "%sreg0 = ", is_ipv4 ? "" : >>>>>>>>> "xx"); >>>>>>>>> + if (gateway) { >>>>>>>>> + ds_put_cstr(&mp_actions, gateway); >>>>>>>>> + } else { >>>>>>>>> + ds_put_format(&mp_actions, "ip%s.dst", is_ipv4 ? "4" >>>>>>>>> : "6"); >>>>>>>>> + } >>>>>>>>> + >>>>>>>>> + ds_put_format(&mp_actions, "; " >>>>>>>>> + "%sreg1 = %s; " >>>>>>>>> + "eth.src = %s; " >>>>>>>>> + "outport = %s; " >>>>>>>>> + "flags.loopback = 1; " >>>>>>>>> + "next;", >>>>>>>>> + is_ipv4 ? "" : "xx", >>>>>>>>> + lrp_addr_s[i], >>>>>>>>> + out_ports[i]->lrp_networks.ea_s, >>>>>>>>> + out_ports[i]->json_key); >>>>>>>>> + >>>>>>>>> + /* Add flow in table 6 to determin the right output port >>>>>>>>> + * for this traffic. */ >>>>>>>>> + ovn_lflow_add(lflows, od, S_ROUTER_IN_MULTIPATH, priority, >>>>>>>>> + ds_cstr(&mp_match), ds_cstr(&mp_actions)); >>>>>>>>> + ds_destroy(&mp_match); >>>>>>>>> + ds_destroy(&mp_actions); >>>>>>>>> + } >>>>>>>>> + ds_destroy(&match); >>>>>>>>> + ds_destroy(&actions); >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> +static bool >>>>>>>>> +verify_nexthop_prefix(const struct nbrec_logical_router_static_ro >>>>>>>>> ute >>>>>>>>> *route, >>>>>>>>> + bool *is_ipv4, char **prefix_s, unsigned >>>>>>>>> int *plen) >>>>>>>>> { >>>>>>>>> ovs_be32 nexthop; >>>>>>>>> - const char *lrp_addr_s = NULL; >>>>>>>>> - unsigned int plen; >>>>>>>>> - bool is_ipv4; >>>>>>>>> >>>>>>>>> /* Verify that the next hop is an IP address with an all-ones >>>>>>>>> mask. >>>>>>>>> */ >>>>>>>>> - char *error = ip_parse_cidr(route->nexthop, &nexthop, &plen); >>>>>>>>> + char *error = ip_parse_cidr(route->nexthop, &nexthop, plen); >>>>>>>>> if (!error) { >>>>>>>>> - if (plen != 32) { >>>>>>>>> + if (*plen != 32) { >>>>>>>>> static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, >>>>>>>>> 1); >>>>>>>>> VLOG_WARN_RL(&rl, "bad next hop mask %s", >>>>>>>>> route->nexthop); >>>>>>>>> - return; >>>>>>>>> + return false; >>>>>>>>> } >>>>>>>>> - is_ipv4 = true; >>>>>>>>> + *is_ipv4 = true; >>>>>>>>> } else { >>>>>>>>> free(error); >>>>>>>>> >>>>>>>>> struct in6_addr ip6; >>>>>>>>> - error = ipv6_parse_cidr(route->nexthop, &ip6, &plen); >>>>>>>>> + error = ipv6_parse_cidr(route->nexthop, &ip6, plen); >>>>>>>>> if (!error) { >>>>>>>>> - if (plen != 128) { >>>>>>>>> + if (*plen != 128) { >>>>>>>>> static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, 1); >>>>>>>>> VLOG_WARN_RL(&rl, "bad next hop mask %s", >>>>>>>>> route->nexthop); >>>>>>>>> - return; >>>>>>>>> + return false; >>>>>>>>> } >>>>>>>>> - is_ipv4 = false; >>>>>>>>> + *is_ipv4 = false; >>>>>>>>> } else { >>>>>>>>> static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, >>>>>>>>> 1); >>>>>>>>> VLOG_WARN_RL(&rl, "bad next hop ip address %s", >>>>>>>>> route->nexthop); >>>>>>>>> free(error); >>>>>>>>> - return; >>>>>>>>> + return false; >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> - char *prefix_s; >>>>>>>>> - if (is_ipv4) { >>>>>>>>> + if (*is_ipv4) { >>>>>>>>> ovs_be32 prefix; >>>>>>>>> /* Verify that ip prefix is a valid IPv4 address. */ >>>>>>>>> - error = ip_parse_cidr(route->ip_prefix, &prefix, &plen); >>>>>>>>> + error = ip_parse_cidr(route->ip_prefix, &prefix, plen); >>>>>>>>> if (error) { >>>>>>>>> static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, >>>>>>>>> 1); >>>>>>>>> VLOG_WARN_RL(&rl, "bad 'ip_prefix' in static routes >>>>>>>>> %s", >>>>>>>>> route->ip_prefix); >>>>>>>>> free(error); >>>>>>>>> - return; >>>>>>>>> + return false; >>>>>>>>> } >>>>>>>>> - prefix_s = xasprintf(IP_FMT, IP_ARGS(prefix & >>>>>>>>> be32_prefix_mask(plen))); >>>>>>>>> + *prefix_s = xasprintf(IP_FMT, IP_ARGS(prefix >>>>>>>>> + & >>>>>>>>> be32_prefix_mask(*plen))); >>>>>>>>> } else { >>>>>>>>> /* Verify that ip prefix is a valid IPv6 address. */ >>>>>>>>> struct in6_addr prefix; >>>>>>>>> - error = ipv6_parse_cidr(route->ip_prefix, &prefix, >>>>>>>>> &plen); >>>>>>>>> + error = ipv6_parse_cidr(route->ip_prefix, &prefix, plen); >>>>>>>>> if (error) { >>>>>>>>> static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, >>>>>>>>> 1); >>>>>>>>> VLOG_WARN_RL(&rl, "bad 'ip_prefix' in static routes >>>>>>>>> %s", >>>>>>>>> route->ip_prefix); >>>>>>>>> free(error); >>>>>>>>> - return; >>>>>>>>> + return false; >>>>>>>>> } >>>>>>>>> - struct in6_addr mask = ipv6_create_mask(plen); >>>>>>>>> + struct in6_addr mask = ipv6_create_mask(*plen); >>>>>>>>> struct in6_addr network = ipv6_addr_bitand(&prefix, >>>>>>>>> &mask); >>>>>>>>> - prefix_s = xmalloc(INET6_ADDRSTRLEN); >>>>>>>>> - inet_ntop(AF_INET6, &network, prefix_s, INET6_ADDRSTRLEN); >>>>>>>>> + *prefix_s = xmalloc(INET6_ADDRSTRLEN); >>>>>>>>> + inet_ntop(AF_INET6, &network, *prefix_s, >>>>>>>>> INET6_ADDRSTRLEN); >>>>>>>>> + } >>>>>>>>> + >>>>>>>>> + return true; >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> +static void >>>>>>>>> +build_static_route_flow(struct hmap *lflows, struct ovn_datapath >>>>>>>>> *od, >>>>>>>>> + struct hmap *ports, >>>>>>>>> + const struct >>>>>>>>> nbrec_logical_router_static_route >>>>>>>>> *route) >>>>>>>>> +{ >>>>>>>>> + const char *lrp_addr_s = NULL; >>>>>>>>> + unsigned int plen; >>>>>>>>> + bool is_ipv4; >>>>>>>>> + char *prefix_s = NULL; >>>>>>>>> + >>>>>>>>> + if (!verify_nexthop_prefix(route, &is_ipv4, &prefix_s, >>>>>>>>> &plen)) { >>>>>>>>> + return; >>>>>>>>> } >>>>>>>>> >>>>>>>>> /* Find the outgoing port. */ >>>>>>>>> @@ -4270,7 +4369,75 @@ build_static_route_flow(struct hmap >>>>>>>>> *lflows, struct >>>>>>>>> ovn_datapath *od, >>>>>>>>> policy); >>>>>>>>> >>>>>>>>> free_prefix_s: >>>>>>>>> - free(prefix_s); >>>>>>>>> + if (prefix_s) { >>>>>>>>> + free(prefix_s); >>>>>>>>> + } >>>>>>>>> +} >>>>>>>>> + >>>>>>>>> +static void >>>>>>>>> +build_multipath_flow(struct hmap *lflows, struct ovn_datapath *od, >>>>>>>>> + struct hmap *ports, >>>>>>>>> + const struct nbrec_logical_router_static_ro >>>>>>>>> ute >>>>>>>>> *route) >>>>>>>>> +{ >>>>>>>>> + unsigned int plen; >>>>>>>>> + bool is_ipv4; >>>>>>>>> + char *prefix_s = NULL; >>>>>>>>> + >>>>>>>>> + if (!verify_nexthop_prefix(route, &is_ipv4, &prefix_s, >>>>>>>>> &plen)) { >>>>>>>>> + return; >>>>>>>>> + } >>>>>>>>> + >>>>>>>>> + /* Find the outgoing port. */ >>>>>>>>> + struct ovn_port **out_ports = xmalloc(route->n_multipath_port >>>>>>>>> * >>>>>>>>> + sizeof(struct >>>>>>>>> ovn_port *)); >>>>>>>>> + const char **lrp_addr_s = xmalloc(route->n_multipath_port * >>>>>>>>> + sizeof(const char *)); >>>>>>>>> + for (int i = 0; i < route->n_multipath_port; i++) { >>>>>>>>> + // TODO May need to consider some ports are not found? >>>>>>>>> + out_ports[i] = ovn_port_find(ports, >>>>>>>>> route->multipath_port[i]); >>>>>>>>> + if (!out_ports[i]) { >>>>>>>>> + static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, >>>>>>>>> 1); >>>>>>>>> + VLOG_WARN_RL(&rl, "Bad out port %s for static route >>>>>>>>> %s", >>>>>>>>> + route->multipath_port[i], >>>>>>>>> route->ip_prefix); >>>>>>>>> + goto free_ports_lrp_addr; >>>>>>>>> + } >>>>>>>>> + >>>>>>>>> + lrp_addr_s[i] = find_lrp_member_ip(out_ports[i], >>>>>>>>> route->nexthop); >>>>>>>>> + if (!lrp_addr_s[i]) { >>>>>>>>> + if (is_ipv4) { >>>>>>>>> + if (out_ports[i]->lrp_networks.n_ipv4_addrs) { >>>>>>>>> + lrp_addr_s[i] = out_ports[i]-> >>>>>>>>> + lrp_networks.ipv4_addrs[0].addr_s; >>>>>>>>> + } >>>>>>>>> + } else { >>>>>>>>> + if (out_ports[i]->lrp_networks.n_ipv6_addrs) { >>>>>>>>> + lrp_addr_s[i] = out_ports[i]-> >>>>>>>>> + lrp_networks.ipv6_addrs[0].addr_s; >>>>>>>>> + } >>>>>>>>> + } >>>>>>>>> + } >>>>>>>>> + if (!lrp_addr_s[i]) { >>>>>>>>> + static struct vlog_rate_limit rl = >>>>>>>>> VLOG_RATE_LIMIT_INIT(5, >>>>>>>>> 1); >>>>>>>>> + VLOG_WARN_RL(&rl, >>>>>>>>> + "%s has no path for static route %s; >>>>>>>>> next hop >>>>>>>>> %s", >>>>>>>>> + route->multipath_port[i], >>>>>>>>> route->ip_prefix, >>>>>>>>> + route->nexthop); >>>>>>>>> + goto free_ports_lrp_addr; >>>>>>>>> + } >>>>>>>>> + } >>>>>>>>> + >>>>>>>>> + >>>>>>>>> + char *policy = route->policy ? route->policy : "dst-ip"; >>>>>>>>> + add_multipath_route(lflows, route->n_multipath_port, >>>>>>>>> + out_ports, lrp_addr_s, od, >>>>>>>>> + prefix_s, plen, route->nexthop, policy); >>>>>>>>> + >>>>>>>>> +free_ports_lrp_addr: >>>>>>>>> + free(out_ports); >>>>>>>>> + free(lrp_addr_s); >>>>>>>>> + if (prefix_s) { >>>>>>>>> + free(prefix_s); >>>>>>>>> + } >>>>>>>>> } >>>>>>>>> >>>>>>>>> static void >>>>>>>>> @@ -5344,7 +5511,7 @@ build_lrouter_flows(struct hmap *datapaths, >>>>>>>>> struct >>>>>>>>> hmap *ports, >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> - /* Convert the static routes to flows. */ >>>>>>>>> + /* Convert the static routes and multipath route to flows. */ >>>>>>>>> HMAP_FOR_EACH (od, key_node, datapaths) { >>>>>>>>> if (!od->nbr) { >>>>>>>>> continue; >>>>>>>>> @@ -5355,12 +5522,24 @@ build_lrouter_flows(struct hmap >>>>>>>>> *datapaths, struct >>>>>>>>> hmap *ports, >>>>>>>>> >>>>>>>>> route = od->nbr->static_routes[i]; >>>>>>>>> build_static_route_flow(lflows, od, ports, route); >>>>>>>>> + /* Logical router ingress table 5-6: Multipath >>>>>>>>> Routing. >>>>>>>>> + * >>>>>>>>> + * If router has configured a traffic has multiple >>>>>>>>> paths >>>>>>>>> + * to destination. The right output port should be >>>>>>>>> firgured >>>>>>>>> + * out by computing IP packet's header */ >>>>>>>>> + if (route->n_multipath_port > 1) { >>>>>>>>> + /* Generate multipath routes in table 5,6 for >>>>>>>>> + * dedicated traffic */ >>>>>>>>> + build_multipath_flow(lflows, od, ports, route); >>>>>>>>> + } >>>>>>>>> } >>>>>>>>> + /* Packets are allowed by default in table 6. */ >>>>>>>>> + ovn_lflow_add(lflows, od, S_ROUTER_IN_MULTIPATH, 0, "1", >>>>>>>>> "next;"); >>>>>>>>> } >>>>>>>>> >>>>>>>>> /* XXX destination unreachable */ >>>>>>>>> >>>>>>>>> - /* Local router ingress table 6: ARP Resolution. >>>>>>>>> + /* Local router ingress table 7: ARP Resolution. >>>>>>>>> * >>>>>>>>> * Any packet that reaches this table is an IP packet whose >>>>>>>>> next-hop >>>>>>>>> IP >>>>>>>>> * address is in reg0. (ip4.dst is the final destination.) >>>>>>>>> This table >>>>>>>>> @@ -5555,7 +5734,7 @@ build_lrouter_flows(struct hmap *datapaths, >>>>>>>>> struct >>>>>>>>> hmap *ports, >>>>>>>>> "get_nd(outport, xxreg0); next;"); >>>>>>>>> } >>>>>>>>> >>>>>>>>> - /* Logical router ingress table 7: Gateway redirect. >>>>>>>>> + /* Logical router ingress table 8: Gateway redirect. >>>>>>>>> * >>>>>>>>> * For traffic with outport equal to the l3dgw_port >>>>>>>>> * on a distributed router, this table redirects a subset >>>>>>>>> @@ -5595,7 +5774,7 @@ build_lrouter_flows(struct hmap *datapaths, >>>>>>>>> struct >>>>>>>>> hmap *ports, >>>>>>>>> ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 0, "1", >>>>>>>>> "next;"); >>>>>>>>> } >>>>>>>>> >>>>>>>>> - /* Local router ingress table 8: ARP request. >>>>>>>>> + /* Local router ingress table 9: ARP request. >>>>>>>>> * >>>>>>>>> * In the common case where the Ethernet destination has been >>>>>>>>> resolved, >>>>>>>>> * this table outputs the packet (priority 0). Otherwise, it >>>>>>>>> composes >>>>>>>>> diff --git a/ovn/ovn-nb.ovsschema b/ovn/ovn-nb.ovsschema >>>>>>>>> index a077bfb..b8bdd42 100644 >>>>>>>>> --- a/ovn/ovn-nb.ovsschema >>>>>>>>> +++ b/ovn/ovn-nb.ovsschema >>>>>>>>> @@ -1,7 +1,7 @@ >>>>>>>>> { >>>>>>>>> "name": "OVN_Northbound", >>>>>>>>> "version": "5.8.0", >>>>>>>>> - "cksum": "2812300190 <(281)%20230-0190> 16766", >>>>>>>>> + "cksum": "1967092589 16903", >>>>>>>>> "tables": { >>>>>>>>> "NB_Global": { >>>>>>>>> "columns": { >>>>>>>>> @@ -235,7 +235,9 @@ >>>>>>>>> >>>>>>>>> "dst-ip"]]}, >>>>>>>>> "min": 0, "max": 1}}, >>>>>>>>> "nexthop": {"type": "string"}, >>>>>>>>> - "output_port": {"type": {"key": "string", "min": >>>>>>>>> 0, >>>>>>>>> "max": 1}}}, >>>>>>>>> + "output_port": {"type": {"key": "string", "min": >>>>>>>>> 0, >>>>>>>>> "max": 1}}, >>>>>>>>> + "multipath_port": {"type": {"key": "string", >>>>>>>>> "min": 0, >>>>>>>>> + "max": "unlimited"}}}, >>>>>>>>> "isRoot": false}, >>>>>>>>> "NAT": { >>>>>>>>> "columns": { >>>>>>>>> diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml >>>>>>>>> index 9869d7e..15feb97 100644 >>>>>>>>> --- a/ovn/ovn-nb.xml >>>>>>>>> +++ b/ovn/ovn-nb.xml >>>>>>>>> @@ -1487,6 +1487,15 @@ >>>>>>>>> address as the one via which the <ref column="nexthop"/> >>>>>>>>> is >>>>>>>>> reachable. >>>>>>>>> </p> >>>>>>>>> </column> >>>>>>>>> + <column name="multipath_port"> >>>>>>>>> + <p> >>>>>>>>> + The name of the <ref table="Logical_Router_Port"/> via >>>>>>>>> which the >>>>>>>>> packet >>>>>>>>> + needs to be sent out. When it contains more than two >>>>>>>>> ports, it >>>>>>>>> means >>>>>>>>> + packet has multiple candidate output ports. OVN uses the >>>>>>>>> packet >>>>>>>>> header >>>>>>>>> + to determin which port the packet would be delivered to. >>>>>>>>> + Currently, OVN consumes destination IP address to figure >>>>>>>>> out >>>>>>>>> port. >>>>>>>>> + </p> >>>>>>>>> + </column> >>>>>>>>> </table> >>>>>>>>> >>>>>>>>> <table name="NAT" title="NAT rules"> >>>>>>>>> -- >>>>>>>>> 1.8.3.1 >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> dev mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> dev mailing list >>>>>>>>> [email protected] >>>>>>>>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
