On Thu, May 12, 2016 at 4:55 PM, Guru Shetty <g...@ovn.org> wrote: > > > On 12 May 2016 at 16:34, Darrell Ball <dlu...@gmail.com> wrote: > >> On Thu, May 12, 2016 at 10:54 AM, Guru Shetty <g...@ovn.org> wrote: >> >> > >> >> I think you misunderstood - having one or more gateway per tenant does >> >> not make Transit LS better in flow scale. >> >> The size of a Transit LS subnet and management across Transit LSs is >> one >> >> the 5 issues I mentioned and it remains the same >> >> as do the other issues. >> >> >> >> Based on the example with 1000 HVs, 1000 tenants, 1000 HV/tenant, one >> >> distributed logical router per tenant >> >> spanning 1000 HVs, one gateway per tenant, we have a Transit LS with >> 1001 >> >> router type logical ports (1000 HVs + one gateway). >> >> >> >> Now, based on your previous assertion earlier: >> >> "If a user uses one gateway, a transit LS only gets connected by 2 >> >> routers. >> >> Other routers get their own transit LS." >> >> >> >> This translates: >> >> one Transit LS per tenant => 1000 Transit LS datapaths in total >> >> 1001 Transit LS logical ports per Transit LS => 1,001,000 Transit LS >> >> logical ports in total >> >> 1001 arp resolve flows per Transit LS => 1,001,000 flows just for arp >> >> resolve. >> >> Each Transit LS comes with many other flows: so we multiply that number >> >> of flows * 1000 Transit LSs = ? flows >> >> 1001 addresses per subnet per Transit LS; I suggested addresses should >> be >> >> reused across subnets, but when each subnet is large >> >> >> > >> > Re-reading. The above is a wrong conclusion making me believe that there >> > is a big disconnect. A subnet in transit LS has only 2 IP addresses (if >> it >> > is only one physical gateway). Every additional physical gateway can add >> > one additional IP address to the subnet (depending on whether the new >> > physical gateway has a gateway router added for that logical topology.). >> > >> >> >> With respect to the IP address usage. I think a diagram would help >> especially the K8 case, >> > Drawing a diagram here is not feasible. Happy to do it on a whiteboard > though. >
Thanks - lets do that; I would like to clarify the addressing requirements and full scope of distributed/gateway router interconnects for K8s. > > >> which I had heard in other conversations may have a separate gateway on >> every HV ?. Hence, I would like to know what that means - i.e. were you >> thinking to run separate gateways routers on every HV for K8 ? >> > Yes, thats the plan (as many as possible). 100 routers is a target. Not > HV, but a VM. > > > >> >> With respect to the other questions, I think its best approach would be to >> ask direct questions so those >> direct questions get answered. >> >> 1) With 1000 HVs, 1000 HVs/tenant, 1 distributed router per tenant, you >> choose the number of gateways/tenant: >> >> a) How many Transit LS distributed datapaths are expected in total ? >> > > One (i.e the same as the distributed router). > i.e. 1000 distributed routers => 1000 Transit LSs > > >> >> b) How many Transit LS logical ports are needed at the HV level ? >> >> what I mean by that is lets say we have one additional logical port at >> northd level and 1000 HVs then if we need to download that port to 1000 >> HVs, I consider that to be 1000 logical ports at the HV level because >> downloading and maintaining state across HVs at scale is more expensive >> than for a single hypervisor. >> > > 1000 additional ones. It is the same as your distributed logical switch or > logical router (this is the case even with the peer routers) > Did you mean 2000 including both ends of the Transit LS ? > > > >> >> c) How many Transit LS arp resolve entries at the HV level ? >> >> what I mean by that is lets say we have one additional arp resolve flow at >> northd level and 1000 HVs then if we need to download that arp resolve >> flow >> to 1000 HVs, I consider that to be 1000 flows at the HV level because >> downloading and maintaining state across multiple HVs is more expensive >> that a single hypervisor. >> > > 2 ARP flows per transit LS * 1000 HVs. > oops; I underestimated by half > Do realize that a single bridge on a single hypervisor typically has flows > in the 100,000 range. Even a million is feasbile. > I know. I am thinking about the coordination across many HVs. > Microsegmentation use cases has 10000 ACLs per logical switch. So that is > 10000 * 1000 for your case form single LS. So do you have some comparative > perspectives. > > > > dev mailing list >> dev@openvswitch.org >> http://openvswitch.org/mailman/listinfo/dev >> > > _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
