Re: [ovs-discuss] Question on distributing snat traffic with OVN
On Tue, May 4, 2021 at 11:40 AM Francois wrote:
>
> On Tue, 4 May 2021 at 19:24, Han Zhou wrote:
> >
> >
> > This is interesting. One question here. Not sure if I understand the
proposal correctly, but if you want to use the chassis's IP for snat, then
how would the return traffic hit the br-int? The return traffic (to the VM,
with destination IP being the chassis IP and destination mac address being
the chassis MAC) would go directly to the host interface (e.g. br0) without
going to the virtual network pipelines, right?
>
> I saw some user documentation.. where the ovn-encap-ip was handled by
> a br-tun bridge, and was hoping there would be hope...
>
> >
> > The original problem was to avoid using floating IPs per VM, but if it
is ok to have an extra IP per chassis, then I think current OVN
implementation already supports it by creating a per chassis Gateway Router
and configuring SNAT using the Gateway Router's uplink IP. Each chassis is
now both a HV and a gateway, and you will need one extra IP per chassis as
the Gateway Router IP. I think this is similar to the ovn-kubernetes
topology. Of course there may be other drawbacks such as you may need a
logical switch per chassis so that you can route the traffic to the
chassis's own Gateway Router. Not sure if it is something that could help
in your use cases.
> >
>
> What I wish to have is VMs without floating IP, from different
> tenants, having their traffic transparently (as in, no fIP or any
> particular thing to add from a user perspective), sent out from the
> chassis directly.
>
> Just to rephrase the suggestion, I can create a logical switch per
> chassis, then give to all the VMs on the chassis a new interface that
> connects them to the second network, and the routing rules on the VM
> must be configured so that the traffic to external is sent through
> that interface.
>
> If I understand correctly, I think it describes well my usecase (SNAT
> is well distributed), but you need some work from the tenant point of
> view ("if you want efficient snat you need to route traffic through
> that second interface") even if we patch OpenStack to magically add
> new ports to VMs.
>
> I am going to check ovn-kubernetes doc then.
In ovn-kubernetes it is done through a single interface per workload. Each
workload is connected to the chassis level LS, which has a default route to
a join router, but the join router can make decision based on src-route
policies to redirect external traffic back to the chassis level gateway
router. Physically this routing decision is made locally by OVS on the
chassis. Of course the logical topology and routing is a little complex,
while with dual interfaces per VM you could make it more flexible (but may
introduce some other operational challenges).
>
> >
> > There is a plan to remove the C version once the DDlog is stable
enough. The timeline is not clear (at least to me).
> > There is also a plan to rewrite ovn-controller in ddlog, but it is more
complex than northd and there are different options moving forward, and the
timeline is even less clear.
> >
> > Thanks,
> > Han
>
> Thanks!
> Francois
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Question on distributing snat traffic with OVN
On Tue, 4 May 2021 at 19:24, Han Zhou wrote:
>
>
> This is interesting. One question here. Not sure if I understand the proposal
> correctly, but if you want to use the chassis's IP for snat, then how would
> the return traffic hit the br-int? The return traffic (to the VM, with
> destination IP being the chassis IP and destination mac address being the
> chassis MAC) would go directly to the host interface (e.g. br0) without going
> to the virtual network pipelines, right?
I saw some user documentation.. where the ovn-encap-ip was handled by
a br-tun bridge, and was hoping there would be hope...
>
> The original problem was to avoid using floating IPs per VM, but if it is ok
> to have an extra IP per chassis, then I think current OVN implementation
> already supports it by creating a per chassis Gateway Router and configuring
> SNAT using the Gateway Router's uplink IP. Each chassis is now both a HV and
> a gateway, and you will need one extra IP per chassis as the Gateway Router
> IP. I think this is similar to the ovn-kubernetes topology. Of course there
> may be other drawbacks such as you may need a logical switch per chassis so
> that you can route the traffic to the chassis's own Gateway Router. Not sure
> if it is something that could help in your use cases.
>
What I wish to have is VMs without floating IP, from different
tenants, having their traffic transparently (as in, no fIP or any
particular thing to add from a user perspective), sent out from the
chassis directly.
Just to rephrase the suggestion, I can create a logical switch per
chassis, then give to all the VMs on the chassis a new interface that
connects them to the second network, and the routing rules on the VM
must be configured so that the traffic to external is sent through
that interface.
If I understand correctly, I think it describes well my usecase (SNAT
is well distributed), but you need some work from the tenant point of
view ("if you want efficient snat you need to route traffic through
that second interface") even if we patch OpenStack to magically add
new ports to VMs.
I am going to check ovn-kubernetes doc then.
>
> There is a plan to remove the C version once the DDlog is stable enough. The
> timeline is not clear (at least to me).
> There is also a plan to rewrite ovn-controller in ddlog, but it is more
> complex than northd and there are different options moving forward, and the
> timeline is even less clear.
>
> Thanks,
> Han
Thanks!
Francois
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Question on distributing snat traffic with OVN
On Tue, May 4, 2021 at 9:25 AM Francois wrote: > > On Tue, 4 May 2021 at 17:03, Numan Siddique wrote: > > > > On Sat, May 1, 2021 at 6:32 AM Francois wrote: > > > > > > Hi Open vSwitch > > > I am running an OVN stack with a dozen chassis, all of them able to > > > act as gateways. > > > I have many VMs without floating IPs on the same logical switch, doing > > > a lot of external traffic. Today, this traffic has to go through the > > > tunnel towards the unique chassis claiming the gateway to perform the > > > snat natting and send the traffic outside the stack. > > > > > > With this current design, I see a lot of BFD traffic, and a clear > > > bottleneck and spof with that single chassis doing the snat. A > > > workaround is to add floating IPs on each VM, but this means the end > > > user has to put the floating IP themself, it also means if a single > > > chassis runs 10 VMs, we need one floating IP per VM just for the snat, > > > while we could instead use a single IP per chassis for that. > > > > > > I was thinking of adding a "br-snat" bridge on each ovs, adding to it > > > one interface with a fixed IP, and (with some minimal development in > > > ovn northd) have the snat traffic of all its ports going out of that > > > interface instead of going through the tunnel towards the gateway. > > > Ideally the IP used today for the tunnel could be used too for the > > > snat traffic, but this seems less trivial to achieve. > > > > > > Before looking at the details of ddlog and the syntax of flows, I > > > would love to get some feedback on the idea, maybe there is something > > > fundamentally broken with my design, or maybe there is a smarter way > > > to achieve this? > > > > This is an interesting idea. In order to do snat, OVN should know what IP > > to use. But this IP should belong to the provider network subnet pool right ? > > > > If you think this can be done, you can probably attempt a quick PoC with just > > changes to the C version and post the patches as RFC. The ddlog part > > can be done later if the approach seems to be fine for the reviewers. > > > Ok! I have no idea if this can be done, but I will attempt something > nevertheless. You need one IP per chassis so if it is set on a different > interface, and statically (similar to the external-ids:ovn-encap-ip) > it should be fine too. > This is interesting. One question here. Not sure if I understand the proposal correctly, but if you want to use the chassis's IP for snat, then how would the return traffic hit the br-int? The return traffic (to the VM, with destination IP being the chassis IP and destination mac address being the chassis MAC) would go directly to the host interface (e.g. br0) without going to the virtual network pipelines, right? The original problem was to avoid using floating IPs per VM, but if it is ok to have an extra IP per chassis, then I think current OVN implementation already supports it by creating a per chassis Gateway Router and configuring SNAT using the Gateway Router's uplink IP. Each chassis is now both a HV and a gateway, and you will need one extra IP per chassis as the Gateway Router IP. I think this is similar to the ovn-kubernetes topology. Of course there may be other drawbacks such as you may need a logical switch per chassis so that you can route the traffic to the chassis's own Gateway Router. Not sure if it is something that could help in your use cases. > What will happen is that traffic going out will be seen from the outside, > as an IP of the chassis (and the compute running a VM is chosen at random > usually). If there are firewall rules to open (on a firewall seating on > the external network), they will need to be opened for all hypervisors, for > all VMs (so firewall rules become less relevant in a sense). It basically > "works" from a security standpoint, when all VMs belong to the same tenant. > Still since we are going to open whole ranges in the firewall, the IPs > should be limited to the IPs used for the SNAT, and not include any fIP, so it > should probably be a different subnet. > > I think (I am still reading the doc!) a somewhat similar work was done when > addressing the MTU issue with the redirect-type=bridged, where packets > are sent through a different port, using statically set mac-mappings. > > ddlog looked funnier! Do you have a plan for the removal of the C > version? Also is there still a plan to have ovn-controller rewritten in > ddlog? There is a plan to remove the C version once the DDlog is stable enough. The timeline is not clear (at least to me). There is also a plan to rewrite ovn-controller in ddlog, but it is more complex than northd and there are different options moving forward, and the timeline is even less clear. Thanks, Han > > Thanks a lot! > Francois > > > > > Thanks > > Numan > > > > > > > > Thanks > > > Francois > > > ___ > > > discuss mailing list > > > disc...@openvswitch.org > > > https://mail.
Re: [ovs-discuss] Question on distributing snat traffic with OVN
On Tue, 4 May 2021 at 17:03, Numan Siddique wrote: > > On Sat, May 1, 2021 at 6:32 AM Francois wrote: > > > > Hi Open vSwitch > > I am running an OVN stack with a dozen chassis, all of them able to > > act as gateways. > > I have many VMs without floating IPs on the same logical switch, doing > > a lot of external traffic. Today, this traffic has to go through the > > tunnel towards the unique chassis claiming the gateway to perform the > > snat natting and send the traffic outside the stack. > > > > With this current design, I see a lot of BFD traffic, and a clear > > bottleneck and spof with that single chassis doing the snat. A > > workaround is to add floating IPs on each VM, but this means the end > > user has to put the floating IP themself, it also means if a single > > chassis runs 10 VMs, we need one floating IP per VM just for the snat, > > while we could instead use a single IP per chassis for that. > > > > I was thinking of adding a "br-snat" bridge on each ovs, adding to it > > one interface with a fixed IP, and (with some minimal development in > > ovn northd) have the snat traffic of all its ports going out of that > > interface instead of going through the tunnel towards the gateway. > > Ideally the IP used today for the tunnel could be used too for the > > snat traffic, but this seems less trivial to achieve. > > > > Before looking at the details of ddlog and the syntax of flows, I > > would love to get some feedback on the idea, maybe there is something > > fundamentally broken with my design, or maybe there is a smarter way > > to achieve this? > > This is an interesting idea. In order to do snat, OVN should know what IP > to use. But this IP should belong to the provider network subnet pool right ? > > If you think this can be done, you can probably attempt a quick PoC with just > changes to the C version and post the patches as RFC. The ddlog part > can be done later if the approach seems to be fine for the reviewers. Ok! I have no idea if this can be done, but I will attempt something nevertheless. You need one IP per chassis so if it is set on a different interface, and statically (similar to the external-ids:ovn-encap-ip) it should be fine too. What will happen is that traffic going out will be seen from the outside, as an IP of the chassis (and the compute running a VM is chosen at random usually). If there are firewall rules to open (on a firewall seating on the external network), they will need to be opened for all hypervisors, for all VMs (so firewall rules become less relevant in a sense). It basically "works" from a security standpoint, when all VMs belong to the same tenant. Still since we are going to open whole ranges in the firewall, the IPs should be limited to the IPs used for the SNAT, and not include any fIP, so it should probably be a different subnet. I think (I am still reading the doc!) a somewhat similar work was done when addressing the MTU issue with the redirect-type=bridged, where packets are sent through a different port, using statically set mac-mappings. ddlog looked funnier! Do you have a plan for the removal of the C version? Also is there still a plan to have ovn-controller rewritten in ddlog? Thanks a lot! Francois > > Thanks > Numan > > > > > Thanks > > Francois > > ___ > > discuss mailing list > > disc...@openvswitch.org > > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Question on distributing snat traffic with OVN
On Sat, May 1, 2021 at 6:32 AM Francois wrote: > > Hi Open vSwitch > I am running an OVN stack with a dozen chassis, all of them able to > act as gateways. > I have many VMs without floating IPs on the same logical switch, doing > a lot of external traffic. Today, this traffic has to go through the > tunnel towards the unique chassis claiming the gateway to perform the > snat natting and send the traffic outside the stack. > > With this current design, I see a lot of BFD traffic, and a clear > bottleneck and spof with that single chassis doing the snat. A > workaround is to add floating IPs on each VM, but this means the end > user has to put the floating IP themself, it also means if a single > chassis runs 10 VMs, we need one floating IP per VM just for the snat, > while we could instead use a single IP per chassis for that. > > I was thinking of adding a "br-snat" bridge on each ovs, adding to it > one interface with a fixed IP, and (with some minimal development in > ovn northd) have the snat traffic of all its ports going out of that > interface instead of going through the tunnel towards the gateway. > Ideally the IP used today for the tunnel could be used too for the > snat traffic, but this seems less trivial to achieve. > > Before looking at the details of ddlog and the syntax of flows, I > would love to get some feedback on the idea, maybe there is something > fundamentally broken with my design, or maybe there is a smarter way > to achieve this? This is an interesting idea. In order to do snat, OVN should know what IP to use. But this IP should belong to the provider network subnet pool right ? If you think this can be done, you can probably attempt a quick PoC with just changes to the C version and post the patches as RFC. The ddlog part can be done later if the approach seems to be fine for the reviewers. Thanks Numan > > Thanks > Francois > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] Regarding OVN tutorial
On Mon, May 3, 2021 at 1:36 AM RUTUJA MADHURE wrote: > > Hi, > I am new to OVN and was looking at this webpage > (https://www.openvswitch.org/support/dist-docs-2.5/tutorial/OVN-Tutorial.md.html). > The links on the page are not accessible (e.g. ovn/env4/packet1.sh). > > Can you please let me know if it has been moved to some other location or if > there is another such tutorial? > Hi, OVN is split from OVS a while back and you can find the github repo and other resources here - https://github.com/ovn-org/ovn - https://www.ovn.org/en/ - https://docs.ovn.org/en/latest/ Thanks Numan > Thanks, > > Regards, > Rutuja Madhure > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
