Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic

2018-04-17 Thread Anil Venkata 
I played adding some flows for forwarding as vlan packets and it worked(i.e
compute and gateway communicated through vlan packets on both sides)
I created a vm on vlan network 172.16.1.1/24 with vlan id 20 and an
external network 10.0.0.0/24 to reach internet.

In the current code, when I tired to reach external network(8.8.8.8) from
this vm, below physical flows will be run for routing and redirecting the
packet through geneve tunnel.

table=15 ip,metadata=0x2
actions=dec_ttl(),load:0xa01->NXM_NX_XXREG0[96..127],load:0xa65->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:49:bb:60->eth_src,set_field:0x3->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,16)
//routing
table=17 reg15=0x3,metadata=0x2
actions=set_field:0x4->reg15,resubmit(,18)  //redirect
table=32, priority=100,reg15=0x4,metadata=0x2
actions=load:0x2->NXM_NX_TUN_ID[0..23],set_field:0x4->tun_metadata0,move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30],bundle(eth_src,0,active_backup,ofport,slaves:3,4)

When I manually added below flows (in same tables) with higher priority for
vlans, packet is forwarded with vlan tag instead of tunneling.

sudo ovs-ofctl add-flow br-int
"table=15,ip,priority=49,metadata=0x2,dl_dst=fa:16:3e:8c:02:5d,actions=dec_ttl(),load:0xac100201->NXM_NX_REG0[],load:0x3->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,16)"
sudo ovs-ofctl add-flow br-int
"table=16,priority=100,reg0=0xac100201,reg15=0x3,metadata=0x2,action=mod_dl_dst:fa:16:3e:8c:02:5d,resubmit(,17)"
sudo ovs-ofctl add-flow br-int
"table=32,priority=200,reg0=0xac100201,reg15=0x4,metadata=0x2,action=mod_vlan_vid:20,output:9,strip_vlan"

Note: Above table 15 flow is added by ovn-controller for each vlan
network(fa:16:3e:8c:02:5d is 172.16.1.1 MAC)

I will work on ovn-controller changes to have these flows.

Thanks
Anil


On Tue, Apr 17, 2018 at 2:20 AM, Ben Pfaff  wrote:

> On Fri, Apr 13, 2018 at 09:22:44PM -0400, Russell Bryant wrote:
> > On Fri, Apr 13, 2018 at 9:01 PM, Russell Bryant  wrote:
> > > On Fri, Apr 13, 2018 at 5:27 PM, Ben Pfaff  wrote:
> > >> On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote:
> > >>> vm created on a vlan tenant network is using geneve tunnel(between
> compute
> > >>> and gateway nodes) to reach external network. Because of this, we
> need to
> > >>> consider tunnelling overhead while assigning MTU for vlan network.
> Can we
> > >>> improve OVN to avoid tunnelling in this case.
> > >>
> > >> When OVN tunnels packets, the tunnel metadata includes information on
> > >> the logical network, logical input port, and logical output port.  The
> > >> logical input port is only used for egress ACLs, so it could be
> omitted
> > >> if egress ACLs are constrained not to match on the logical input port.
> > >> The logical network and logical output port are still needed, though,
> so
> > >> to encode that in a VLAN they would have to add up to 12 bits or less.
> > >> That's pretty constraining.  Do you have some idea for how to do it?
> > >
> > > I don't think ACLs are a factor here because it's actually the logical
> > > router pipeline forwarded the packet over a tunnel.  The only logical
> > > switches involved are VLAN networks (a switch with a localnet port).
> > >
> > > The unexpected behavior here is that despite using all VLAN networks,
> > > a Geneve tunnel is used when the packet is sent to the L3 gateway node
> > > that's doing SNAT.  Note that the type of router configured here is
> > > the hybrid-type, where routing is fully distributed in all cases
> > > except when NAT is required, then it gets redirected to a central
> > > point.  That redirect is what we're seeing here.
> > >
> > > I've thought of two ways out of this:
> > >
> > > 1) In this scenario, if you really don't want any tunneling in use,
> > > configure a fully centralized router instead.  The downside is that
> > > East-West routing will be centralized, as well.
> > >
> > > 2) Use two routers.  ls1 (VLAN) with all ports for VMs <-> distributed
> > > east-west router <-> ls2 (VLAN) used just to interconnect the routers
> > > <-> centralized router for SNAT.  The downside here is that you still
> > > lose the ability to bind floating IPs directly to compute nodes like
> > > today.
> > >
> > > 3) Figure out a way for OVN to do this redirect to the gateway host
> > > over a VLAN network.  I suspect this isn't trivial and honestly
> > > haven't spent the time to figure out what it would take, but this does
> > > seem like the ideal behavior.
> > >
> > > --
> > > Russell Bryant
> >
> > After an internal conversation on this topic, I wrote the following
> > doc to summarize what was observed and to capture ideas for next
> > steps:
> >
> > https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflH
> ACp8WSRnKCFSgg/edit?usp=sharing
>
> I wonder whether this should be added to the OVN FAQ I starting writing
> earlier today:
> https://patchwork.ozlabs.org/patch/898868/
>

Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic

2018-04-16 Thread Ben Pfaff 
On Fri, Apr 13, 2018 at 09:22:44PM -0400, Russell Bryant wrote:
> On Fri, Apr 13, 2018 at 9:01 PM, Russell Bryant  wrote:
> > On Fri, Apr 13, 2018 at 5:27 PM, Ben Pfaff  wrote:
> >> On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote:
> >>> vm created on a vlan tenant network is using geneve tunnel(between compute
> >>> and gateway nodes) to reach external network. Because of this, we need to
> >>> consider tunnelling overhead while assigning MTU for vlan network. Can we
> >>> improve OVN to avoid tunnelling in this case.
> >>
> >> When OVN tunnels packets, the tunnel metadata includes information on
> >> the logical network, logical input port, and logical output port.  The
> >> logical input port is only used for egress ACLs, so it could be omitted
> >> if egress ACLs are constrained not to match on the logical input port.
> >> The logical network and logical output port are still needed, though, so
> >> to encode that in a VLAN they would have to add up to 12 bits or less.
> >> That's pretty constraining.  Do you have some idea for how to do it?
> >
> > I don't think ACLs are a factor here because it's actually the logical
> > router pipeline forwarded the packet over a tunnel.  The only logical
> > switches involved are VLAN networks (a switch with a localnet port).
> >
> > The unexpected behavior here is that despite using all VLAN networks,
> > a Geneve tunnel is used when the packet is sent to the L3 gateway node
> > that's doing SNAT.  Note that the type of router configured here is
> > the hybrid-type, where routing is fully distributed in all cases
> > except when NAT is required, then it gets redirected to a central
> > point.  That redirect is what we're seeing here.
> >
> > I've thought of two ways out of this:
> >
> > 1) In this scenario, if you really don't want any tunneling in use,
> > configure a fully centralized router instead.  The downside is that
> > East-West routing will be centralized, as well.
> >
> > 2) Use two routers.  ls1 (VLAN) with all ports for VMs <-> distributed
> > east-west router <-> ls2 (VLAN) used just to interconnect the routers
> > <-> centralized router for SNAT.  The downside here is that you still
> > lose the ability to bind floating IPs directly to compute nodes like
> > today.
> >
> > 3) Figure out a way for OVN to do this redirect to the gateway host
> > over a VLAN network.  I suspect this isn't trivial and honestly
> > haven't spent the time to figure out what it would take, but this does
> > seem like the ideal behavior.
> >
> > --
> > Russell Bryant
> 
> After an internal conversation on this topic, I wrote the following
> doc to summarize what was observed and to capture ideas for next
> steps:
> 
> https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflHACp8WSRnKCFSgg/edit?usp=sharing

I wonder whether this should be added to the OVN FAQ I starting writing
earlier today:
https://patchwork.ozlabs.org/patch/898868/
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic

2018-04-13 Thread Han Zhou 
On Fri, Apr 13, 2018 at 6:22 PM, Russell Bryant  wrote:
>
> On Fri, Apr 13, 2018 at 9:01 PM, Russell Bryant  wrote:
> > On Fri, Apr 13, 2018 at 5:27 PM, Ben Pfaff  wrote:
> >> On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote:
> >>> vm created on a vlan tenant network is using geneve tunnel(between
compute
> >>> and gateway nodes) to reach external network. Because of this, we
need to
> >>> consider tunnelling overhead while assigning MTU for vlan network.
Can we
> >>> improve OVN to avoid tunnelling in this case.
> >>
> >> When OVN tunnels packets, the tunnel metadata includes information on
> >> the logical network, logical input port, and logical output port.  The
> >> logical input port is only used for egress ACLs, so it could be omitted
> >> if egress ACLs are constrained not to match on the logical input port.
> >> The logical network and logical output port are still needed, though,
so
> >> to encode that in a VLAN they would have to add up to 12 bits or less.
> >> That's pretty constraining.  Do you have some idea for how to do it?
> >
> > I don't think ACLs are a factor here because it's actually the logical
> > router pipeline forwarded the packet over a tunnel.  The only logical
> > switches involved are VLAN networks (a switch with a localnet port).
> >
> > The unexpected behavior here is that despite using all VLAN networks,
> > a Geneve tunnel is used when the packet is sent to the L3 gateway node
> > that's doing SNAT.  Note that the type of router configured here is
> > the hybrid-type, where routing is fully distributed in all cases
> > except when NAT is required, then it gets redirected to a central
> > point.  That redirect is what we're seeing here.
> >
> > I've thought of two ways out of this:
> >
> > 1) In this scenario, if you really don't want any tunneling in use,
> > configure a fully centralized router instead.  The downside is that
> > East-West routing will be centralized, as well.
> >
> > 2) Use two routers.  ls1 (VLAN) with all ports for VMs <-> distributed
> > east-west router <-> ls2 (VLAN) used just to interconnect the routers
> > <-> centralized router for SNAT.  The downside here is that you still
> > lose the ability to bind floating IPs directly to compute nodes like
> > today.
> >
> > 3) Figure out a way for OVN to do this redirect to the gateway host
> > over a VLAN network.  I suspect this isn't trivial and honestly
> > haven't spent the time to figure out what it would take, but this does
> > seem like the ideal behavior.

For the VLAN network to be used to redirect to a gateway, direct physical
L2 connection is required, but Gateway router in OVN generally doesn't
require L2 connection.

> >
> > --
> > Russell Bryant
>
> After an internal conversation on this topic, I wrote the following
> doc to summarize what was observed and to capture ideas for next
> steps:
>
>
https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflHACp8WSRnKCFSgg/edit?usp=sharing
>
> --
> Russell Bryant
> ___
> discuss mailing list
> disc...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic

2018-04-13 Thread Russell Bryant 
On Fri, Apr 13, 2018 at 9:01 PM, Russell Bryant  wrote:
> On Fri, Apr 13, 2018 at 5:27 PM, Ben Pfaff  wrote:
>> On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote:
>>> vm created on a vlan tenant network is using geneve tunnel(between compute
>>> and gateway nodes) to reach external network. Because of this, we need to
>>> consider tunnelling overhead while assigning MTU for vlan network. Can we
>>> improve OVN to avoid tunnelling in this case.
>>
>> When OVN tunnels packets, the tunnel metadata includes information on
>> the logical network, logical input port, and logical output port.  The
>> logical input port is only used for egress ACLs, so it could be omitted
>> if egress ACLs are constrained not to match on the logical input port.
>> The logical network and logical output port are still needed, though, so
>> to encode that in a VLAN they would have to add up to 12 bits or less.
>> That's pretty constraining.  Do you have some idea for how to do it?
>
> I don't think ACLs are a factor here because it's actually the logical
> router pipeline forwarded the packet over a tunnel.  The only logical
> switches involved are VLAN networks (a switch with a localnet port).
>
> The unexpected behavior here is that despite using all VLAN networks,
> a Geneve tunnel is used when the packet is sent to the L3 gateway node
> that's doing SNAT.  Note that the type of router configured here is
> the hybrid-type, where routing is fully distributed in all cases
> except when NAT is required, then it gets redirected to a central
> point.  That redirect is what we're seeing here.
>
> I've thought of two ways out of this:
>
> 1) In this scenario, if you really don't want any tunneling in use,
> configure a fully centralized router instead.  The downside is that
> East-West routing will be centralized, as well.
>
> 2) Use two routers.  ls1 (VLAN) with all ports for VMs <-> distributed
> east-west router <-> ls2 (VLAN) used just to interconnect the routers
> <-> centralized router for SNAT.  The downside here is that you still
> lose the ability to bind floating IPs directly to compute nodes like
> today.
>
> 3) Figure out a way for OVN to do this redirect to the gateway host
> over a VLAN network.  I suspect this isn't trivial and honestly
> haven't spent the time to figure out what it would take, but this does
> seem like the ideal behavior.
>
> --
> Russell Bryant

After an internal conversation on this topic, I wrote the following
doc to summarize what was observed and to capture ideas for next
steps:

https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflHACp8WSRnKCFSgg/edit?usp=sharing

-- 
Russell Bryant
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic

2018-04-13 Thread Russell Bryant 
On Fri, Apr 13, 2018 at 5:27 PM, Ben Pfaff  wrote:
> On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote:
>> vm created on a vlan tenant network is using geneve tunnel(between compute
>> and gateway nodes) to reach external network. Because of this, we need to
>> consider tunnelling overhead while assigning MTU for vlan network. Can we
>> improve OVN to avoid tunnelling in this case.
>
> When OVN tunnels packets, the tunnel metadata includes information on
> the logical network, logical input port, and logical output port.  The
> logical input port is only used for egress ACLs, so it could be omitted
> if egress ACLs are constrained not to match on the logical input port.
> The logical network and logical output port are still needed, though, so
> to encode that in a VLAN they would have to add up to 12 bits or less.
> That's pretty constraining.  Do you have some idea for how to do it?

I don't think ACLs are a factor here because it's actually the logical
router pipeline forwarded the packet over a tunnel.  The only logical
switches involved are VLAN networks (a switch with a localnet port).

The unexpected behavior here is that despite using all VLAN networks,
a Geneve tunnel is used when the packet is sent to the L3 gateway node
that's doing SNAT.  Note that the type of router configured here is
the hybrid-type, where routing is fully distributed in all cases
except when NAT is required, then it gets redirected to a central
point.  That redirect is what we're seeing here.

I've thought of two ways out of this:

1) In this scenario, if you really don't want any tunneling in use,
configure a fully centralized router instead.  The downside is that
East-West routing will be centralized, as well.

2) Use two routers.  ls1 (VLAN) with all ports for VMs <-> distributed
east-west router <-> ls2 (VLAN) used just to interconnect the routers
<-> centralized router for SNAT.  The downside here is that you still
lose the ability to bind floating IPs directly to compute nodes like
today.

3) Figure out a way for OVN to do this redirect to the gateway host
over a VLAN network.  I suspect this isn't trivial and honestly
haven't spent the time to figure out what it would take, but this does
seem like the ideal behavior.

-- 
Russell Bryant
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic

2018-04-13 Thread Ben Pfaff 
On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote:
> vm created on a vlan tenant network is using geneve tunnel(between compute
> and gateway nodes) to reach external network. Because of this, we need to
> consider tunnelling overhead while assigning MTU for vlan network. Can we
> improve OVN to avoid tunnelling in this case.

When OVN tunnels packets, the tunnel metadata includes information on
the logical network, logical input port, and logical output port.  The
logical input port is only used for egress ACLs, so it could be omitted
if egress ACLs are constrained not to match on the logical input port.
The logical network and logical output port are still needed, though, so
to encode that in a VLAN they would have to add up to 12 bits or less.
That's pretty constraining.  Do you have some idea for how to do it?
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss