Re: [openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Russell Bryant]

On 09/24/2015 01:17 AM, Amitabha Biswas wrote:
> Hi everyone,
> 
> I want to open up the discussion regarding how to support OVN
> VTEP gateway deployment and its lifecycle in Neutron. 

Thanks a lot for looking into this!

> In the "Life Cycle of a VTEP gateway" part in the OVN architecture
> document (http://www.russellbryant.net/ovs-docs/ovn-architecture.7.pdf),
> step 3 is where the Neutron OVN plugin is involved. At a minimum, the
> Neutron OVN plugin will enable setting the type as "vtep" and the
> vtep-logical-switch and vtep-physical-switch options in the
> OVN_Northbound database.

I have the docs published there just to make it easier to read the
rendered version.  The source of that document is:

https://github.com/openvswitch/ovs/blob/master/ovn/ovn-architecture.7.xml

> There are 2 parts to the proposal/discussion - a short term solution and
> a long term one:
> 
> A short term solution (proposed by Russell Bryant) is similar to the
> work that was done for container support in OVN - using a binding
> profile http://networking-ovn.readthedocs.org/en/latest/containers.html.
> A ovn logical network/switch can be mapped to a vtep logical gateway by
> creating a port in that logical network and creating a binding profile
> for that port in the following manner:
> 
> neutron port-create --binding-profile
> '{"vtep-logical-switch":"vtep_lswitch_key",
> "vtep-physical-switch":"vtep_pswitch_key"}' private.
> 
> Where vtep-logical-switch and vtep-physical-switch should have been
> defined in the OVN_Southbound database by the previous steps (1,2) in
> the life cycle. 

Yes, this sounds great to me.  Since there's not a clear well accepted
API to use, we should go this route to get the functionality exposed
more quickly.  We should also include in our documentation that this is
not expected to be how this is done long term.

The comparison to the containers-in-VMs support is a good one.  In that
case we used binding:profile as a quick way to expose it, but we're
aiming to support a proper API.  For that feature, we've identified the
"VLAN aware VMs" API as the way forward, which will hopefully be
available next cycle.

> For the longer term solution, there needs to be a discussion:
> 
> Should the knowledge about the physical and logical step gateway should
> be exposed to Neutron - if yes how? This would allow a Neutron NB
> API/extension to bind a “known” vtep gateway to the neutron logical
> network. This would be similar to the workflow done in the
> networking-l2gw extension
> https://review.openstack.org/#/c/144173/3/specs/kilo/l2-gateway-api.rst
> 
> 1. Allow the admin to define and manage the vtep gateway through Neutron
> REST API.
> 
> 2. Define connections between Neutron networks and gateways. This is
> conceptually similar to Step 3 of the step gateway performed by the OVN
> Plugin in the short term solution.

networking-l2gw does seem to be the closest thing to what's needed, but
it's not a small amount of work.  I think the API might need to be
extended a bit for our needs.  A bigger concern for me is actually with
some of the current implementation details.

One particular issue is that the project implements the ovsdb protocol
from scratch.  The ovs project provides a Python library for this.  Both
Neutron and networking-ovn use it, at least.  From some discussion, I've
gathered that the ovs Python library lacked one feature that was needed,
but has since been added because we wanted the same thing in networking-ovn.

The networking-l2gw route will require some pretty significant work.
It's still the closest existing effort, so I think we should explore it
until it's absolutely clear that it *can't* work for what we need.

> OR
> 
> Should OVN pursue it’s own Neutron extension (including vtep gateway
> support).

I don't think this option provides a lot of value over the short term
binding:profile solution.  Both are OVN specific.  I think I'd rather
just stick to binding:profile as the OVN specific stopgap because it's a
*lot* less work.

Thanks again,

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Salvatore Orlando]

Random comments inline.

Salvatore

On 24 September 2015 at 14:05, Russell Bryant  wrote:

> On 09/24/2015 01:17 AM, Amitabha Biswas wrote:
> > Hi everyone,
> >
> > I want to open up the discussion regarding how to support OVN
> > VTEP gateway deployment and its lifecycle in Neutron.
>
> Thanks a lot for looking into this!
>
> > In the "Life Cycle of a VTEP gateway" part in the OVN architecture
> > document (http://www.russellbryant.net/ovs-docs/ovn-architecture.7.pdf),
> > step 3 is where the Neutron OVN plugin is involved. At a minimum, the
> > Neutron OVN plugin will enable setting the type as "vtep" and the
> > vtep-logical-switch and vtep-physical-switch options in the
> > OVN_Northbound database.
>
> I have the docs published there just to make it easier to read the
> rendered version.  The source of that document is:
>
> https://github.com/openvswitch/ovs/blob/master/ovn/ovn-architecture.7.xml
>
> > There are 2 parts to the proposal/discussion - a short term solution and
> > a long term one:
> >
> > A short term solution (proposed by Russell Bryant) is similar to the
> > work that was done for container support in OVN - using a binding
> > profile http://networking-ovn.readthedocs.org/en/latest/containers.html.
> > A ovn logical network/switch can be mapped to a vtep logical gateway by
> > creating a port in that logical network and creating a binding profile
> > for that port in the following manner:
> >
> > neutron port-create --binding-profile
> > '{"vtep-logical-switch":"vtep_lswitch_key",
> > "vtep-physical-switch":"vtep_pswitch_key"}' private.
> >
> > Where vtep-logical-switch and vtep-physical-switch should have been
> > defined in the OVN_Southbound database by the previous steps (1,2) in
> > the life cycle.
>
> Yes, this sounds great to me.  Since there's not a clear well accepted
> API to use, we should go this route to get the functionality exposed
> more quickly.  We should also include in our documentation that this is
> not expected to be how this is done long term.
>
> The comparison to the containers-in-VMs support is a good one.  In that
> case we used binding:profile as a quick way to expose it, but we're
> aiming to support a proper API.  For that feature, we've identified the
> "VLAN aware VMs" API as the way forward, which will hopefully be
> available next cycle.
>
> > For the longer term solution, there needs to be a discussion:
> >
> > Should the knowledge about the physical and logical step gateway should
> > be exposed to Neutron - if yes how? This would allow a Neutron NB
> > API/extension to bind a “known” vtep gateway to the neutron logical
> > network. This would be similar to the workflow done in the
> > networking-l2gw extension
> > https://review.openstack.org/#/c/144173/3/specs/kilo/l2-gateway-api.rst
> >
> > 1. Allow the admin to define and manage the vtep gateway through Neutron
> > REST API.
> >
> > 2. Define connections between Neutron networks and gateways. This is
> > conceptually similar to Step 3 of the step gateway performed by the OVN
> > Plugin in the short term solution.
>
> networking-l2gw does seem to be the closest thing to what's needed, but
> it's not a small amount of work.  I think the API might need to be
> extended a bit for our needs.  A bigger concern for me is actually with
> some of the current implementation details.
>

It is indeed. While I like very much the solution based on binding profiles
it does not work very well from a UX perspective in environments where
operators control the whole cloud with openstack tools.


>
> One particular issue is that the project implements the ovsdb protocol
> from scratch.  The ovs project provides a Python library for this.  Both
> Neutron and networking-ovn use it, at least.  From some discussion, I've
> gathered that the ovs Python library lacked one feature that was needed,
> but has since been added because we wanted the same thing in
> networking-ovn.
>

My take here is that we don't need to use the whole implementation of
networking-l2gw, but only the APIs and the DB management layer it exposes.
Networking-l2gw provides a VTEP network gateway solution that, if you want,
will eventually be part of Neutron's "reference" control plane.
OVN provides its implementation; I think it should be possible to leverage
networking-l2gw either by pushing an OVN driver there, or implementing the
same driver in openstack/networking-ovn.


>
> The networking-l2gw route will require some pretty significant work.
> It's still the closest existing effort, so I think we should explore it
> until it's absolutely clear that it *can't* work for what we need.
>

I would say that it is definitely not trivial but probably a bit less than
"significant". abhraut from my team has done something quite similar for
openstack/vmware-nsx [1]


> > OR
> >
> > Should OVN pursue it’s own Neutron extension (including vtep gateway
> > support).
>
> I don't think this option provides a lot of value over the short 

Re: [openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Russell Bryant]

On 09/24/2015 10:18 AM, Salvatore Orlando wrote:
> One particular issue is that the project implements the ovsdb protocol
> from scratch.  The ovs project provides a Python library for this.  Both
> Neutron and networking-ovn use it, at least.  From some discussion, I've
> gathered that the ovs Python library lacked one feature that was needed,
> but has since been added because we wanted the same thing in
> networking-ovn.
> 
> 
> My take here is that we don't need to use the whole implementation of
> networking-l2gw, but only the APIs and the DB management layer it exposes.
> Networking-l2gw provides a VTEP network gateway solution that, if you
> want, will eventually be part of Neutron's "reference" control plane.
> OVN provides its implementation; I think it should be possible to
> leverage networking-l2gw either by pushing an OVN driver there, or
> implementing the same driver in openstack/networking-ovn.

>From a quick look, it seemed like networking-l2gw was doing 2 things.

  1) Management of vtep switches themselves

  2) Management of connectivity between Neutron networks and VTEP
 gateways

I figured the implementation of #1 would be the same whether you were
using ML2+OVS, OVN, (or whatever else).  This part is not addressed in
OVN.  You point OVN at VTEP gateways, but it's expected you manage the
gateway provisioning some other way.

It's #2 that has a very different implementation.  For OVN, it's just
creating a row in OVN's northbound database.

or did I mis-interpret what networking-l2gw is doing?

> The networking-l2gw route will require some pretty significant work.
> It's still the closest existing effort, so I think we should explore it
> until it's absolutely clear that it *can't* work for what we need.
> 
> 
> I would say that it is definitely not trivial but probably a bit less
> than "significant". abhraut from my team has done something quite
> similar for openstack/vmware-nsx [1]

but specific to nsx.  :(

Does it look like networking-l2gw could be a common API for what's
needed for NSX?

> 
> 
> > OR
> >
> > Should OVN pursue it’s own Neutron extension (including vtep gateway
> > support).
> 
> I don't think this option provides a lot of value over the short term
> binding:profile solution.  Both are OVN specific.  I think I'd rather
> just stick to binding:profile as the OVN specific stopgap because it's a
> *lot* less work.
> 
> 
> I totally agree. The solution based on the binding profile is indeed a
> decent one in my opinion.
> If OVN cannot converge on the extension proposed by networking-l2gw then
> I'd keep using the binding profile for specifying gateway ports.

Great, thanks for the feedback!

> [1] https://review.openstack.org/#/c/210623/

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Armando M.]

On 24 September 2015 at 09:12, Russell Bryant  wrote:

> On 09/24/2015 10:18 AM, Salvatore Orlando wrote:
> > One particular issue is that the project implements the ovsdb
> protocol
> > from scratch.  The ovs project provides a Python library for this.
> Both
> > Neutron and networking-ovn use it, at least.  From some discussion,
> I've
> > gathered that the ovs Python library lacked one feature that was
> needed,
> > but has since been added because we wanted the same thing in
> > networking-ovn.
> >
> >
> > My take here is that we don't need to use the whole implementation of
> > networking-l2gw, but only the APIs and the DB management layer it
> exposes.
> > Networking-l2gw provides a VTEP network gateway solution that, if you
> > want, will eventually be part of Neutron's "reference" control plane.
> > OVN provides its implementation; I think it should be possible to
> > leverage networking-l2gw either by pushing an OVN driver there, or
> > implementing the same driver in openstack/networking-ovn.
>
> From a quick look, it seemed like networking-l2gw was doing 2 things.
>
>   1) Management of vtep switches themselves
>
>   2) Management of connectivity between Neutron networks and VTEP
>  gateways
>
> I figured the implementation of #1 would be the same whether you were
> using ML2+OVS, OVN, (or whatever else).  This part is not addressed in
> OVN.  You point OVN at VTEP gateways, but it's expected you manage the
> gateway provisioning some other way.
>
> It's #2 that has a very different implementation.  For OVN, it's just
> creating a row in OVN's northbound database.
>
> or did I mis-interpret what networking-l2gw is doing?
>

No, you did not misinterpret what the objective of the project were (which
I reinstate here):

* Provide an API to OpenStack admins to extend neutron logical networks
into unmanaged pre-existing vlans. Bear in mind that things like address
collision prevention is left in the hands on the operator. Other aspects
like L2/L3 interoperability instead should be taken care of, at least from
an implementation point of view.

* Provide a pluggable framework for multiple drivers of the API.

* Provide an PoC implementation on top of the ovsdb vtep schema. This can
be implemented both in hardware (ToR switches) and software (software L2
gateways).



>
> > The networking-l2gw route will require some pretty significant work.
> > It's still the closest existing effort, so I think we should explore
> it
> > until it's absolutely clear that it *can't* work for what we need.
>

We may have fallen short of some/all expectations, but I would like to
believe than it is nothing that can't be fixed by iterating on, especially
if active project participation raises.

I don't think there's a procedural mandate to make OVN abide by the l2gw
proposed API. As you said, it is not a clear well accepted API, but that's
only because we live in a brand new world, where people should be allowed
to experiment and reconcile later as community forces play out.

That said, should the conclusion that "it (the API) *can't* work for what
OVN needs" be reached, I would like to understand/document why for the sake
of all us involved so that lessons will yield from our mistakes.

>
> >
> > I would say that it is definitely not trivial but probably a bit less
> > than "significant". abhraut from my team has done something quite
> > similar for openstack/vmware-nsx [1]
>
> but specific to nsx.  :(
>
> Does it look like networking-l2gw could be a common API for what's
> needed for NSX?
>
> >
> >
> > > OR
> > >
> > > Should OVN pursue it’s own Neutron extension (including vtep
> gateway
> > > support).
> >
> > I don't think this option provides a lot of value over the short term
> > binding:profile solution.  Both are OVN specific.  I think I'd rather
> > just stick to binding:profile as the OVN specific stopgap because
> it's a
> > *lot* less work.
> >
> >
> > I totally agree. The solution based on the binding profile is indeed a
> > decent one in my opinion.
> > If OVN cannot converge on the extension proposed by networking-l2gw then
> > I'd keep using the binding profile for specifying gateway ports.
>
> Great, thanks for the feedback!
>
> > [1] https://review.openstack.org/#/c/210623/
>
> --
> Russell Bryant
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Russell Bryant]

On 09/24/2015 01:25 PM, Armando M. wrote:
> 
> 
> 
> On 24 September 2015 at 09:12, Russell Bryant  > wrote:
> 
> On 09/24/2015 10:18 AM, Salvatore Orlando wrote:
> > One particular issue is that the project implements the ovsdb 
> protocol
> > from scratch.  The ovs project provides a Python library for this.  
> Both
> > Neutron and networking-ovn use it, at least.  From some discussion, 
> I've
> > gathered that the ovs Python library lacked one feature that was 
> needed,
> > but has since been added because we wanted the same thing in
> > networking-ovn.
> >
> >
> > My take here is that we don't need to use the whole implementation of
> > networking-l2gw, but only the APIs and the DB management layer it 
> exposes.
> > Networking-l2gw provides a VTEP network gateway solution that, if you
> > want, will eventually be part of Neutron's "reference" control plane.
> > OVN provides its implementation; I think it should be possible to
> > leverage networking-l2gw either by pushing an OVN driver there, or
> > implementing the same driver in openstack/networking-ovn.
> 
> From a quick look, it seemed like networking-l2gw was doing 2 things.
> 
>   1) Management of vtep switches themselves
> 
>   2) Management of connectivity between Neutron networks and VTEP
>  gateways
> 
> I figured the implementation of #1 would be the same whether you were
> using ML2+OVS, OVN, (or whatever else).  This part is not addressed in
> OVN.  You point OVN at VTEP gateways, but it's expected you manage the
> gateway provisioning some other way.
> 
> It's #2 that has a very different implementation.  For OVN, it's just
> creating a row in OVN's northbound database.
> 
> or did I mis-interpret what networking-l2gw is doing?
> 
> 
> No, you did not misinterpret what the objective of the project were
> (which I reinstate here):
> 
> * Provide an API to OpenStack admins to extend neutron logical networks
> into unmanaged pre-existing vlans. Bear in mind that things like address
> collision prevention is left in the hands on the operator. Other aspects
> like L2/L3 interoperability instead should be taken care of, at least
> from an implementation point of view.
> 
> * Provide a pluggable framework for multiple drivers of the API.
> 
> * Provide an PoC implementation on top of the ovsdb vtep schema. This
> can be implemented both in hardware (ToR switches) and software
> (software L2 gateways). 

Thanks for clarifying the project's goals!

> > The networking-l2gw route will require some pretty significant work.
> > It's still the closest existing effort, so I think we should 
> explore it
> > until it's absolutely clear that it *can't* work for what we need.
> 
> 
> We may have fallen short of some/all expectations, but I would like to
> believe than it is nothing that can't be fixed by iterating on,
> especially if active project participation raises.
> 
> I don't think there's a procedural mandate to make OVN abide by the l2gw
> proposed API. As you said, it is not a clear well accepted API, but
> that's only because we live in a brand new world, where people should be
> allowed to experiment and reconcile later as community forces play out.
> 
> That said, should the conclusion that "it (the API) *can't* work for
> what OVN needs" be reached, I would like to understand/document why for
> the sake of all us involved so that lessons will yield from our mistakes.

My gut says we should be able to work together and make it work.  I
expect we'll talk in more detail in the next cycle.  :-)

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][networking-ovn][vtep] Proposal: support for vtep-gateway in ovn

[Amitabha Biswas]

Hi everyone,
I want to open up the discussion regarding how to support OVN VTEP gateway deployment and its lifecycle in Neutron. 
In the "Life Cycle of a VTEP gateway" part in the OVN architecture document (http://www.russellbryant.net/ovs-docs/ovn-architecture.7.pdf), step 3 is where the Neutron OVN plugin is involved. At a minimum, the Neutron OVN plugin will enable setting the type as "vtep" and the vtep-logical-switch and vtep-physical-switch options in the OVN_Northbound database.
 
There are 2 parts to the proposal/discussion - a short term solution and a long term one:
A short term solution (proposed by Russell Bryant) is similar to the work that was done for container support in OVN - using a binding profile http://networking-ovn.readthedocs.org/en/latest/containers.html. A ovn logical network/switch can be mapped to a vtep logical gateway by creating a port in that logical network and creating a binding profile for that port in the following manner:
neutron port-create --binding-profile '{"vtep-logical-switch":"vtep_lswitch_key", "vtep-physical-switch":"vtep_pswitch_key"}' private.
Where vtep-logical-switch and vtep-physical-switch should have been defined in the OVN_Southbound database by the previous steps (1,2) in the life cycle. 
 
For the longer term solution, there needs to be a discussion:
Should the knowledge about the physical and logical step gateway should be exposed to Neutron - if yes how? This would allow a Neutron NB API/extension to bind a “known” vtep gateway to the neutron logical network. This would be similar to the workflow done in the networking-l2gw extension https://review.openstack.org/#/c/144173/3/specs/kilo/l2-gateway-api.rst
1. Allow the admin to define and manage the vtep gateway through Neutron REST API.
2. Define connections between Neutron networks and gateways. This is conceptually similar to Step 3 of the step gateway performed by the OVN Plugin in the short term solution.
OR
Should OVN pursue it’s own Neutron extension (including vtep gateway support).
 
Thanks Amitabha



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev