Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-18 Thread Germy Lure
Hi Trinath,

I think the vendor company has many experts to review their codes. They can
do it well.

But I still have some comments inline.

Germy

On Thu, Sep 18, 2014 at 1:42 PM, trinath.soman...@freescale.com 
trinath.soman...@freescale.com wrote:

  Though Code reviews for vendor code takes more time, I feel it must go
 through Core reviews.



 Since, Vendors might submit the code that is working fine within their
 third party CI environment but the Code review make it more efficient with
 respect to the coding standards followed in the community.



 Also, for all the vendor plugins/drivers the code reviews (+1s and +2s)
 give a feedback on the quality they must be in to be with Neutron.

I think the quality of a software mainly lies on developers, otherwise
reviewers will be very very busy.
We suppose that all core members reviewed your plugin and gave feedback
many +, so can you guarantee the plugin high quality? even no BUGs?
I think only the vendor, cooperating with customer and providing plugin and
driver, can and must guarantee the quality. But those *private* releases
only exist in vendor's disk and running in customer's machine. It cannot be
updated to community because of approving waiting, because of not efficient
enough, because of the coding standards, 



 But one suggestion I want to put forward, when an -1 or -2 is given to the
 code, Reviewers might give a brief comment on why this was given, what
 might be preferred solution and Is there any reference implementation that
 can be considered for the code in review to move away from these errors.
 This can help the developers.

If core members prefer Cisco's implementation, all the other vendors follow
it? Why different plugins? Only one is enough.
Of course, this is a very extreme assumption. We just discuss a problem.









 --

 Trinath Somanchi - B39208

 trinath.soman...@freescale.com | extn: 4048
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-17 Thread Germy Lure
Hi Salvatore,
Thanks for your hyperlink. It's really a monster thread that contains
everyone's opinion. But it's useful to me.
So, Before we focus on the Neutron core itself, we should firstly release a
suite standardized APIs and a framework for vendors' codes.
About this job, I think most of it is already OK. We have 20+ monolithic
plugins following NB API and plugin framework.
We need publish an API doc for internal interface(I prefer to call it SB
API, stand on the Neutron core's point to consider, vendors' codes
do not belong to core.) and other things unsuitable now.

In my opinion, the Neutron core's main responsibility is data model and DB,
schedule and dispatch, API and validation, framework and workflow.

Some more comments inline.


This is a very important discussion - very closely related to the one going
on in this other thread
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045768.html
.
Unfortunately it is also a discussion that tends to easily fragment and
move in a thousand different directions.
A few months ago I was too of the opinion that vendor plugins and drivers
were the main reason of unnecessary load for the core team. I still think
that they're an unnecessary heavy load, but I reckon the problem does not
really lies with open source versus vendor code. It lies in matching
people's competencies with subsystems and proper interface across them - as
already pointed out in this thread.
Yes, it's really important.

I have some more comments inline, but unless growing another monster thread
I'd rather start a different, cross-project discussion (which will
hopefully not become just a cross-project monster thread!)

Salvatore

On 15 September 2014 08:29, Germy Lure germy.l...@gmail.com wrote:

 Obviously, to a vendor's plugin/driver, the most important thing is
 API.Yes?
 NB API for a monolithic plugin or a service plugin and SB API for a
 service driver or agent, even MD. That's the basic.
 Now we have released a set of NB APIs with relative stability. The SB
 APIs' standardization are needed.


The internal interface between the API and the plugins is standardized at
the moment through use of classes like [1]. A similar interface exists for
ML2 drivers [2].
To the monolithic plugins, [1] is useful. Vendors can implement those APIs
and keep their codes locally.

At the moment the dispatch of an API call to the plugin or from a plugin to
a ML2 driver is purely a local call so these interfaces are working fairly
well at the moment. I don't know yet however whether they will be
sufficient in case plugins are split into different repos. ML2 Driver
maintainers have however been warned in the past that the driver interface
is to be considered internal and can be changed at any time. This does not
apply to the plugin interface which has been conceived in this way to
facilitate the development of out of tree plugins.
Indeed, it's difficult to split MDs from ML2 plugin framework. I think it
need some adaption.

On the other hand, if by SB interfaces you are referring to the RPC
interfaces for communicating between the servers and the various plugin, I
would say that they should be considered internal at the moment.

[1]
https://github.com/openstack/neutron/blob/master/neutron/neutron_plugin_base_v2.py#L28
[2]
https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driver_api.py


 Some comments inline.



 On Fri, Sep 12, 2014 at 5:18 PM, Kevin Benton blak...@gmail.com wrote:

  So my suggestion is remove all vendors' plugins and drivers except
 opensource as built-in.

 Yes, I think this is currently the view held by the PTL (Kyle) and some
 of the other cores so what you're suggesting will definitely come up at the
 summit.

 Good!


The discussion however will not be that different from the one we're seeing
on that huge thread on splitting out drivers, which has become in my
opinion a frankenthread.
Nevertheless, that thread points out that this is far from being merely a
neutron topic (despite neutron being the project with the highest number of
drivers and plugins).




  Why do we need a different repo to store vendors' codes? That's not the
 community business.
  I think only a proper architecture and normal NBSB API can bring a
 clear separation between plugins(or drivers) and core code, not a
 different repo.

 The problem is that that architecture won't stay stable if there is no
 shared community plugin depending on its stability. Let me ask you the
 inverse question. Why do you think the reference driver should stay in the
 core repo?

 A separate repo won't have an impact on what is packaged and released so
 it should have no impact on user experience, complete versions,
 providing code examples,  or developing new features. In fact, it will
 likely help with the last two because it will provide a clear delineation
 between what a plugin is responsible for vs. what the core API is
 responsible for. And, because new cores can be added faster to the 

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-17 Thread Amit Das
+1

 I don't think it will be seen as punitive. Vendors can write their plugins
 or drivers when a deal occurs and they do not need to submit code to
 community and wait for approving.


Being a third party vendor, i do not think this is punitive. OpenStack has
already established through processes like cert tests via tempest, external
CI, etc. However, waiting for months for reviews and some more days for
next round of reviews is really disturbing.
Vendor plugins can always always provide repository links, cert tests,
external CI logs, pep8 logs, flake8 logs, code coverage stats etc. to
confirm if they are abiding by the OpenStack processes.


Regards,
Amit
*CloudByte Inc.* http://www.cloudbyte.com/

On Thu, Sep 18, 2014 at 9:56 AM, Germy Lure germy.l...@gmail.com wrote:

 Hi Salvatore,
 Thanks for your hyperlink. It's really a monster thread that contains
 everyone's opinion. But it's useful to me.
 So, Before we focus on the Neutron core itself, we should firstly release
 a suite standardized APIs and a framework for vendors' codes.
 About this job, I think most of it is already OK. We have 20+ monolithic
 plugins following NB API and plugin framework.
 We need publish an API doc for internal interface(I prefer to call it SB
 API, stand on the Neutron core's point to consider, vendors' codes
 do not belong to core.) and other things unsuitable now.

 In my opinion, the Neutron core's main responsibility is data model and
 DB, schedule and dispatch, API and validation, framework and workflow.

 Some more comments inline.


 This is a very important discussion - very closely related to the one
 going on in this other thread
 http://lists.openstack.org/pipermail/openstack-dev/2014-September/045768.html
 .
 Unfortunately it is also a discussion that tends to easily fragment and
 move in a thousand different directions.
 A few months ago I was too of the opinion that vendor plugins and drivers
 were the main reason of unnecessary load for the core team. I still think
 that they're an unnecessary heavy load, but I reckon the problem does not
 really lies with open source versus vendor code. It lies in matching
 people's competencies with subsystems and proper interface across them - as
 already pointed out in this thread.
 Yes, it's really important.

 I have some more comments inline, but unless growing another monster
 thread I'd rather start a different, cross-project discussion (which will
 hopefully not become just a cross-project monster thread!)

 Salvatore

 On 15 September 2014 08:29, Germy Lure germy.l...@gmail.com wrote:

 Obviously, to a vendor's plugin/driver, the most important thing is
 API.Yes?
 NB API for a monolithic plugin or a service plugin and SB API for a
 service driver or agent, even MD. That's the basic.
 Now we have released a set of NB APIs with relative stability. The SB
 APIs' standardization are needed.


 The internal interface between the API and the plugins is standardized at
 the moment through use of classes like [1]. A similar interface exists for
 ML2 drivers [2].
 To the monolithic plugins, [1] is useful. Vendors can implement those APIs
 and keep their codes locally.

 At the moment the dispatch of an API call to the plugin or from a plugin
 to a ML2 driver is purely a local call so these interfaces are working
 fairly well at the moment. I don't know yet however whether they will be
 sufficient in case plugins are split into different repos. ML2 Driver
 maintainers have however been warned in the past that the driver interface
 is to be considered internal and can be changed at any time. This does not
 apply to the plugin interface which has been conceived in this way to
 facilitate the development of out of tree plugins.
 Indeed, it's difficult to split MDs from ML2 plugin framework. I think it
 need some adaption.

 On the other hand, if by SB interfaces you are referring to the RPC
 interfaces for communicating between the servers and the various plugin, I
 would say that they should be considered internal at the moment.

 [1]
 https://github.com/openstack/neutron/blob/master/neutron/neutron_plugin_base_v2.py#L28
 [2]
 https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driver_api.py


 Some comments inline.



 On Fri, Sep 12, 2014 at 5:18 PM, Kevin Benton blak...@gmail.com wrote:

  So my suggestion is remove all vendors' plugins and drivers except
 opensource as built-in.

 Yes, I think this is currently the view held by the PTL (Kyle) and some
 of the other cores so what you're suggesting will definitely come up at the
 summit.

 Good!


 The discussion however will not be that different from the one we're
 seeing on that huge thread on splitting out drivers, which has become in my
 opinion a frankenthread.
 Nevertheless, that thread points out that this is far from being merely a
 neutron topic (despite neutron being the project with the highest number of
 drivers and plugins).




  Why do we need a different repo to store vendors' codes? 

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-17 Thread trinath.soman...@freescale.com
Though Code reviews for vendor code takes more time, I feel it must go through 
Core reviews.

Since, Vendors might submit the code that is working fine within their third 
party CI environment but the Code review make it more efficient with respect to 
the coding standards followed in the community.

Also, for all the vendor plugins/drivers the code reviews (+1s and +2s) give a 
feedback on the quality they must be in to be with Neutron.

But one suggestion I want to put forward, when an -1 or -2 is given to the 
code, Reviewers might give a brief comment on why this was given, what might be 
preferred solution and Is there any reference implementation that can be 
considered for the code in review to move away from these errors. This can help 
the developers.




--
Trinath Somanchi - B39208
trinath.soman...@freescale.com | extn: 4048

From: Amit Das [mailto:amit@cloudbyte.com]
Sent: Thursday, September 18, 2014 11:01 AM
To: OpenStack Development Mailing List (not for usage questions)
Cc: tanny...@huawei.com
Subject: Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third 
vendors' plugin and driver

+1
I don't think it will be seen as punitive. Vendors can write their plugins or 
drivers when a deal occurs and they do not need to submit code to community and 
wait for approving.

Being a third party vendor, i do not think this is punitive. OpenStack has 
already established through processes like cert tests via tempest, external CI, 
etc. However, waiting for months for reviews and some more days for next round 
of reviews is really disturbing.
Vendor plugins can always always provide repository links, cert tests, external 
CI logs, pep8 logs, flake8 logs, code coverage stats etc. to confirm if they 
are abiding by the OpenStack processes.


Regards,
Amit
CloudByte Inc.http://www.cloudbyte.com/

On Thu, Sep 18, 2014 at 9:56 AM, Germy Lure 
germy.l...@gmail.commailto:germy.l...@gmail.com wrote:
Hi Salvatore,
Thanks for your hyperlink. It's really a monster thread that contains 
everyone's opinion. But it's useful to me.
So, Before we focus on the Neutron core itself, we should firstly release a 
suite standardized APIs and a framework for vendors' codes.
About this job, I think most of it is already OK. We have 20+ monolithic 
plugins following NB API and plugin framework.
We need publish an API doc for internal interface(I prefer to call it SB API, 
stand on the Neutron core's point to consider, vendors' codes
do not belong to core.) and other things unsuitable now.

In my opinion, the Neutron core's main responsibility is data model and DB, 
schedule and dispatch, API and validation, framework and workflow.

Some more comments inline.


This is a very important discussion - very closely related to the one going on 
in this other thread 
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045768.html.
Unfortunately it is also a discussion that tends to easily fragment and move in 
a thousand different directions.
A few months ago I was too of the opinion that vendor plugins and drivers were 
the main reason of unnecessary load for the core team. I still think that 
they're an unnecessary heavy load, but I reckon the problem does not really 
lies with open source versus vendor code. It lies in matching people's 
competencies with subsystems and proper interface across them - as already 
pointed out in this thread.
Yes, it's really important.

I have some more comments inline, but unless growing another monster thread I'd 
rather start a different, cross-project discussion (which will hopefully not 
become just a cross-project monster thread!)

Salvatore

On 15 September 2014 08:29, Germy Lure 
germy.l...@gmail.commailto:germy.l...@gmail.com wrote:
Obviously, to a vendor's plugin/driver, the most important thing is API.Yes?
NB API for a monolithic plugin or a service plugin and SB API for a service 
driver or agent, even MD. That's the basic.
Now we have released a set of NB APIs with relative stability. The SB APIs' 
standardization are needed.

The internal interface between the API and the plugins is standardized at the 
moment through use of classes like [1]. A similar interface exists for ML2 
drivers [2].
To the monolithic plugins, [1] is useful. Vendors can implement those APIs and 
keep their codes locally.

At the moment the dispatch of an API call to the plugin or from a plugin to a 
ML2 driver is purely a local call so these interfaces are working fairly well 
at the moment. I don't know yet however whether they will be sufficient in case 
plugins are split into different repos. ML2 Driver maintainers have however 
been warned in the past that the driver interface is to be considered internal 
and can be changed at any time. This does not apply to the plugin interface 
which has been conceived in this way to facilitate the development of out of 
tree plugins.
Indeed, it's difficult to split MDs from ML2 plugin framework. I think it need 
some adaption

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-15 Thread Germy Lure
Obviously, to a vendor's plugin/driver, the most important thing is API.Yes?
NB API for a monolithic plugin or a service plugin and SB API for a service
driver or agent, even MD. That's the basic.
Now we have released a set of NB APIs with relative stability. The SB APIs'
standardization are needed.

Some comments inline.



On Fri, Sep 12, 2014 at 5:18 PM, Kevin Benton blak...@gmail.com wrote:

  So my suggestion is remove all vendors' plugins and drivers except
 opensource as built-in.

 Yes, I think this is currently the view held by the PTL (Kyle) and some of
 the other cores so what you're suggesting will definitely come up at the
 summit.

Good!



  Why do we need a different repo to store vendors' codes? That's not the
 community business.
  I think only a proper architecture and normal NBSB API can bring a
 clear separation between plugins(or drivers) and core code, not a
 different repo.

 The problem is that that architecture won't stay stable if there is no
 shared community plugin depending on its stability. Let me ask you the
 inverse question. Why do you think the reference driver should stay in the
 core repo?

 A separate repo won't have an impact on what is packaged and released so
 it should have no impact on user experience, complete versions,
 providing code examples,  or developing new features. In fact, it will
 likely help with the last two because it will provide a clear delineation
 between what a plugin is responsible for vs. what the core API is
 responsible for. And, because new cores can be added faster to the open
 source plugins repo due to a smaller code base to learn, it will help with
 developing new features by reducing reviewer load.

OK, the key point is that vendors' code should be kept by themselves NOT by
the community. But in the same time, the community should provide
some open source reference as standard examples for those new cores and
vendors.
U are right, A separate repo won't have an impact on what is packaged and
released. The open source can stays in the core repo or a different one.
In any case, we need them there for referencing and version releasing.
Any vendor would not maintain the open source codes, the community only.



 On Fri, Sep 12, 2014 at 1:50 AM, Germy Lure germy.l...@gmail.com wrote:



 On Fri, Sep 12, 2014 at 11:11 AM, Kevin Benton blak...@gmail.com wrote:


  Maybe I missed something, but what's the solution?

 There isn't one yet. That's why it's going to be discussed at the summit.

 So my suggestion is remove all vendors' plugins and drivers except
 opensource as built-in.
 By leaving open source plugins and drivers in the tree , we can resolve
 such problems:
   1)release a workable and COMPLETE version
   2)user experience(especially for beginners)
   3)provide code example to learn for new contributors and vendors
   4)develop and verify new features



  I think we should release a workable version.

 Definitely. But that doesn't have anything to do with it living in the
 same repository. By putting it in a different repo, it provides smaller
 code bases to learn for new contributors wanting to become a core developer
 in addition to a clear separation between plugins and core code.

 Why do we need a different repo to store vendors' codes? That's not the
 community business.
 I think only a proper architecture and normal NBSB API can bring a
 clear separation between plugins(or drivers) and core code, not a
 different repo.
 Of course, if the community provides a wiki page for vendors to add
 hyperlink of their codes, I think it's perfect.


  Besides of user experience, the open source drivers are also used for
 developing and verifying new features, even small-scale case.

 Sure, but this also isn't affected by the code being in a separate repo.

 See comments above.


  The community should and just need focus on the Neutron core and
 provide framework for vendors' devices.

 I agree, but without the open source drivers being separated as well,
 it's very difficult for the framework for external drivers to be stable
 enough to be useful.

 Architecture and API. The community should ensure core and API stable
 enough and high quality. Vendors for external drivers.
 Who provides, who maintains(including development, storage, distribution,
 quality, etc).


 On Thu, Sep 11, 2014 at 7:24 PM, Germy Lure germy.l...@gmail.com
 wrote:

 Some comments inline.

 BR,
 Germy

 On Thu, Sep 11, 2014 at 5:47 PM, Kevin Benton blak...@gmail.com
 wrote:

 This has been brought up several times already and I believe is going
 to be discussed at the Kilo summit.

 Maybe I missed something, but what's the solution?


 I agree that reviewing third party patches eats community time.
 However, claiming that the community pays 46% of it's energy to maintain
 vendor-specific code doesn't make any sense. LOC in the repo has very
 little to do with ongoing required maintenance. Assuming the APIs for the
 plugins stay consistent, there should be few 

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-15 Thread Salvatore Orlando
This is a very important discussion - very closely related to the one going
on in this other thread
http://lists.openstack.org/pipermail/openstack-dev/2014-September/045768.html
.
Unfortunately it is also a discussion that tends to easily fragment and
move in a thousand different directions.
A few months ago I was too of the opinion that vendor plugins and drivers
were the main reason of unnecessary load for the core team. I still think
that they're an unnecessary heavy load, but I reckon the problem does not
really lies with open source versus vendor code. It lies in matching
people's competencies with subsystems and proper interface across them - as
already pointed out in this thread.

I have some more comments inline, but unless growing another monster thread
I'd rather start a different, cross-project discussion (which will
hopefully not become just a cross-project monster thread!)

Salvatore

On 15 September 2014 08:29, Germy Lure germy.l...@gmail.com wrote:

 Obviously, to a vendor's plugin/driver, the most important thing is
 API.Yes?
 NB API for a monolithic plugin or a service plugin and SB API for a
 service driver or agent, even MD. That's the basic.
 Now we have released a set of NB APIs with relative stability. The SB
 APIs' standardization are needed.


The internal interface between the API and the plugins is standardized at
the moment through use of classes like [1]. A similar interface exists for
ML2 drivers [2].
At the moment the dispatch of an API call to the plugin or from a plugin to
a ML2 driver is purely a local call so these interfaces are working fairly
well at the moment. I don't know yet however whether they will be
sufficient in case plugins are split into different repos. ML2 Driver
maintainers have however been warned in the past that the driver interface
is to be considered internal and can be changed at any time. This does not
apply to the plugin interface which has been conceived in this way to
facilitate the development of out of tree plugins.

On the other hand, if by SB interfaces you are referring to the RPC
interfaces for communicating between the servers and the various plugin, I
would say that they should be considered internal at the moment.

[1]
https://github.com/openstack/neutron/blob/master/neutron/neutron_plugin_base_v2.py#L28
[2]
https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/driver_api.py


 Some comments inline.



 On Fri, Sep 12, 2014 at 5:18 PM, Kevin Benton blak...@gmail.com wrote:

  So my suggestion is remove all vendors' plugins and drivers except
 opensource as built-in.

 Yes, I think this is currently the view held by the PTL (Kyle) and some
 of the other cores so what you're suggesting will definitely come up at the
 summit.

 Good!


The discussion however will not be that different from the one we're seeing
on that huge thread on splitting out drivers, which has become in my
opinion a frankenthread.
Nevertheless, that thread points out that this is far from being merely a
neutron topic (despite neutron being the project with the highest number of
drivers and plugins).




  Why do we need a different repo to store vendors' codes? That's not the
 community business.
  I think only a proper architecture and normal NBSB API can bring a
 clear separation between plugins(or drivers) and core code, not a
 different repo.

 The problem is that that architecture won't stay stable if there is no
 shared community plugin depending on its stability. Let me ask you the
 inverse question. Why do you think the reference driver should stay in the
 core repo?

 A separate repo won't have an impact on what is packaged and released so
 it should have no impact on user experience, complete versions,
 providing code examples,  or developing new features. In fact, it will
 likely help with the last two because it will provide a clear delineation
 between what a plugin is responsible for vs. what the core API is
 responsible for. And, because new cores can be added faster to the open
 source plugins repo due to a smaller code base to learn, it will help with
 developing new features by reducing reviewer load.

 OK, the key point is that vendors' code should be kept by themselves NOT
 by the community. But in the same time, the community should provide
 some open source reference as standard examples for those new cores and
 vendors.
 U are right, A separate repo won't have an impact on what is packaged and
 released. The open source can stays in the core repo or a different one.
 In any case, we need them there for referencing and version releasing.
 Any vendor would not maintain the open source codes, the community only.


I think that we are probably focusing too much on the separate repo
issue, which is probably being seen as punitive for drivers and plugins.
The separate repo would be just a possible tool for achieving the goal of
reducing the review load imposed by drivers on the core team while keeping
them part of the integrated release.

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-14 Thread Irena Berezovsky
Hi,
While keeping focused on defining proper approach to deal with Neutron third 
vendors’ plugin and driver, we also need to provide solution for complimentary 
critical piece of code maintained in the Nova code base.
Introducing new vif_type by neutron L2 Plugin/Driver, requires adding vif 
plugging support at Nova side.
I think it is very important to enable  virt driver  extensibility to support 
out of the tree/future vif_types.
If the direction is to keep vendor plugins/drivers external to Neutron core, it 
seems reasonable to impose same policy on the Nova side.

BR,
Irena

From: Kevin Benton [mailto:blak...@gmail.com]
Sent: Friday, September 12, 2014 12:19 PM
To: OpenStack Development Mailing List (not for usage questions)
Cc: tanny...@huawei.com
Subject: Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third 
vendors' plugin and driver

 So my suggestion is remove all vendors' plugins and drivers except opensource 
 as built-in.

Yes, I think this is currently the view held by the PTL (Kyle) and some of the 
other cores so what you're suggesting will definitely come up at the summit.


 Why do we need a different repo to store vendors' codes? That's not the 
 community business.
 I think only a proper architecture and normal NBSB API can bring a clear 
 separation between plugins(or drivers) and core code, not a different repo.

The problem is that that architecture won't stay stable if there is no shared 
community plugin depending on its stability. Let me ask you the inverse 
question. Why do you think the reference driver should stay in the core repo?

A separate repo won't have an impact on what is packaged and released so it 
should have no impact on user experience, complete versions, providing 
code examples,  or developing new features. In fact, it will likely help 
with the last two because it will provide a clear delineation between what a 
plugin is responsible for vs. what the core API is responsible for. And, 
because new cores can be added faster to the open source plugins repo due to a 
smaller code base to learn, it will help with developing new features by 
reducing reviewer load.

On Fri, Sep 12, 2014 at 1:50 AM, Germy Lure 
germy.l...@gmail.commailto:germy.l...@gmail.com wrote:


On Fri, Sep 12, 2014 at 11:11 AM, Kevin Benton 
blak...@gmail.commailto:blak...@gmail.com wrote:

 Maybe I missed something, but what's the solution?

There isn't one yet. That's why it's going to be discussed at the summit.
So my suggestion is remove all vendors' plugins and drivers except opensource 
as built-in.
By leaving open source plugins and drivers in the tree , we can resolve such 
problems:
  1)release a workable and COMPLETE version
  2)user experience(especially for beginners)
  3)provide code example to learn for new contributors and vendors
  4)develop and verify new features


 I think we should release a workable version.

Definitely. But that doesn't have anything to do with it living in the same 
repository. By putting it in a different repo, it provides smaller code bases 
to learn for new contributors wanting to become a core developer in addition to 
a clear separation between plugins and core code.
Why do we need a different repo to store vendors' codes? That's not the 
community business.
I think only a proper architecture and normal NBSB API can bring a clear 
separation between plugins(or drivers) and core code, not a different repo.
Of course, if the community provides a wiki page for vendors to add hyperlink 
of their codes, I think it's perfect.

 Besides of user experience, the open source drivers are also used for 
 developing and verifying new features, even small-scale case.

Sure, but this also isn't affected by the code being in a separate repo.
See comments above.

 The community should and just need focus on the Neutron core and provide 
 framework for vendors' devices.

I agree, but without the open source drivers being separated as well, it's very 
difficult for the framework for external drivers to be stable enough to be 
useful.
Architecture and API. The community should ensure core and API stable enough 
and high quality. Vendors for external drivers.
Who provides, who maintains(including development, storage, distribution, 
quality, etc).

On Thu, Sep 11, 2014 at 7:24 PM, Germy Lure 
germy.l...@gmail.commailto:germy.l...@gmail.com wrote:
Some comments inline.

BR,
Germy

On Thu, Sep 11, 2014 at 5:47 PM, Kevin Benton 
blak...@gmail.commailto:blak...@gmail.com wrote:
This has been brought up several times already and I believe is going to be 
discussed at the Kilo summit.
Maybe I missed something, but what's the solution?

I agree that reviewing third party patches eats community time. However, 
claiming that the community pays 46% of it's energy to maintain vendor-specific 
code doesn't make any sense. LOC in the repo has very little to do with ongoing 
required maintenance. Assuming the APIs for the plugins stay consistent, there 
should be few

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-12 Thread Germy Lure
On Fri, Sep 12, 2014 at 11:11 AM, Kevin Benton blak...@gmail.com wrote:


  Maybe I missed something, but what's the solution?

 There isn't one yet. That's why it's going to be discussed at the summit.

So my suggestion is remove all vendors' plugins and drivers except
opensource as built-in.
By leaving open source plugins and drivers in the tree , we can resolve
such problems:
  1)release a workable and COMPLETE version
  2)user experience(especially for beginners)
  3)provide code example to learn for new contributors and vendors
  4)develop and verify new features



  I think we should release a workable version.

 Definitely. But that doesn't have anything to do with it living in the
 same repository. By putting it in a different repo, it provides smaller
 code bases to learn for new contributors wanting to become a core developer
 in addition to a clear separation between plugins and core code.

Why do we need a different repo to store vendors' codes? That's not the
community business.
I think only a proper architecture and normal NBSB API can bring a clear
separation between plugins(or drivers) and core code, not a different repo.
Of course, if the community provides a wiki page for vendors to add
hyperlink of their codes, I think it's perfect.


  Besides of user experience, the open source drivers are also used for
 developing and verifying new features, even small-scale case.

 Sure, but this also isn't affected by the code being in a separate repo.

See comments above.


  The community should and just need focus on the Neutron core and provide
 framework for vendors' devices.

 I agree, but without the open source drivers being separated as well, it's
 very difficult for the framework for external drivers to be stable enough
 to be useful.

Architecture and API. The community should ensure core and API stable
enough and high quality. Vendors for external drivers.
Who provides, who maintains(including development, storage, distribution,
quality, etc).


 On Thu, Sep 11, 2014 at 7:24 PM, Germy Lure germy.l...@gmail.com wrote:

 Some comments inline.

 BR,
 Germy

 On Thu, Sep 11, 2014 at 5:47 PM, Kevin Benton blak...@gmail.com wrote:

 This has been brought up several times already and I believe is going to
 be discussed at the Kilo summit.

 Maybe I missed something, but what's the solution?


 I agree that reviewing third party patches eats community time. However,
 claiming that the community pays 46% of it's energy to maintain
 vendor-specific code doesn't make any sense. LOC in the repo has very
 little to do with ongoing required maintenance. Assuming the APIs for the
 plugins stay consistent, there should be few 'maintenance' changes required
 to a plugin once it's in the tree. If there are that many changes to
 plugins just to keep them operational, that means Neutron is far too
 unstable to support drivers living outside of the tree anyway.

 Yes, you are right. Neutron is far too unstable to support drivers
 living outside of the tree anyway. So I think this is really our important
 point.
 The community should focus on standardizing NBSB API, introducing and
 improving new features NOT wasting energy to introduce and maintain
 vendor-specific codes.


 On a related note, if we are going to pull plugins/drivers out of
 Neutron, I think all of them should be removed, including the OVS and
 LinuxBridge ones. There is no reason for them to be there if Neutron has
 stable enough internal APIs to eject the 3rd party plugins from the repo.
 They should be able to live in a separate neutron-opensource-drivers repo
 or something along those lines. This will free up significant amounts of
 developer/reviewer cycles for neutron to work on the API refactor, task
 based workflows, performance improvements for the DB operations, etc.

 I think we should release a workable version. User can experience the
 functions powered by built-in components. And they can replace them with
 the release of those vendors who cooperate with them. The community
 should not work for vendor's codes.


 If the open source drivers stay in the tree and the others are removed,
 there is little incentive to keep the internal APIs stable and 3rd party
 drivers sitting outside of the tree will break on every refactor or data
 structure change. If that's the way we want to treat external driver
 developers, let's be explicit about it and just post warnings that 3rd
 party drivers can break at any point and that the onus is on the external
 developers to learn what changed an react to it. At some point they will
 stop bothering with Neutron completely in their deployments and mimic its
 public API.

 Besides of user experience, the open source drivers are also used for
 developing and verifying new features, even small-scale case.


 A clear separation of the open source drivers/plugins and core Neutron
 would give a much better model for 3rd party driver developers to follow
 and would enforce a stable internal API in the 

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-12 Thread Kevin Benton
 So my suggestion is remove all vendors' plugins and drivers except
opensource as built-in.

Yes, I think this is currently the view held by the PTL (Kyle) and some of
the other cores so what you're suggesting will definitely come up at the
summit.


 Why do we need a different repo to store vendors' codes? That's not the
community business.
 I think only a proper architecture and normal NBSB API can bring a
clear separation between plugins(or drivers) and core code, not a
different repo.

The problem is that that architecture won't stay stable if there is no
shared community plugin depending on its stability. Let me ask you the
inverse question. Why do you think the reference driver should stay in the
core repo?

A separate repo won't have an impact on what is packaged and released so it
should have no impact on user experience, complete versions, providing
code examples,  or developing new features. In fact, it will likely help
with the last two because it will provide a clear delineation between what
a plugin is responsible for vs. what the core API is responsible for. And,
because new cores can be added faster to the open source plugins repo due
to a smaller code base to learn, it will help with developing new features
by reducing reviewer load.

On Fri, Sep 12, 2014 at 1:50 AM, Germy Lure germy.l...@gmail.com wrote:



 On Fri, Sep 12, 2014 at 11:11 AM, Kevin Benton blak...@gmail.com wrote:


  Maybe I missed something, but what's the solution?

 There isn't one yet. That's why it's going to be discussed at the summit.

 So my suggestion is remove all vendors' plugins and drivers except
 opensource as built-in.
 By leaving open source plugins and drivers in the tree , we can resolve
 such problems:
   1)release a workable and COMPLETE version
   2)user experience(especially for beginners)
   3)provide code example to learn for new contributors and vendors
   4)develop and verify new features



  I think we should release a workable version.

 Definitely. But that doesn't have anything to do with it living in the
 same repository. By putting it in a different repo, it provides smaller
 code bases to learn for new contributors wanting to become a core developer
 in addition to a clear separation between plugins and core code.

 Why do we need a different repo to store vendors' codes? That's not the
 community business.
 I think only a proper architecture and normal NBSB API can bring a clear
 separation between plugins(or drivers) and core code, not a different repo.
 Of course, if the community provides a wiki page for vendors to add
 hyperlink of their codes, I think it's perfect.


  Besides of user experience, the open source drivers are also used for
 developing and verifying new features, even small-scale case.

 Sure, but this also isn't affected by the code being in a separate repo.

 See comments above.


  The community should and just need focus on the Neutron core and
 provide framework for vendors' devices.

 I agree, but without the open source drivers being separated as well,
 it's very difficult for the framework for external drivers to be stable
 enough to be useful.

 Architecture and API. The community should ensure core and API stable
 enough and high quality. Vendors for external drivers.
 Who provides, who maintains(including development, storage, distribution,
 quality, etc).


 On Thu, Sep 11, 2014 at 7:24 PM, Germy Lure germy.l...@gmail.com wrote:

 Some comments inline.

 BR,
 Germy

 On Thu, Sep 11, 2014 at 5:47 PM, Kevin Benton blak...@gmail.com wrote:

 This has been brought up several times already and I believe is going
 to be discussed at the Kilo summit.

 Maybe I missed something, but what's the solution?


 I agree that reviewing third party patches eats community time.
 However, claiming that the community pays 46% of it's energy to maintain
 vendor-specific code doesn't make any sense. LOC in the repo has very
 little to do with ongoing required maintenance. Assuming the APIs for the
 plugins stay consistent, there should be few 'maintenance' changes required
 to a plugin once it's in the tree. If there are that many changes to
 plugins just to keep them operational, that means Neutron is far too
 unstable to support drivers living outside of the tree anyway.

 Yes, you are right. Neutron is far too unstable to support drivers
 living outside of the tree anyway. So I think this is really our important
 point.
 The community should focus on standardizing NBSB API, introducing and
 improving new features NOT wasting energy to introduce and maintain
 vendor-specific codes.


 On a related note, if we are going to pull plugins/drivers out of
 Neutron, I think all of them should be removed, including the OVS and
 LinuxBridge ones. There is no reason for them to be there if Neutron has
 stable enough internal APIs to eject the 3rd party plugins from the repo.
 They should be able to live in a separate neutron-opensource-drivers repo
 or something along those lines. This 

Re: [openstack-dev] [Neutron][Architecture]Suggestions for the third vendors' plugin and driver

2014-09-11 Thread Kevin Benton
This has been brought up several times already and I believe is going to be
discussed at the Kilo summit.

I agree that reviewing third party patches eats community time. However,
claiming that the community pays 46% of it's energy to maintain
vendor-specific code doesn't make any sense. LOC in the repo has very
little to do with ongoing required maintenance. Assuming the APIs for the
plugins stay consistent, there should be few 'maintenance' changes required
to a plugin once it's in the tree. If there are that many changes to
plugins just to keep them operational, that means Neutron is far too
unstable to support drivers living outside of the tree anyway.

On a related note, if we are going to pull plugins/drivers out of Neutron,
I think all of them should be removed, including the OVS and LinuxBridge
ones. There is no reason for them to be there if Neutron has stable enough
internal APIs to eject the 3rd party plugins from the repo. They should be
able to live in a separate neutron-opensource-drivers repo or something
along those lines. This will free up significant amounts of
developer/reviewer cycles for neutron to work on the API refactor, task
based workflows, performance improvements for the DB operations, etc.

If the open source drivers stay in the tree and the others are removed,
there is little incentive to keep the internal APIs stable and 3rd party
drivers sitting outside of the tree will break on every refactor or data
structure change. If that's the way we want to treat external driver
developers, let's be explicit about it and just post warnings that 3rd
party drivers can break at any point and that the onus is on the external
developers to learn what changed an react to it. At some point they will
stop bothering with Neutron completely in their deployments and mimic its
public API.

A clear separation of the open source drivers/plugins and core Neutron
would give a much better model for 3rd party driver developers to follow
and would enforce a stable internal API in the Neutron core.



On Thu, Sep 11, 2014 at 1:54 AM, Germy Lure germy.l...@gmail.com wrote:

 Hi stackers,

 According to my statistics(J2), the LOC of vendors' plugin and driver is
 about 102K, while the whole under neutron is 220K.
 That is to say the community has paid and is paying over 46% energy to
 maintain vendors' code. If we take mails, bugs,
 BPs  and so on into consideration, this percentage will be more.

 Most of these codes are just plugins and drivers implementing almost  the
 same functions. Every vendor submits a plugin,
 and the community only do the same thing, repeat and repeat. Meaningless.I
 think it's time to move them out.
 Let's focus on improving those exist but still weak features, on
 introducing important and interesting new features.

 My suggestions now:
 1.monopolized plugins
   1)The community only standards NB API and keeps built-ins, such as ML2,
 OVS and Linux bridge plugins.
   2)Vendors maintain their plugins locally.
   3)Users get neutron from community and plugin from some vendor on demand.
 2.service plugins
   1)The community standards SB API and keeps open source driver(iptables,
 openSwan and etc.) as built-in.
   2)Vendors only provide drivers not plugin. And those drivers also need
 not deliver to community.
   3)Like above, Users can get code on demand from vendors or just use open
 source.
 3.ML2 plugin
   1)Like service and monopolized plugin, the community just keep open
 source implementations as built-in.
   2)L2-population should be kept.

 I am very happy to discuss this further.

 vendors' code stat. table(excluding built-in plugins and drivers)
 
 Path Size
 neutron-master\neutron\plugins\63170
 neutron-master\neutron\services\ 4052
 neutron-master\neutron\tests\ 35756

 BR,
 Germy

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev