Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Anne Gentle]

On Thu, Aug 28, 2014 at 12:22 PM, Richard Woo 
wrote:

> I have another question about incubator proposal, for CLI and GUI. Do we
> imply that the incubator feature will need to branch python-neutron client,
> Horizon, and or Nova ( if changes are needed)?
>
>
>
>
This is as good a place for the docs perspective as any on this thread.

I think it's great to increase your dev velocity, but in the proposal I
need to see a well-thought-out plan for when documentation should be on
docs.openstack.org for deployers and users and when documentation should be
in the API reference page at http://developer.openstack.org/api-ref.html. A
section addressing the timing and requirements would be sufficient.

The docs affected are:
CLI Reference http://docs.openstack.org/cli-reference/content/
Config Reference
http://docs.openstack.org/icehouse/config-reference/content/
User Guide http://docs.openstack.org/user-guide/content/
Admin User Guide http://docs.openstack.org/user-guide-admin/content/
Cloud Admin User Guide http://docs.openstack.org/admin-guide-cloud/content/
API Reference http://developer.openstack.org/api-ref.html

I won't argue that the Install Guide should be included as these items
aren't exactly "happy path" quite yet.

Also in the wiki page, I see a line saying " link to commits in private
trees is acceptable" -- is it really? How would readers get to it?

Thanks,
Anne


>
> On Tue, Aug 26, 2014 at 7:09 PM, James E. Blair 
> wrote:
>
>> Hi,
>>
>> After reading https://wiki.openstack.org/wiki/Network/Incubator I have
>> some thoughts about the proposed workflow.
>>
>> We have quite a bit of experience and some good tools around splitting
>> code out of projects and into new projects.  But we don't generally do a
>> lot of importing code into projects.  We've done this once, to my
>> recollection, in a way that preserved history, and that was with the
>> switch to keystone-lite.
>>
>> It wasn't easy; it's major git surgery and would require significant
>> infra-team involvement any time we wanted to do it.
>>
>> However, reading the proposal, it occurred to me that it's pretty clear
>> that we expect these tools to be able to operate outside of the Neutron
>> project itself, to even be releasable on their own.  Why not just stick
>> with that?  In other words, the goal of this process should be to create
>> separate projects with their own development lifecycle that will
>> continue indefinitely, rather than expecting the code itself to merge
>> into the neutron repo.
>>
>> This has advantages in simplifying workflow and making it more
>> consistent.  Plus it builds on known integration mechanisms like APIs
>> and python project versions.
>>
>> But more importantly, it helps scale the neutron project itself.  I
>> think that a focused neutron core upon which projects like these can
>> build on in a reliable fashion would be ideal.
>>
>> -Jim
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

Yes, in theory all of the plugins should be removable from the core neutron
repo. So then it would only need to be responsible for the APIs, db models,
etc. However, IIRC there are no plans to move any reference plugins from
the tree.


On Thu, Aug 28, 2014 at 11:20 AM, Jeremy Stanley  wrote:

> On 2014-08-28 08:31:26 -0700 (-0700), Kevin Benton wrote:
> [...]
> > DVR completely changed the reference L3 service plugin, which
> > lives in the main tree.
> >
> > A well-defined, versioned internal API would not have helped any
> > of the issues I brought up.
> [...]
>
> Except, perhaps, insofar as that (in some ideal world) it might
> allow the reference L3 service plugin to be extracted from the main
> tree and developed within a separate source code repository with its
> own life cycle.
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Jeremy Stanley]

On 2014-08-28 08:31:26 -0700 (-0700), Kevin Benton wrote:
[...]
> DVR completely changed the reference L3 service plugin, which
> lives in the main tree. 
> 
> A well-defined, versioned internal API would not have helped any
> of the issues I brought up.
[...]

Except, perhaps, insofar as that (in some ideal world) it might
allow the reference L3 service plugin to be extracted from the main
tree and developed within a separate source code repository with its
own life cycle.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Richard Woo]

I have another question about incubator proposal, for CLI and GUI. Do we
imply that the incubator feature will need to branch python-neutron client,
Horizon, and or Nova ( if changes are needed)?




On Tue, Aug 26, 2014 at 7:09 PM, James E. Blair  wrote:

> Hi,
>
> After reading https://wiki.openstack.org/wiki/Network/Incubator I have
> some thoughts about the proposed workflow.
>
> We have quite a bit of experience and some good tools around splitting
> code out of projects and into new projects.  But we don't generally do a
> lot of importing code into projects.  We've done this once, to my
> recollection, in a way that preserved history, and that was with the
> switch to keystone-lite.
>
> It wasn't easy; it's major git surgery and would require significant
> infra-team involvement any time we wanted to do it.
>
> However, reading the proposal, it occurred to me that it's pretty clear
> that we expect these tools to be able to operate outside of the Neutron
> project itself, to even be releasable on their own.  Why not just stick
> with that?  In other words, the goal of this process should be to create
> separate projects with their own development lifecycle that will
> continue indefinitely, rather than expecting the code itself to merge
> into the neutron repo.
>
> This has advantages in simplifying workflow and making it more
> consistent.  Plus it builds on known integration mechanisms like APIs
> and python project versions.
>
> But more importantly, it helps scale the neutron project itself.  I
> think that a focused neutron core upon which projects like these can
> build on in a reliable fashion would be ideal.
>
> -Jim
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Mark McClain]

On Aug 28, 2014, at 10:45 AM, Jay Pipes  wrote:

> On 08/27/2014 04:28 PM, Kevin Benton wrote:
>> What are you talking about? The only reply was from me clarifying that
>> one of the purposes of the incubator was for components of neutron that
>> are experimental but are intended to be merged.
> 
> Right. The special unicorns.
> 
> > In that case it might
>> not make sense to have a life cycle of their own in another repo
>> indefinitely.
> 
> The main reasons these "experimental components" don't make sense to live in 
> their own repo indefinitely are:
> 
> a) Neutron's design doesn't make it easy or straightforward to build/layer 
> other things on top of it, or:
> 

Correct and this something I want the team to address in Kilo.  Many of the L7 
services would be easier to build if we invest some time early in the cycle to 
establishing a well defined interface for a few items.  I’m sure the LBaaS team 
has good feedback to share with everyone.

> b) The experimental piece of code intends to replace whole-hog a large chunk 
> of Neutron's existing codebase, or:
> 
> c) The experimental piece of code relies so heavily on inconsistent, 
> unversioned internal interface and plugin calls that it cannot be designed 
> externally due to the fragility of those interfaces

I’m glad Jim reminded us of the pain of merging histories and the availability 
of feature branches.  I think for items where we’re replacing large chunks of 
code feature branches make lots of sense.

> 
> Fixing a) is the solution to these problems. An incubator area where 
> "experimental components" can live will just continue to mask the true 
> problem domain, which is that Neutron's design is cumbersome to build on top 
> of, and its cross-component interfaces need to be versioned, made consistent, 
> and cleaned up to use versioned data structures instead of passing random 
> nested dicts of randomly-prefixed string key/values.
> 
> Frankly, we're going through a similar problem in Nova right now. There is a 
> group of folks who believe that separating the nova-scheduler code into the 
> Gantt project will magically make placement decision code and solver 
> components *easier* to work on (because the pace of coding can be increased 
> if there wasn't that pesky nova-core review process). But this is not 
> correct, IMO. Separating out the scheduler into its own project before 
> internal interfaces and data structures are cleaned up and versioned will 
> just lead to greater technical debt and an increase in frustration on the 
> part of Nova developers and scheduler developers alike.

Right hopefully the incubator will allow us to develop components that should 
be independent from the start without incurring too much debt.

> 
> -jay
> 
>> On Wed, Aug 27, 2014 at 11:52 AM, Jay Pipes > > wrote:
>> 
>>On 08/26/2014 07:09 PM, James E. Blair wrote:
>> 
>>Hi,
>> 
>>After reading
>>https://wiki.openstack.org/__wiki/Network/Incubator
>> I have
>>some thoughts about the proposed workflow.
>> 
>>We have quite a bit of experience and some good tools around
>>splitting
>>code out of projects and into new projects.  But we don't
>>generally do a
>>lot of importing code into projects.  We've done this once, to my
>>recollection, in a way that preserved history, and that was with the
>>switch to keystone-lite.
>> 
>>It wasn't easy; it's major git surgery and would require significant
>>infra-team involvement any time we wanted to do it.
>> 
>>However, reading the proposal, it occurred to me that it's
>>pretty clear
>>that we expect these tools to be able to operate outside of the
>>Neutron
>>project itself, to even be releasable on their own.  Why not
>>just stick
>>with that?  In other words, the goal of this process should be
>>to create
>>separate projects with their own development lifecycle that will
>>continue indefinitely, rather than expecting the code itself to
>>merge
>>into the neutron repo.
>> 
>>This has advantages in simplifying workflow and making it more
>>consistent.  Plus it builds on known integration mechanisms like
>>APIs
>>and python project versions.
>> 
>>But more importantly, it helps scale the neutron project itself.  I
>>think that a focused neutron core upon which projects like these can
>>build on in a reliable fashion would be ideal.
>> 
>> 
>>Despite replies to you saying that certain branches of Neutron
>>development work are special unicorns, I wanted to say I *fully*
>>support your above statement.
>> 
>>Best,
>>-jay
>> 
>> 
>> 
>>_
>>OpenStack-dev mailing list
>>OpenStack-dev@lists.openstack

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

>Right. The special unicorns.

Repeating this without defining it isn't helping anything.

>b) The experimental piece of code intends to replace whole-hog a large
chunk of Neutron's existing codebase, or:

In the DVR example I gave this is is the only relevant reason. Regardless
of how well the internal Neutron APIs are defined, this same problem would
have arisen. DVR completely changed the reference L3 service plugin, which
lives in the main tree.

A well-defined, versioned internal API would not have helped any of the
issues I brought up. The 'experimental' part of the DVR work isn't
referring to its internal API modifications, its referring to how the L3
service plugin will integrate with the data plane.


On Thu, Aug 28, 2014 at 7:45 AM, Jay Pipes  wrote:

> On 08/27/2014 04:28 PM, Kevin Benton wrote:
>
>> What are you talking about? The only reply was from me clarifying that
>> one of the purposes of the incubator was for components of neutron that
>> are experimental but are intended to be merged.
>>
>
> Right. The special unicorns.
>
>
> > In that case it might
>
>> not make sense to have a life cycle of their own in another repo
>> indefinitely.
>>
>
> The main reasons these "experimental components" don't make sense to live
> in their own repo indefinitely are:
>
> a) Neutron's design doesn't make it easy or straightforward to build/layer
> other things on top of it, or:
>
> b) The experimental piece of code intends to replace whole-hog a large
> chunk of Neutron's existing codebase, or:
>
> c) The experimental piece of code relies so heavily on inconsistent,
> unversioned internal interface and plugin calls that it cannot be designed
> externally due to the fragility of those interfaces
>
> Fixing a) is the solution to these problems. An incubator area where
> "experimental components" can live will just continue to mask the true
> problem domain, which is that Neutron's design is cumbersome to build on
> top of, and its cross-component interfaces need to be versioned, made
> consistent, and cleaned up to use versioned data structures instead of
> passing random nested dicts of randomly-prefixed string key/values.
>
> Frankly, we're going through a similar problem in Nova right now. There is
> a group of folks who believe that separating the nova-scheduler code into
> the Gantt project will magically make placement decision code and solver
> components *easier* to work on (because the pace of coding can be increased
> if there wasn't that pesky nova-core review process). But this is not
> correct, IMO. Separating out the scheduler into its own project before
> internal interfaces and data structures are cleaned up and versioned will
> just lead to greater technical debt and an increase in frustration on the
> part of Nova developers and scheduler developers alike.
>
> -jay
>
>  On Wed, Aug 27, 2014 at 11:52 AM, Jay Pipes > > wrote:
>>
>> On 08/26/2014 07:09 PM, James E. Blair wrote:
>>
>> Hi,
>>
>> After reading
>> https://wiki.openstack.org/__wiki/Network/Incubator
>>
>>  I have
>> some thoughts about the proposed workflow.
>>
>> We have quite a bit of experience and some good tools around
>> splitting
>> code out of projects and into new projects.  But we don't
>> generally do a
>> lot of importing code into projects.  We've done this once, to my
>> recollection, in a way that preserved history, and that was with
>> the
>> switch to keystone-lite.
>>
>> It wasn't easy; it's major git surgery and would require
>> significant
>> infra-team involvement any time we wanted to do it.
>>
>> However, reading the proposal, it occurred to me that it's
>> pretty clear
>> that we expect these tools to be able to operate outside of the
>> Neutron
>> project itself, to even be releasable on their own.  Why not
>> just stick
>> with that?  In other words, the goal of this process should be
>> to create
>> separate projects with their own development lifecycle that will
>> continue indefinitely, rather than expecting the code itself to
>> merge
>> into the neutron repo.
>>
>> This has advantages in simplifying workflow and making it more
>> consistent.  Plus it builds on known integration mechanisms like
>> APIs
>> and python project versions.
>>
>> But more importantly, it helps scale the neutron project itself.
>> I
>> think that a focused neutron core upon which projects like these
>> can
>> build on in a reliable fashion would be ideal.
>>
>>
>> Despite replies to you saying that certain branches of Neutron
>> development work are special unicorns, I wanted to say I *fully*
>> support your above statement.
>>
>> Best,
>> -jay
>>
>>
>>
>> 

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Jay Pipes]

On 08/27/2014 04:28 PM, Kevin Benton wrote:

What are you talking about? The only reply was from me clarifying that
one of the purposes of the incubator was for components of neutron that
are experimental but are intended to be merged.


Right. The special unicorns.

> In that case it might

not make sense to have a life cycle of their own in another repo
indefinitely.


The main reasons these "experimental components" don't make sense to 
live in their own repo indefinitely are:


a) Neutron's design doesn't make it easy or straightforward to 
build/layer other things on top of it, or:


b) The experimental piece of code intends to replace whole-hog a large 
chunk of Neutron's existing codebase, or:


c) The experimental piece of code relies so heavily on inconsistent, 
unversioned internal interface and plugin calls that it cannot be 
designed externally due to the fragility of those interfaces


Fixing a) is the solution to these problems. An incubator area where 
"experimental components" can live will just continue to mask the true 
problem domain, which is that Neutron's design is cumbersome to build on 
top of, and its cross-component interfaces need to be versioned, made 
consistent, and cleaned up to use versioned data structures instead of 
passing random nested dicts of randomly-prefixed string key/values.


Frankly, we're going through a similar problem in Nova right now. There 
is a group of folks who believe that separating the nova-scheduler code 
into the Gantt project will magically make placement decision code and 
solver components *easier* to work on (because the pace of coding can be 
increased if there wasn't that pesky nova-core review process). But this 
is not correct, IMO. Separating out the scheduler into its own project 
before internal interfaces and data structures are cleaned up and 
versioned will just lead to greater technical debt and an increase in 
frustration on the part of Nova developers and scheduler developers alike.


-jay


On Wed, Aug 27, 2014 at 11:52 AM, Jay Pipes mailto:jaypi...@gmail.com>> wrote:

On 08/26/2014 07:09 PM, James E. Blair wrote:

Hi,

After reading
https://wiki.openstack.org/__wiki/Network/Incubator
 I have
some thoughts about the proposed workflow.

We have quite a bit of experience and some good tools around
splitting
code out of projects and into new projects.  But we don't
generally do a
lot of importing code into projects.  We've done this once, to my
recollection, in a way that preserved history, and that was with the
switch to keystone-lite.

It wasn't easy; it's major git surgery and would require significant
infra-team involvement any time we wanted to do it.

However, reading the proposal, it occurred to me that it's
pretty clear
that we expect these tools to be able to operate outside of the
Neutron
project itself, to even be releasable on their own.  Why not
just stick
with that?  In other words, the goal of this process should be
to create
separate projects with their own development lifecycle that will
continue indefinitely, rather than expecting the code itself to
merge
into the neutron repo.

This has advantages in simplifying workflow and making it more
consistent.  Plus it builds on known integration mechanisms like
APIs
and python project versions.

But more importantly, it helps scale the neutron project itself.  I
think that a focused neutron core upon which projects like these can
build on in a reliable fashion would be ideal.


Despite replies to you saying that certain branches of Neutron
development work are special unicorns, I wanted to say I *fully*
support your above statement.

Best,
-jay



_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 





--
Kevin Benton


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Ivar Lazzaro]

On Wed, Aug 27, 2014 at 5:06 PM, Jeremy Stanley  wrote:
>
> Good point, and that definitely is a reason to just keep those
> pieces in their own separate Git repositories outside of the core
> Neutron repository in perpetuity (even after they "graduate" from
> incubation). One package per repository. That should be chiseled in
> stone somewhere.


Just asking, wasn't there a proposal about doing something similar for all
the neutron services? (l3/fw/lb/vnp ...)

Ivar.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Dean Troyer]

On Wed, Aug 27, 2014 at 6:59 PM, Mandeep Dhami 
wrote:

>
> Also note that one of the "features" of the incubator is that it can be
> packaged/released independently (like python-neutronclient) and a feature
> branch is not designed for that workflow.
>

Which is a good reason to not use the word 'incubator'.  It also has the
connotations from Oslo of being 'copy-pasta' code.

dt

-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Jeremy Stanley]

On 2014-08-27 16:59:54 -0700 (-0700), Mandeep Dhami wrote:
> Also note that one of the "features" of the incubator is that it can be
> packaged/released independently (like python-neutronclient) and a feature
> branch is not designed for that workflow.

Good point, and that definitely is a reason to just keep those
pieces in their own separate Git repositories outside of the core
Neutron repository in perpetuity (even after they "graduate" from
incubation). One package per repository. That should be chiseled in
stone somewhere.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Jeremy Stanley]

On 2014-08-27 16:53:39 -0700 (-0700), Clark Boylan wrote:
[...]
> I thought there was a wiki article on how they work but I can't
> find it. Maybe someone else can link it here.
[...]

https://wiki.openstack.org/wiki/GerritJenkinsGit#Merge_Commits

-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Mandeep Dhami]

Also note that one of the "features" of the incubator is that it can be
packaged/released independently (like python-neutronclient) and a feature
branch is not designed for that workflow.

Regards,
Mandeep



On Wed, Aug 27, 2014 at 4:55 PM, Jeremy Stanley  wrote:

> On 2014-08-27 16:05:36 -0700 (-0700), Joe Gordon wrote:
> > We have feature branches? How do we use them? I am not even sure
> > if feature branches were evaluated as an option when the incubator
> > was proposed.
>
> Keystone's new API used a feature branch, the EC work in Swift did,
> so did the Gearman integration for Zuul... however it's also an
> option not to be taken lightly and comes with its own set of unique
> challenges.
> --
> Jeremy Stanley
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Jeremy Stanley]

On 2014-08-27 16:05:36 -0700 (-0700), Joe Gordon wrote:
> We have feature branches? How do we use them? I am not even sure
> if feature branches were evaluated as an option when the incubator
> was proposed.

Keystone's new API used a feature branch, the EC work in Swift did,
so did the Gearman integration for Zuul... however it's also an
option not to be taken lightly and comes with its own set of unique
challenges.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Clark Boylan]

On Wed, Aug 27, 2014, at 04:05 PM, Joe Gordon wrote:
> On Wed, Aug 27, 2014 at 12:30 PM, James E. Blair 
> wrote:
> 
> > Kevin Benton  writes:
> >
> > > From what I understand, the intended projects for the incubator can't
> > > operate without neutron because they are just extensions/plugins/drivers.
> >
> > I could have phrased that better.  What I meant was that they could
> > operate without being actually in the Neutron repo, not that they could
> > not operate without Neutron itself.
> >
> > The proposal for the incubator is that extensions be developed outside
> > of the Neutron repo.  My proposed refinement is that they stay outside
> > of the Neutron repo.  They live their entire lives as extension modules
> > in separate projects.
> >
> > > For example, if the DVR modifications to the reference reference L3
> > plugin
> > > weren't already being developed in the tree, DVR could have been
> > developed
> > > in the incubator and then merged into Neutron once the bugs were ironed
> > out
> > > so a huge string of Gerrit patches didn't need to be tracked. If that had
> > > happened, would it make sense to keep the L3 plugin as a completely
> > > separate project or merge it? I understand this is the approach the load
> > > balancer folks took by making Octavia a separate project, but I think it
> > > can still operate on its own, where the reference L3 plugin (and many of
> > > the other incubator projects) are just classes that expect to be able to
> > > make core Neutron calls.
> >
> > The list of Juno/Kilo candidates doesn't seem to have projects that are
> > quite so low-level.
> >
> > If a feature is going to become part of the neutron core, then it should
> > be developed in the neutron repository.  If we need a place to land code
> > that isn't master, it's actually far easier to just use a feature branch
> > on the neutron repo.  Commits can land there as needed, master can be
> > periodically merged into it, and when the feature is ready, the feature
> > branch can be merged into master.
> >
> 
> We have feature branches? How do we use them? I am not even sure if
> feature
> branches were evaluated as an option when the incubator was proposed.
> 
We do. I thought there was a wiki article on how they work but I can't
find it. Maybe someone else can link it here.

The basics are someone creates a branch called feature/foo via the
Gerrit GUI [0], for OpenStack projects members of the release managers
group can do this and for other projects we can update ACLs to give the
-release group permissions to do this (and infra can always do it). Then
you push code against that branch `git review feature/foo` (it will push
to the correct branch if you edit the .gitreview file on that branch).

Periodically you will want to merge master back into your feature
branches so that you don't lag too far behind and when the feature is
ready you merge it back into master. You need special perms to push
merge commits to Gerrit so that is something that may need to be worked
out depending on how the existing ACLs look for a project. The swift
devs recently did this for their EC feature and may have more feedback.

[0]
https://review.openstack.org/#/admin/projects/openstack/neutron,branches
> 
> >
> > I think between those two options: incubate/spin-out components that are
> > high-level enough not to have deep integration in the neutron core, and
> > using feature branches for large experimental changes to the core
> > itself, we can handle the problems the incubator repo is intended to
> > address.
> >
> > -Jim
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Clark Boylan]

On Wed, Aug 27, 2014, at 03:52 PM, Ivar Lazzaro wrote:
> Quoting Stefano from a different thread [0]:
> 
> The rationale for the separate repository is that Neutron's code needs a
> > lot of love for the *existing* codebase, before new features can be
> > added (and before core team can accept more responsibilities for it).
> > But progress is unstoppable: new features are being proposed every cycle
> > and reviewers bandwidth is not infinite.
> 
> 
> The long term purpose of the incubator should be to gather all the big
> new
> features that may require fast iteration before becoming stable and being
> eventually merged into the main tree. This is especially important for
> "core" new features, that may have a higher impact on Neutron's
> stability.
> A great example was made by Kevin with DVR.
>
The issue that Jim was trying to point out is that 'being eventually
merged into the main tree' is not something you can just do. It is
extremely costly to merge two repos together in this fashion and we
should avoid it at all costs. Feature branches make this relatively
cheap and should probably be considered instead.
> 
> [0]
> http://lists.openstack.org/pipermail/openstack-dev/2014-August/044224.html
> 
> 
> On Wed, Aug 27, 2014 at 1:32 PM, Kevin Benton  wrote:
> 
> > I agree that components isolated to one service or something along those
> > lines where there is a clear plugin point in Neutron, it might make sense
> > to separate them permanently. However, at that point why even bother with
> > the Neutron incubator when a new project can be started?
> >
> > The feature branch idea sounds interesting for the one-time big
> > experimental changes. Are there any other projects that do this right now?
> >
> >
> > On Wed, Aug 27, 2014 at 12:30 PM, James E. Blair 
> > wrote:
> >
> >> Kevin Benton  writes:
> >>
> >> > From what I understand, the intended projects for the incubator can't
> >> > operate without neutron because they are just
> >> extensions/plugins/drivers.
> >>
> >> I could have phrased that better.  What I meant was that they could
> >> operate without being actually in the Neutron repo, not that they could
> >> not operate without Neutron itself.
> >>
> >> The proposal for the incubator is that extensions be developed outside
> >> of the Neutron repo.  My proposed refinement is that they stay outside
> >> of the Neutron repo.  They live their entire lives as extension modules
> >> in separate projects.
> >>
> >> > For example, if the DVR modifications to the reference reference L3
> >> plugin
> >> > weren't already being developed in the tree, DVR could have been
> >> developed
> >> > in the incubator and then merged into Neutron once the bugs were ironed
> >> out
> >> > so a huge string of Gerrit patches didn't need to be tracked. If that
> >> had
> >> > happened, would it make sense to keep the L3 plugin as a completely
> >> > separate project or merge it? I understand this is the approach the load
> >> > balancer folks took by making Octavia a separate project, but I think it
> >> > can still operate on its own, where the reference L3 plugin (and many of
> >> > the other incubator projects) are just classes that expect to be able to
> >> > make core Neutron calls.
> >>
> >> The list of Juno/Kilo candidates doesn't seem to have projects that are
> >> quite so low-level.
> >>
> >> If a feature is going to become part of the neutron core, then it should
> >> be developed in the neutron repository.  If we need a place to land code
> >> that isn't master, it's actually far easier to just use a feature branch
> >> on the neutron repo.  Commits can land there as needed, master can be
> >> periodically merged into it, and when the feature is ready, the feature
> >> branch can be merged into master.
> >>
> >> I think between those two options: incubate/spin-out components that are
> >> high-level enough not to have deep integration in the neutron core, and
> >> using feature branches for large experimental changes to the core
> >> itself, we can handle the problems the incubator repo is intended to
> >> address.
> >>
> >> -Jim
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> > --
> > Kevin Benton
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Joe Gordon]

On Wed, Aug 27, 2014 at 12:30 PM, James E. Blair 
wrote:

> Kevin Benton  writes:
>
> > From what I understand, the intended projects for the incubator can't
> > operate without neutron because they are just extensions/plugins/drivers.
>
> I could have phrased that better.  What I meant was that they could
> operate without being actually in the Neutron repo, not that they could
> not operate without Neutron itself.
>
> The proposal for the incubator is that extensions be developed outside
> of the Neutron repo.  My proposed refinement is that they stay outside
> of the Neutron repo.  They live their entire lives as extension modules
> in separate projects.
>
> > For example, if the DVR modifications to the reference reference L3
> plugin
> > weren't already being developed in the tree, DVR could have been
> developed
> > in the incubator and then merged into Neutron once the bugs were ironed
> out
> > so a huge string of Gerrit patches didn't need to be tracked. If that had
> > happened, would it make sense to keep the L3 plugin as a completely
> > separate project or merge it? I understand this is the approach the load
> > balancer folks took by making Octavia a separate project, but I think it
> > can still operate on its own, where the reference L3 plugin (and many of
> > the other incubator projects) are just classes that expect to be able to
> > make core Neutron calls.
>
> The list of Juno/Kilo candidates doesn't seem to have projects that are
> quite so low-level.
>
> If a feature is going to become part of the neutron core, then it should
> be developed in the neutron repository.  If we need a place to land code
> that isn't master, it's actually far easier to just use a feature branch
> on the neutron repo.  Commits can land there as needed, master can be
> periodically merged into it, and when the feature is ready, the feature
> branch can be merged into master.
>

We have feature branches? How do we use them? I am not even sure if feature
branches were evaluated as an option when the incubator was proposed.


>
> I think between those two options: incubate/spin-out components that are
> high-level enough not to have deep integration in the neutron core, and
> using feature branches for large experimental changes to the core
> itself, we can handle the problems the incubator repo is intended to
> address.
>
> -Jim
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Ivar Lazzaro]

Quoting Stefano from a different thread [0]:

The rationale for the separate repository is that Neutron's code needs a
> lot of love for the *existing* codebase, before new features can be
> added (and before core team can accept more responsibilities for it).
> But progress is unstoppable: new features are being proposed every cycle
> and reviewers bandwidth is not infinite.


The long term purpose of the incubator should be to gather all the big new
features that may require fast iteration before becoming stable and being
eventually merged into the main tree. This is especially important for
"core" new features, that may have a higher impact on Neutron's stability.
A great example was made by Kevin with DVR.

[0]
http://lists.openstack.org/pipermail/openstack-dev/2014-August/044224.html


On Wed, Aug 27, 2014 at 1:32 PM, Kevin Benton  wrote:

> I agree that components isolated to one service or something along those
> lines where there is a clear plugin point in Neutron, it might make sense
> to separate them permanently. However, at that point why even bother with
> the Neutron incubator when a new project can be started?
>
> The feature branch idea sounds interesting for the one-time big
> experimental changes. Are there any other projects that do this right now?
>
>
> On Wed, Aug 27, 2014 at 12:30 PM, James E. Blair 
> wrote:
>
>> Kevin Benton  writes:
>>
>> > From what I understand, the intended projects for the incubator can't
>> > operate without neutron because they are just
>> extensions/plugins/drivers.
>>
>> I could have phrased that better.  What I meant was that they could
>> operate without being actually in the Neutron repo, not that they could
>> not operate without Neutron itself.
>>
>> The proposal for the incubator is that extensions be developed outside
>> of the Neutron repo.  My proposed refinement is that they stay outside
>> of the Neutron repo.  They live their entire lives as extension modules
>> in separate projects.
>>
>> > For example, if the DVR modifications to the reference reference L3
>> plugin
>> > weren't already being developed in the tree, DVR could have been
>> developed
>> > in the incubator and then merged into Neutron once the bugs were ironed
>> out
>> > so a huge string of Gerrit patches didn't need to be tracked. If that
>> had
>> > happened, would it make sense to keep the L3 plugin as a completely
>> > separate project or merge it? I understand this is the approach the load
>> > balancer folks took by making Octavia a separate project, but I think it
>> > can still operate on its own, where the reference L3 plugin (and many of
>> > the other incubator projects) are just classes that expect to be able to
>> > make core Neutron calls.
>>
>> The list of Juno/Kilo candidates doesn't seem to have projects that are
>> quite so low-level.
>>
>> If a feature is going to become part of the neutron core, then it should
>> be developed in the neutron repository.  If we need a place to land code
>> that isn't master, it's actually far easier to just use a feature branch
>> on the neutron repo.  Commits can land there as needed, master can be
>> periodically merged into it, and when the feature is ready, the feature
>> branch can be merged into master.
>>
>> I think between those two options: incubate/spin-out components that are
>> high-level enough not to have deep integration in the neutron core, and
>> using feature branches for large experimental changes to the core
>> itself, we can handle the problems the incubator repo is intended to
>> address.
>>
>> -Jim
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Kevin Benton
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

I agree that components isolated to one service or something along those
lines where there is a clear plugin point in Neutron, it might make sense
to separate them permanently. However, at that point why even bother with
the Neutron incubator when a new project can be started?

The feature branch idea sounds interesting for the one-time big
experimental changes. Are there any other projects that do this right now?


On Wed, Aug 27, 2014 at 12:30 PM, James E. Blair 
wrote:

> Kevin Benton  writes:
>
> > From what I understand, the intended projects for the incubator can't
> > operate without neutron because they are just extensions/plugins/drivers.
>
> I could have phrased that better.  What I meant was that they could
> operate without being actually in the Neutron repo, not that they could
> not operate without Neutron itself.
>
> The proposal for the incubator is that extensions be developed outside
> of the Neutron repo.  My proposed refinement is that they stay outside
> of the Neutron repo.  They live their entire lives as extension modules
> in separate projects.
>
> > For example, if the DVR modifications to the reference reference L3
> plugin
> > weren't already being developed in the tree, DVR could have been
> developed
> > in the incubator and then merged into Neutron once the bugs were ironed
> out
> > so a huge string of Gerrit patches didn't need to be tracked. If that had
> > happened, would it make sense to keep the L3 plugin as a completely
> > separate project or merge it? I understand this is the approach the load
> > balancer folks took by making Octavia a separate project, but I think it
> > can still operate on its own, where the reference L3 plugin (and many of
> > the other incubator projects) are just classes that expect to be able to
> > make core Neutron calls.
>
> The list of Juno/Kilo candidates doesn't seem to have projects that are
> quite so low-level.
>
> If a feature is going to become part of the neutron core, then it should
> be developed in the neutron repository.  If we need a place to land code
> that isn't master, it's actually far easier to just use a feature branch
> on the neutron repo.  Commits can land there as needed, master can be
> periodically merged into it, and when the feature is ready, the feature
> branch can be merged into master.
>
> I think between those two options: incubate/spin-out components that are
> high-level enough not to have deep integration in the neutron core, and
> using feature branches for large experimental changes to the core
> itself, we can handle the problems the incubator repo is intended to
> address.
>
> -Jim
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

What are you talking about? The only reply was from me clarifying that one
of the purposes of the incubator was for components of neutron that are
experimental but are intended to be merged. In that case it might not make
sense to have a life cycle of their own in another repo indefinitely.


On Wed, Aug 27, 2014 at 11:52 AM, Jay Pipes  wrote:

> On 08/26/2014 07:09 PM, James E. Blair wrote:
>
>> Hi,
>>
>> After reading https://wiki.openstack.org/wiki/Network/Incubator I have
>> some thoughts about the proposed workflow.
>>
>> We have quite a bit of experience and some good tools around splitting
>> code out of projects and into new projects.  But we don't generally do a
>> lot of importing code into projects.  We've done this once, to my
>> recollection, in a way that preserved history, and that was with the
>> switch to keystone-lite.
>>
>> It wasn't easy; it's major git surgery and would require significant
>> infra-team involvement any time we wanted to do it.
>>
>> However, reading the proposal, it occurred to me that it's pretty clear
>> that we expect these tools to be able to operate outside of the Neutron
>> project itself, to even be releasable on their own.  Why not just stick
>> with that?  In other words, the goal of this process should be to create
>> separate projects with their own development lifecycle that will
>> continue indefinitely, rather than expecting the code itself to merge
>> into the neutron repo.
>>
>> This has advantages in simplifying workflow and making it more
>> consistent.  Plus it builds on known integration mechanisms like APIs
>> and python project versions.
>>
>> But more importantly, it helps scale the neutron project itself.  I
>> think that a focused neutron core upon which projects like these can
>> build on in a reliable fashion would be ideal.
>>
>
> Despite replies to you saying that certain branches of Neutron development
> work are special unicorns, I wanted to say I *fully* support your above
> statement.
>
> Best,
> -jay
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[James E. Blair]

Kevin Benton  writes:

> From what I understand, the intended projects for the incubator can't
> operate without neutron because they are just extensions/plugins/drivers.

I could have phrased that better.  What I meant was that they could
operate without being actually in the Neutron repo, not that they could
not operate without Neutron itself.

The proposal for the incubator is that extensions be developed outside
of the Neutron repo.  My proposed refinement is that they stay outside
of the Neutron repo.  They live their entire lives as extension modules
in separate projects.

> For example, if the DVR modifications to the reference reference L3 plugin
> weren't already being developed in the tree, DVR could have been developed
> in the incubator and then merged into Neutron once the bugs were ironed out
> so a huge string of Gerrit patches didn't need to be tracked. If that had
> happened, would it make sense to keep the L3 plugin as a completely
> separate project or merge it? I understand this is the approach the load
> balancer folks took by making Octavia a separate project, but I think it
> can still operate on its own, where the reference L3 plugin (and many of
> the other incubator projects) are just classes that expect to be able to
> make core Neutron calls.

The list of Juno/Kilo candidates doesn't seem to have projects that are
quite so low-level.

If a feature is going to become part of the neutron core, then it should
be developed in the neutron repository.  If we need a place to land code
that isn't master, it's actually far easier to just use a feature branch
on the neutron repo.  Commits can land there as needed, master can be
periodically merged into it, and when the feature is ready, the feature
branch can be merged into master.

I think between those two options: incubate/spin-out components that are
high-level enough not to have deep integration in the neutron core, and
using feature branches for large experimental changes to the core
itself, we can handle the problems the incubator repo is intended to
address.

-Jim

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Jay Pipes]

On 08/26/2014 07:09 PM, James E. Blair wrote:

Hi,

After reading https://wiki.openstack.org/wiki/Network/Incubator I have
some thoughts about the proposed workflow.

We have quite a bit of experience and some good tools around splitting
code out of projects and into new projects.  But we don't generally do a
lot of importing code into projects.  We've done this once, to my
recollection, in a way that preserved history, and that was with the
switch to keystone-lite.

It wasn't easy; it's major git surgery and would require significant
infra-team involvement any time we wanted to do it.

However, reading the proposal, it occurred to me that it's pretty clear
that we expect these tools to be able to operate outside of the Neutron
project itself, to even be releasable on their own.  Why not just stick
with that?  In other words, the goal of this process should be to create
separate projects with their own development lifecycle that will
continue indefinitely, rather than expecting the code itself to merge
into the neutron repo.

This has advantages in simplifying workflow and making it more
consistent.  Plus it builds on known integration mechanisms like APIs
and python project versions.

But more importantly, it helps scale the neutron project itself.  I
think that a focused neutron core upon which projects like these can
build on in a reliable fashion would be ideal.


Despite replies to you saying that certain branches of Neutron 
development work are special unicorns, I wanted to say I *fully* support 
your above statement.


Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

>Flag is only for admin use, tenant can't see it, and the default policy
for router is setup by config file.

It's still a public API that will have to follow a deprecation cycle. If a
new API was going to be introduced for admins to control the distributed
nature of routers, it would have been nice to introduce a little more
control than distributed=True/False (e.g. how many SNAT IPs are consumed,
etc). At this point we are stuck because any new API entries would require
a blueprint and the feature proposal freeze deadline is long gone.

>It COULD be compatible for DVR work on vlan, but there were some bugs
several months before, that DVR mac are not written successfully on the
egress packet, I'm not sure if it is fixed in the merged code.

The current DVR wiki[1] still shows that the tenant_network_type has to be
VXLAN. I didn't know about this issue until just recently and I'm not sure
if there are plans yet to fix it for Juno. If it were in an incubation tree
somewhere and not on Gerrit for the majority of the cycle, the bug could
have been worked on in parallel by other volunteers interested in VLAN
support. Now that DVR is solidifying, VLAN support may even require a
blueprint unless it's blessed by the right cores, which would mean people
with VLAN deployments would not be able to use DVR until Kilo is released
next next May.

The whole point is that there is nowhere to work on big features like this
in a fast, iterative, and open manner. The incubator could be a perfect
place for this type of work.


1. https://wiki.openstack.org/wiki/Neutron/DVR/HowTo


On Wed, Aug 27, 2014 at 1:22 AM, loy wolfe  wrote:

>
>
>
> On Wed, Aug 27, 2014 at 2:44 PM, Kevin Benton  wrote:
>
>> >Incubator doesn't mean being kicked out of tree, it just mean that the
>> API and resource model needs to be baked for fast iteration, and can't be
>> put in tree temporarily.
>>
>> That was exactly my point about developing a major feature like DVR. Even
>> with a limited API change (the new distributed flag), it has an impact on
>> the the router/agent DB resource model and currently isn't compatible
>> with VLAN based ML2 deployments. It's not exactly a hidden optimization
>> like an improvement to some RPC handling code.
>>
>>
> Flag is only for admin use, tenant can't see it, and the default policy
> for router is setup by config file.
>
> It COULD be compatible for DVR work on vlan, but there were some bugs
> several months before, that DVR mac are not written successfully on the
> egress packet, I'm not sure if it is fixed in the merged code.
>
>
>> A huge piece of the DVR development had to happen in Neutron forks and
>> 40+ revision chains of Gerrit patches. It was very difficult to follow
>> without being heavily involved with the L3 team. This would have been a
>> great candidate to develop in the incubator if it existed at the time. It
>> would have been easier to try as it was developed and to explore the entire
>> codebase. Also, more people could have been contributing bug fixes and
>> improvements since an entire section of code wouldn't be 'owned' by one
>> person like it is with the author of a Gerrit review.
>>
>> >For DVR, as it has no influence on tenant facing API resource model, it
>> works as the built-in backend, and this feature has accepted wide common
>> interests,
>>
>> As was pointed out before, common interest has nothing to do with
>> incubation. Incubation is to rapidly iterate on a new feature for Neutron.
>> It shouldn't be restricted to API changes, it should be used for any major
>> new features that are possible to develop outside of the Neutron core. If
>> we are going to have this new incubator tool, we should use it to the
>> fullest extent possible.
>>
>
>>
>> On Tue, Aug 26, 2014 at 6:19 PM, loy wolfe  wrote:
>>
>>> Incubator doesn't mean being kicked out of tree, it just mean that the
>>> API and resource model needs to be baked for fast iteration, and can't be
>>> put in tree temporarily. As kyle has said, incubator is not talking about
>>> moving 3rd drivers out of tree, which is in another thread.
>>>
>>> For DVR, as it has no influence on tenant facing API resource model, it
>>> works as the built-in backend, and this feature has accepted wide common
>>> interests, it's just the internal performance optimization tightly coupled
>>> with existing code, so it should be developed in tree.
>>>
>>>
>>> On Wed, Aug 27, 2014 at 8:08 AM, Kevin Benton  wrote:
>>>
 From what I understand, the intended projects for the incubator can't
 operate without neutron because they are just extensions/plugins/drivers.

 For example, if the DVR modifications to the reference reference L3
 plugin weren't already being developed in the tree, DVR could have been
 developed in the incubator and then merged into Neutron once the bugs were
 ironed out so a huge string of Gerrit patches didn't need to be tracked. If
 that had happened, would it make sense to keep the L3 plugin as 

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[loy wolfe]

On Wed, Aug 27, 2014 at 2:44 PM, Kevin Benton  wrote:

> >Incubator doesn't mean being kicked out of tree, it just mean that the
> API and resource model needs to be baked for fast iteration, and can't be
> put in tree temporarily.
>
> That was exactly my point about developing a major feature like DVR. Even
> with a limited API change (the new distributed flag), it has an impact on
> the the router/agent DB resource model and currently isn't compatible
> with VLAN based ML2 deployments. It's not exactly a hidden optimization
> like an improvement to some RPC handling code.
>
>
Flag is only for admin use, tenant can't see it, and the default policy for
router is setup by config file.

It COULD be compatible for DVR work on vlan, but there were some bugs
several months before, that DVR mac are not written successfully on the
egress packet, I'm not sure if it is fixed in the merged code.


> A huge piece of the DVR development had to happen in Neutron forks and 40+
> revision chains of Gerrit patches. It was very difficult to follow without
> being heavily involved with the L3 team. This would have been a
> great candidate to develop in the incubator if it existed at the time. It
> would have been easier to try as it was developed and to explore the entire
> codebase. Also, more people could have been contributing bug fixes and
> improvements since an entire section of code wouldn't be 'owned' by one
> person like it is with the author of a Gerrit review.
>
> >For DVR, as it has no influence on tenant facing API resource model, it
> works as the built-in backend, and this feature has accepted wide common
> interests,
>
> As was pointed out before, common interest has nothing to do with
> incubation. Incubation is to rapidly iterate on a new feature for Neutron.
> It shouldn't be restricted to API changes, it should be used for any major
> new features that are possible to develop outside of the Neutron core. If
> we are going to have this new incubator tool, we should use it to the
> fullest extent possible.
>

>
> On Tue, Aug 26, 2014 at 6:19 PM, loy wolfe  wrote:
>
>> Incubator doesn't mean being kicked out of tree, it just mean that the
>> API and resource model needs to be baked for fast iteration, and can't be
>> put in tree temporarily. As kyle has said, incubator is not talking about
>> moving 3rd drivers out of tree, which is in another thread.
>>
>> For DVR, as it has no influence on tenant facing API resource model, it
>> works as the built-in backend, and this feature has accepted wide common
>> interests, it's just the internal performance optimization tightly coupled
>> with existing code, so it should be developed in tree.
>>
>>
>> On Wed, Aug 27, 2014 at 8:08 AM, Kevin Benton  wrote:
>>
>>> From what I understand, the intended projects for the incubator can't
>>> operate without neutron because they are just extensions/plugins/drivers.
>>>
>>> For example, if the DVR modifications to the reference reference L3
>>> plugin weren't already being developed in the tree, DVR could have been
>>> developed in the incubator and then merged into Neutron once the bugs were
>>> ironed out so a huge string of Gerrit patches didn't need to be tracked. If
>>> that had happened, would it make sense to keep the L3 plugin as a
>>> completely separate project or merge it? I understand this is the approach
>>> the load balancer folks took by making Octavia a separate project, but I
>>> think it can still operate on its own, where the reference L3 plugin (and
>>> many of the other incubator projects) are just classes that expect to be
>>> able to make core Neutron calls.
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Kevin Benton
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

>Incubator doesn't mean being kicked out of tree, it just mean that the API
and resource model needs to be baked for fast iteration, and can't be put
in tree temporarily.

That was exactly my point about developing a major feature like DVR. Even
with a limited API change (the new distributed flag), it has an impact on
the the router/agent DB resource model and currently isn't compatible with
VLAN based ML2 deployments. It's not exactly a hidden optimization like an
improvement to some RPC handling code.

A huge piece of the DVR development had to happen in Neutron forks and 40+
revision chains of Gerrit patches. It was very difficult to follow without
being heavily involved with the L3 team. This would have been a
great candidate to develop in the incubator if it existed at the time. It
would have been easier to try as it was developed and to explore the entire
codebase. Also, more people could have been contributing bug fixes and
improvements since an entire section of code wouldn't be 'owned' by one
person like it is with the author of a Gerrit review.

>For DVR, as it has no influence on tenant facing API resource model, it
works as the built-in backend, and this feature has accepted wide common
interests,

As was pointed out before, common interest has nothing to do with
incubation. Incubation is to rapidly iterate on a new feature for Neutron.
It shouldn't be restricted to API changes, it should be used for any major
new features that are possible to develop outside of the Neutron core. If
we are going to have this new incubator tool, we should use it to the
fullest extent possible.


On Tue, Aug 26, 2014 at 6:19 PM, loy wolfe  wrote:

> Incubator doesn't mean being kicked out of tree, it just mean that the API
> and resource model needs to be baked for fast iteration, and can't be put
> in tree temporarily. As kyle has said, incubator is not talking about
> moving 3rd drivers out of tree, which is in another thread.
>
> For DVR, as it has no influence on tenant facing API resource model, it
> works as the built-in backend, and this feature has accepted wide common
> interests, it's just the internal performance optimization tightly coupled
> with existing code, so it should be developed in tree.
>
>
> On Wed, Aug 27, 2014 at 8:08 AM, Kevin Benton  wrote:
>
>> From what I understand, the intended projects for the incubator can't
>> operate without neutron because they are just extensions/plugins/drivers.
>>
>> For example, if the DVR modifications to the reference reference L3
>> plugin weren't already being developed in the tree, DVR could have been
>> developed in the incubator and then merged into Neutron once the bugs were
>> ironed out so a huge string of Gerrit patches didn't need to be tracked. If
>> that had happened, would it make sense to keep the L3 plugin as a
>> completely separate project or merge it? I understand this is the approach
>> the load balancer folks took by making Octavia a separate project, but I
>> think it can still operate on its own, where the reference L3 plugin (and
>> many of the other incubator projects) are just classes that expect to be
>> able to make core Neutron calls.
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[loy wolfe]

Incubator doesn't mean being kicked out of tree, it just mean that the API
and resource model needs to be baked for fast iteration, and can't be put
in tree temporarily. As kyle has said, incubator is not talking about
moving 3rd drivers out of tree, which is in another thread.

For DVR, as it has no influence on tenant facing API resource model, it
works as the built-in backend, and this feature has accepted wide common
interests, it's just the internal performance optimization tightly coupled
with existing code, so it should be developed in tree.


On Wed, Aug 27, 2014 at 8:08 AM, Kevin Benton  wrote:

> From what I understand, the intended projects for the incubator can't
> operate without neutron because they are just extensions/plugins/drivers.
>
> For example, if the DVR modifications to the reference reference L3 plugin
> weren't already being developed in the tree, DVR could have been developed
> in the incubator and then merged into Neutron once the bugs were ironed out
> so a huge string of Gerrit patches didn't need to be tracked. If that had
> happened, would it make sense to keep the L3 plugin as a completely
> separate project or merge it? I understand this is the approach the load
> balancer folks took by making Octavia a separate project, but I think it
> can still operate on its own, where the reference L3 plugin (and many of
> the other incubator projects) are just classes that expect to be able to
> make core Neutron calls.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow

[Kevin Benton]

>From what I understand, the intended projects for the incubator can't
operate without neutron because they are just extensions/plugins/drivers.

For example, if the DVR modifications to the reference reference L3 plugin
weren't already being developed in the tree, DVR could have been developed
in the incubator and then merged into Neutron once the bugs were ironed out
so a huge string of Gerrit patches didn't need to be tracked. If that had
happened, would it make sense to keep the L3 plugin as a completely
separate project or merge it? I understand this is the approach the load
balancer folks took by making Octavia a separate project, but I think it
can still operate on its own, where the reference L3 plugin (and many of
the other incubator projects) are just classes that expect to be able to
make core Neutron calls.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev