Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[John Perkins]

I've refreshed the agent config patch [1] with the needed-by/depends-on
using the example patch in Dragonflow [2].

[1] https://review.openstack.org/#/c/343045/
[2] https://review.openstack.org/#/c/391853/


On Fri, Oct 28, 2016 at 4:25 PM, Armando M.  wrote:

>
>
> On 27 October 2016 at 22:18, Brandon Logan 
> wrote:
>
>> Hello Neutrinos,
>> I've come across an issue that I'd like to get input/opinions on.  I've
>> been reviewing some of the centralize config options reviews and have
>> come across a few that would cause issues with other projects that are
>> importing these options, especially stadium projects.  High level view
>> of the issue:
>>
>> [1] would cause at least 22 projects to need to be fixed based on [2]
>>
>> [3] would cause at least 12 projects to need to be fixed based on [4]
>>
>> [5] looks to affect many other projects as well (I'm being lazy and
>> not  counting them right now)
>>
>> Initially, the thinking was that moving the config options around would
>> cause some breakage with projects outside of neutron, but that would be
>> fine because projects shouldn't really be using neutron as a library
>> and using it to register config options.  However, with these 3
>> patches, I definitely don't feel comfortable breaking the amount of
>> projects these would break.  It also makes me think that maybe these
>> options should be in neutron-lib since they're consumed so widely.
>> Anyway, I've come up with some possible options to deal with this, but
>> would like to hear others' opinions on this:
>>
>> 1) Let the patches merge and break those projects as a signal that
>> importing these shouldn't be done.  The affected projects can choose to
>> push fixes that continue importing the neutron config options or
>> defining their own config options.
>> 2) Deprecate the old locations for some timeframe, and then remove
>> later.
>> 3) Texas Three-Step: change the neutron patches to keep pointers in the
>> old locations to the new, and then push patches to the affected repos
>> with Depends-On directives.  Once all patches merge, push up one more
>> patch to neutron to remove the old location.
>> 4) Abandon these reviews and do nothing.
>> 5) Move these config options to neutron-lib so that they can be used by
>> any project.  This still requires doing one of the above options,
>> however.
>
> 6) Any others I can't think of?
>>
>
> Slight variation, call it option 6:
>
> 1) Identify the most impacted (coupled) project affected by these changes.
> 2) Fix it in order to provide folks with a recipe for how to address the
> breakages.
> 3) Use the patch and make it needed-by the neutron changes.
> 4) Evangelize patch 2 one more time on the ML.
> 5) We'll bring this up at the team meeting, for another form or record.
> 6) Wait another few days for projects to catch up.
> 7) Merge the patch in neutron.
> 8) We all move on.
>
>
>>
>>
>> [1] https://review.openstack.org/#/c/343045/
>> [2] http://codesearch.openstack.org/?q=from%20neutron.agent.common%20im
>> port%20config=nope==
>> 
>>
>> [3] https://review.openstack.org/#/c/340228/
>> [4] http://codesearch.openstack.org/?q=neutron.plugins.ml2%20import%20c
>> onfig=nope==
>> 
>>
>> [5] https://review.openstack.org/#/c/347867/
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Doug Hellmann]

Excerpts from Brandon Logan's message of 2016-11-01 22:56:45 +:
> On Tue, 2016-11-01 at 13:13 +0100, Ihar Hrachyshka wrote:
> > Brandon Logan  wrote:
> > 
> > > Hello Neutrinos,
> > > I've come across an issue that I'd like to get input/opinions
> > > on.  I've
> > > been reviewing some of the centralize config options reviews and
> > > have
> > > come across a few that would cause issues with other projects that
> > > are
> > > importing these options, especially stadium projects.  High level
> > > view
> > > of the issue:
> > > 
> > > [1] would cause at least 22 projects to need to be fixed based on
> > > [2]
> > > [3] would cause at least 12 projects to need to be fixed based on
> > > [4]
> > > 
> > > [5] looks to affect many other projects as well (I'm being lazy and
> > > not  counting them right now)
> > > 
> > > Initially, the thinking was that moving the config options around
> > > would
> > > cause some breakage with projects outside of neutron, but that
> > > would be
> > > fine because projects shouldn't really be using neutron as a
> > > library
> > > and using it to register config options.  However, with these 3
> > > patches, I definitely don't feel comfortable breaking the amount of
> > > projects these would break.  It also makes me think that maybe
> > > these
> > > options should be in neutron-lib since they're consumed so widely.
> > 
> > Definitely not neutron-lib material (unless carefully hidden behind
> > clearly  
> > public API).
> > 
> > There is a reason why oslo folks explicitly deny any support for  
> > configuration option names and locations their libraries expose
> > [1].  
> > Options are for operators to change in configuration files, but not
> > to  
> > access them or set programmatically. If there are options that
> > subprojects  
> > need access to, we should expose them via explicitly public API, like
> > we  
> > did with global_physnet_mtu [2].
> > 
> > [1]  
> > http://docs.openstack.org/developer/oslo.config/faq.html#why-are-conf
> > iguration-options-not-part-of-a-library-s-api
> > [2]  
> > https://github.com/openstack/neutron/blob/181bdb374fc0c944b1168f27ac7
> > b5cbb0ff0f3c3/neutron/plugins/common/utils.py#L43
> 
> Yeah allowing the options to be imported directly from code outside the
> repo doesn't make sense.  When you talk about a public API in neutron-
> lib for these options, are only talking about READ access as the
> example you gave? OR are you also talking about being able to register

We have seen patterns including an API for reading values and an API for
setting the defaults for values (useful for an application to override
the defaults set in a library).

> these options as well via functions that require no access to the
> options?  If that is the case, then these centralize config option

Options should be registered at runtime by the code that uses them. We
still have lots of code registering options when modules are imported,
but that's not the preferred way to do it because there are potentially
execution order issues when services start up. It's safe to register the
same option more than once, so doing it in some code that will use the
option is fine. For example, when constructing an instance of a class
that uses configuration options, the constructor can register the
options and then the other methods of the class know the options are
present and can be read.

> patches are basically doing that, except not in neutron-lib.  Do you
> think we should move these to neutron-lib instead?  This would mean the
> config options themselves would then probably end up living in neutron-
> lib (though I guess they wouldn't have to).  We'll still have to figure
> out what to do with the subprojects though, but having them in neutron-
> lib and neutron at the same time during a transition period might make
> this easier.

It could also lead to options being registered multiple times with
slightly different settings, which will throw errors. That just means
you need to make sure you don't change the help text or any other
attributions of the options while there are 2 copies.

> 
> > 
> > > Anyway, I've come up with some possible options to deal with this,
> > > but
> > > would like to hear others' opinions on this:
> > > 
> > > 1) Let the patches merge and break those projects as a signal that
> > > importing these shouldn't be done.  The affected projects can
> > > choose to
> > > push fixes that continue importing the neutron config options or
> > > defining their own config options.
> > > 2) Deprecate the old locations for some timeframe, and then remove
> > > later.
> > > 3) Texas Three-Step: change the neutron patches to keep pointers in
> > > the
> > > old locations to the new, and then push patches to the affected
> > > repos
> > > with Depends-On directives.  Once all patches merge, push up one
> > > more
> > > patch to neutron to remove the old location.
> > > 4) Abandon these reviews and do nothing.
> > > 5) 

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Brandon Logan]

On Tue, 2016-11-01 at 13:17 +0100, Ihar Hrachyshka wrote:
> Armando M.  wrote:
> > 
> > Slight variation, call it option 6:
> > 
> > 1) Identify the most impacted (coupled) project affected by these
> > changes.
> > 2) Fix it in order to provide folks with a recipe for how to
> > address the  
> > breakages.
> 
> ^ the step depends on how far we want to go with the adoption. If
> it’s just  
> changing import paths, then it will be easy but will give wrong
> signal to  
> subprojects that the whole practice of them piggy-backing on
> neutron  
> options is acceptable. On the other hand, we could help them get off
> the  
> hook, by working in scope of neutron-lib to expose whatever they
> *really*  
> need (and switch off neutron options completely where there is no
> clear  
> semantic requirement for the option value to be shared).
> 
> While the latter path is not as easy, I would prefer it any time. It
> is in  
> line with the neutron-lib effort goal to decouple stadium subprojects
> from  
> neutron code base.

Yeah this is pretty much the real question of this thread.  How much
impact to subprojects is acceptable in getting to the 100% decoupled
state?

> 
> > 3) Use the patch and make it needed-by the neutron changes.
> > 4) Evangelize patch 2 one more time on the ML.
> > 5) We'll bring this up at the team meeting, for another form or
> > record.
> > 6) Wait another few days for projects to catch up.
> > 7) Merge the patch in neutron.
> > 8) We all move on.
> 
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Brandon Logan]

On Tue, 2016-11-01 at 13:13 +0100, Ihar Hrachyshka wrote:
> Brandon Logan  wrote:
> 
> > Hello Neutrinos,
> > I've come across an issue that I'd like to get input/opinions
> > on.  I've
> > been reviewing some of the centralize config options reviews and
> > have
> > come across a few that would cause issues with other projects that
> > are
> > importing these options, especially stadium projects.  High level
> > view
> > of the issue:
> > 
> > [1] would cause at least 22 projects to need to be fixed based on
> > [2]
> > [3] would cause at least 12 projects to need to be fixed based on
> > [4]
> > 
> > [5] looks to affect many other projects as well (I'm being lazy and
> > not  counting them right now)
> > 
> > Initially, the thinking was that moving the config options around
> > would
> > cause some breakage with projects outside of neutron, but that
> > would be
> > fine because projects shouldn't really be using neutron as a
> > library
> > and using it to register config options.  However, with these 3
> > patches, I definitely don't feel comfortable breaking the amount of
> > projects these would break.  It also makes me think that maybe
> > these
> > options should be in neutron-lib since they're consumed so widely.
> 
> Definitely not neutron-lib material (unless carefully hidden behind
> clearly  
> public API).
> 
> There is a reason why oslo folks explicitly deny any support for  
> configuration option names and locations their libraries expose
> [1].  
> Options are for operators to change in configuration files, but not
> to  
> access them or set programmatically. If there are options that
> subprojects  
> need access to, we should expose them via explicitly public API, like
> we  
> did with global_physnet_mtu [2].
> 
> [1]  
> http://docs.openstack.org/developer/oslo.config/faq.html#why-are-conf
> iguration-options-not-part-of-a-library-s-api
> [2]  
> https://github.com/openstack/neutron/blob/181bdb374fc0c944b1168f27ac7
> b5cbb0ff0f3c3/neutron/plugins/common/utils.py#L43

Yeah allowing the options to be imported directly from code outside the
repo doesn't make sense.  When you talk about a public API in neutron-
lib for these options, are only talking about READ access as the
example you gave? OR are you also talking about being able to register
these options as well via functions that require no access to the
options?  If that is the case, then these centralize config option
patches are basically doing that, except not in neutron-lib.  Do you
think we should move these to neutron-lib instead?  This would mean the
config options themselves would then probably end up living in neutron-
lib (though I guess they wouldn't have to).  We'll still have to figure
out what to do with the subprojects though, but having them in neutron-
lib and neutron at the same time during a transition period might make
this easier.

> 
> > Anyway, I've come up with some possible options to deal with this,
> > but
> > would like to hear others' opinions on this:
> > 
> > 1) Let the patches merge and break those projects as a signal that
> > importing these shouldn't be done.  The affected projects can
> > choose to
> > push fixes that continue importing the neutron config options or
> > defining their own config options.
> > 2) Deprecate the old locations for some timeframe, and then remove
> > later.
> > 3) Texas Three-Step: change the neutron patches to keep pointers in
> > the
> > old locations to the new, and then push patches to the affected
> > repos
> > with Depends-On directives.  Once all patches merge, push up one
> > more
> > patch to neutron to remove the old location.
> > 4) Abandon these reviews and do nothing.
> > 5) Move these config options to neutron-lib so that they can be
> > used by
> > any project.  This still requires doing one of the above options,
> > however.
> > 6) Any others I can't think of?
> 
> Whatever makes subprojects stop accessing neutron configuration
> options  
> programmatically. If we indeed identify something that plugins MUST
> have  
> access to, then let’s work in scope of neutron-lib to expose those
> options  
> through public API (not CONF object!)
> 
> I believe it's subprojects who should identify and vouch for and
> propose  
> neutron-lib patches to expose values they may need in their
> extensions;  
> Neutron should of course give a notice about its plans; we could
> broadly  
> target those changes to early Pike if we think it’s worth a
> postponement,  
> and start talking to subprojects about them driving readiness for
> the  
> changes during Ocata.

Makes sense.

> 
> Ihar
> 
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Ihar Hrachyshka]

Nate Johnston  wrote:


On Tue, Nov 01, 2016 at 01:17:25PM +0100, Ihar Hrachyshka wrote:

Armando M.  wrote:

Slight variation, call it option 6:

1) Identify the most impacted (coupled) project affected by these  
changes.

2) Fix it in order to provide folks with a recipe for how to address the
breakages.


^ the step depends on how far we want to go with the adoption. If it’s  
just

changing import paths, then it will be easy but will give wrong signal to
subprojects that the whole practice of them piggy-backing on neutron  
options

is acceptable. On the other hand, we could help them get off the hook, by
working in scope of neutron-lib to expose whatever they *really* need (and
switch off neutron options completely where there is no clear semantic
requirement for the option value to be shared).

While the latter path is not as easy, I would prefer it any time. It is in
line with the neutron-lib effort goal to decouple stadium subprojects from
neutron code base.


Looking at neutron-fwaas as an example, it looks like there are two  
significant

things that fwaas pulls from neutron config:

1.) FWaaS needs to know the L3 agent's agent_mode to differentiate the  
settings

necessary depending on whether the L3 agent is running DVR or not, from
neutron.conf.agent.l3.l3_config.


I believe we can live with maintaining is_dvr_enabled() function in  
neutron-lib.



2.) In unit tests use of load_paste_app to instantiate a WSGI app instance,
from neutron.common.config.

I'll think about the best way to handle the first case; the second is
basically a two line convenience function that can be trivially copied  
in-tree.


For the second one, some may even argue that if you need to run full API,  
it’s not a unit test anymore, and hence those test cases are better to live  
in corresponding -api/-scenario jobs.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Nate Johnston]

On Tue, Nov 01, 2016 at 01:17:25PM +0100, Ihar Hrachyshka wrote:
> Armando M.  wrote:
> >
> >Slight variation, call it option 6:
> >
> >1) Identify the most impacted (coupled) project affected by these changes.
> >2) Fix it in order to provide folks with a recipe for how to address the
> >breakages.
> 
> ^ the step depends on how far we want to go with the adoption. If it’s just
> changing import paths, then it will be easy but will give wrong signal to
> subprojects that the whole practice of them piggy-backing on neutron options
> is acceptable. On the other hand, we could help them get off the hook, by
> working in scope of neutron-lib to expose whatever they *really* need (and
> switch off neutron options completely where there is no clear semantic
> requirement for the option value to be shared).
> 
> While the latter path is not as easy, I would prefer it any time. It is in
> line with the neutron-lib effort goal to decouple stadium subprojects from
> neutron code base.

Looking at neutron-fwaas as an example, it looks like there are two significant
things that fwaas pulls from neutron config:

1.) FWaaS needs to know the L3 agent's agent_mode to differentiate the settings
necessary depending on whether the L3 agent is running DVR or not, from 
neutron.conf.agent.l3.l3_config.
2.) In unit tests use of load_paste_app to instantiate a WSGI app instance,
from neutron.common.config.

I'll think about the best way to handle the first case; the second is
basically a two line convenience function that can be trivially copied in-tree.

--N.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Doug Hellmann]

Excerpts from Ihar Hrachyshka's message of 2016-11-01 13:13:41 +0100:
> Brandon Logan  wrote:
> 
> > Hello Neutrinos,
> > I've come across an issue that I'd like to get input/opinions on.  I've
> > been reviewing some of the centralize config options reviews and have
> > come across a few that would cause issues with other projects that are
> > importing these options, especially stadium projects.  High level view
> > of the issue:
> >
> > [1] would cause at least 22 projects to need to be fixed based on [2]
> > [3] would cause at least 12 projects to need to be fixed based on [4]
> >
> > [5] looks to affect many other projects as well (I'm being lazy and
> > not  counting them right now)
> >
> > Initially, the thinking was that moving the config options around would
> > cause some breakage with projects outside of neutron, but that would be
> > fine because projects shouldn't really be using neutron as a library
> > and using it to register config options.  However, with these 3
> > patches, I definitely don't feel comfortable breaking the amount of
> > projects these would break.  It also makes me think that maybe these
> > options should be in neutron-lib since they're consumed so widely.
> 
> Definitely not neutron-lib material (unless carefully hidden behind clearly  
> public API).
> 
> There is a reason why oslo folks explicitly deny any support for  
> configuration option names and locations their libraries expose [1].  
> Options are for operators to change in configuration files, but not to  
> access them or set programmatically. If there are options that subprojects  
> need access to, we should expose them via explicitly public API, like we  
> did with global_physnet_mtu [2].

+1 to creating a public API for managing the values instead of exposing
the options directly.

Doug

> 
> [1]  
> http://docs.openstack.org/developer/oslo.config/faq.html#why-are-configuration-options-not-part-of-a-library-s-api
> [2]  
> https://github.com/openstack/neutron/blob/181bdb374fc0c944b1168f27ac7b5cbb0ff0f3c3/neutron/plugins/common/utils.py#L43
> 
> > Anyway, I've come up with some possible options to deal with this, but
> > would like to hear others' opinions on this:
> >
> > 1) Let the patches merge and break those projects as a signal that
> > importing these shouldn't be done.  The affected projects can choose to
> > push fixes that continue importing the neutron config options or
> > defining their own config options.
> > 2) Deprecate the old locations for some timeframe, and then remove
> > later.
> > 3) Texas Three-Step: change the neutron patches to keep pointers in the
> > old locations to the new, and then push patches to the affected repos
> > with Depends-On directives.  Once all patches merge, push up one more
> > patch to neutron to remove the old location.
> > 4) Abandon these reviews and do nothing.
> > 5) Move these config options to neutron-lib so that they can be used by
> > any project.  This still requires doing one of the above options,
> > however.
> > 6) Any others I can't think of?
> 
> Whatever makes subprojects stop accessing neutron configuration options  
> programmatically. If we indeed identify something that plugins MUST have  
> access to, then let’s work in scope of neutron-lib to expose those options  
> through public API (not CONF object!)
> 
> I believe it's subprojects who should identify and vouch for and propose  
> neutron-lib patches to expose values they may need in their extensions;  
> Neutron should of course give a notice about its plans; we could broadly  
> target those changes to early Pike if we think it’s worth a postponement,  
> and start talking to subprojects about them driving readiness for the  
> changes during Ocata.
> 
> Ihar
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Ihar Hrachyshka]

Armando M.  wrote:


Slight variation, call it option 6:

1) Identify the most impacted (coupled) project affected by these changes.
2) Fix it in order to provide folks with a recipe for how to address the  
breakages.


^ the step depends on how far we want to go with the adoption. If it’s just  
changing import paths, then it will be easy but will give wrong signal to  
subprojects that the whole practice of them piggy-backing on neutron  
options is acceptable. On the other hand, we could help them get off the  
hook, by working in scope of neutron-lib to expose whatever they *really*  
need (and switch off neutron options completely where there is no clear  
semantic requirement for the option value to be shared).


While the latter path is not as easy, I would prefer it any time. It is in  
line with the neutron-lib effort goal to decouple stadium subprojects from  
neutron code base.



3) Use the patch and make it needed-by the neutron changes.
4) Evangelize patch 2 one more time on the ML.
5) We'll bring this up at the team meeting, for another form or record.
6) Wait another few days for projects to catch up.
7) Merge the patch in neutron.
8) We all move on.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Ihar Hrachyshka]

Brandon Logan  wrote:


Hello Neutrinos,
I've come across an issue that I'd like to get input/opinions on.  I've
been reviewing some of the centralize config options reviews and have
come across a few that would cause issues with other projects that are
importing these options, especially stadium projects.  High level view
of the issue:

[1] would cause at least 22 projects to need to be fixed based on [2]
[3] would cause at least 12 projects to need to be fixed based on [4]

[5] looks to affect many other projects as well (I'm being lazy and
not  counting them right now)

Initially, the thinking was that moving the config options around would
cause some breakage with projects outside of neutron, but that would be
fine because projects shouldn't really be using neutron as a library
and using it to register config options.  However, with these 3
patches, I definitely don't feel comfortable breaking the amount of
projects these would break.  It also makes me think that maybe these
options should be in neutron-lib since they're consumed so widely.


Definitely not neutron-lib material (unless carefully hidden behind clearly  
public API).


There is a reason why oslo folks explicitly deny any support for  
configuration option names and locations their libraries expose [1].  
Options are for operators to change in configuration files, but not to  
access them or set programmatically. If there are options that subprojects  
need access to, we should expose them via explicitly public API, like we  
did with global_physnet_mtu [2].


[1]  
http://docs.openstack.org/developer/oslo.config/faq.html#why-are-configuration-options-not-part-of-a-library-s-api
[2]  
https://github.com/openstack/neutron/blob/181bdb374fc0c944b1168f27ac7b5cbb0ff0f3c3/neutron/plugins/common/utils.py#L43



Anyway, I've come up with some possible options to deal with this, but
would like to hear others' opinions on this:

1) Let the patches merge and break those projects as a signal that
importing these shouldn't be done.  The affected projects can choose to
push fixes that continue importing the neutron config options or
defining their own config options.
2) Deprecate the old locations for some timeframe, and then remove
later.
3) Texas Three-Step: change the neutron patches to keep pointers in the
old locations to the new, and then push patches to the affected repos
with Depends-On directives.  Once all patches merge, push up one more
patch to neutron to remove the old location.
4) Abandon these reviews and do nothing.
5) Move these config options to neutron-lib so that they can be used by
any project.  This still requires doing one of the above options,
however.
6) Any others I can't think of?


Whatever makes subprojects stop accessing neutron configuration options  
programmatically. If we indeed identify something that plugins MUST have  
access to, then let’s work in scope of neutron-lib to expose those options  
through public API (not CONF object!)


I believe it's subprojects who should identify and vouch for and propose  
neutron-lib patches to expose values they may need in their extensions;  
Neutron should of course give a notice about its plans; we could broadly  
target those changes to early Pike if we think it’s worth a postponement,  
and start talking to subprojects about them driving readiness for the  
changes during Ocata.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Armando M.]

On 27 October 2016 at 22:18, Brandon Logan 
wrote:

> Hello Neutrinos,
> I've come across an issue that I'd like to get input/opinions on.  I've
> been reviewing some of the centralize config options reviews and have
> come across a few that would cause issues with other projects that are
> importing these options, especially stadium projects.  High level view
> of the issue:
>
> [1] would cause at least 22 projects to need to be fixed based on [2]
>
> [3] would cause at least 12 projects to need to be fixed based on [4]
>
> [5] looks to affect many other projects as well (I'm being lazy and
> not  counting them right now)
>
> Initially, the thinking was that moving the config options around would
> cause some breakage with projects outside of neutron, but that would be
> fine because projects shouldn't really be using neutron as a library
> and using it to register config options.  However, with these 3
> patches, I definitely don't feel comfortable breaking the amount of
> projects these would break.  It also makes me think that maybe these
> options should be in neutron-lib since they're consumed so widely.
> Anyway, I've come up with some possible options to deal with this, but
> would like to hear others' opinions on this:
>
> 1) Let the patches merge and break those projects as a signal that
> importing these shouldn't be done.  The affected projects can choose to
> push fixes that continue importing the neutron config options or
> defining their own config options.
> 2) Deprecate the old locations for some timeframe, and then remove
> later.
> 3) Texas Three-Step: change the neutron patches to keep pointers in the
> old locations to the new, and then push patches to the affected repos
> with Depends-On directives.  Once all patches merge, push up one more
> patch to neutron to remove the old location.
> 4) Abandon these reviews and do nothing.
> 5) Move these config options to neutron-lib so that they can be used by
> any project.  This still requires doing one of the above options,
> however.

6) Any others I can't think of?
>

Slight variation, call it option 6:

1) Identify the most impacted (coupled) project affected by these changes.
2) Fix it in order to provide folks with a recipe for how to address the
breakages.
3) Use the patch and make it needed-by the neutron changes.
4) Evangelize patch 2 one more time on the ML.
5) We'll bring this up at the team meeting, for another form or record.
6) Wait another few days for projects to catch up.
7) Merge the patch in neutron.
8) We all move on.


>
>
> [1] https://review.openstack.org/#/c/343045/
> [2] http://codesearch.openstack.org/?q=from%20neutron.agent.common%20im
> port%20config=nope==
>
> [3] https://review.openstack.org/#/c/340228/
> [4] http://codesearch.openstack.org/?q=neutron.plugins.ml2%20import%20c
> onfig=nope==
>
> [5] https://review.openstack.org/#/c/347867/
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Brandon Logan]

Yeah most of them are unit tests, but since the particular reviews I
linked are actually moving the module they're importing, once that
review merges it'll break their tests.  So there would still need to be
some decision made on how this is done.  Doing the oslo.config
attribute may make it easier to fix for them, so that is another
option.  

Thanks,
Brandon

On Fri, 2016-10-28 at 15:19 +, Sean M. Collins wrote:
> It appears though that from the code search that it's all just based
> on
> unit tests, and overriding some configuration stuff.
> 
> Since a lot of these unit test classes inherit from
> Ml2PluginV2TestCase,
> and it looks like there is a lot of copy & paste / cargo-cult that
> occurred where the same line was copied across the stadium[1] to just
> change some configuration before running unit tests, maybe we should
> provide an attribute in Ml2PluginV2TestCase to get the oslo.config
> instance so that overrides can be called?
> 
> This is assuming I grok the problem, I've only had one cup of coffee
> so
> far
> 
> [2]: http://codesearch.openstack.org/?q=ml2_config.cfg.CONF.set_overr
> ide=nope==
> 
> 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Sean M. Collins]

It appears though that from the code search that it's all just based on
unit tests, and overriding some configuration stuff.

Since a lot of these unit test classes inherit from Ml2PluginV2TestCase,
and it looks like there is a lot of copy & paste / cargo-cult that
occurred where the same line was copied across the stadium[1] to just
change some configuration before running unit tests, maybe we should
provide an attribute in Ml2PluginV2TestCase to get the oslo.config
instance so that overrides can be called?

This is assuming I grok the problem, I've only had one cup of coffee so
far

[2]: 
http://codesearch.openstack.org/?q=ml2_config.cfg.CONF.set_override=nope==


-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron] Centralizing some config options will break many stadium projects

[Brandon Logan]

Hello Neutrinos,
I've come across an issue that I'd like to get input/opinions on.  I've
been reviewing some of the centralize config options reviews and have
come across a few that would cause issues with other projects that are
importing these options, especially stadium projects.  High level view
of the issue:

[1] would cause at least 22 projects to need to be fixed based on [2]

[3] would cause at least 12 projects to need to be fixed based on [4]

[5] looks to affect many other projects as well (I'm being lazy and
not  counting them right now)

Initially, the thinking was that moving the config options around would
cause some breakage with projects outside of neutron, but that would be
fine because projects shouldn't really be using neutron as a library
and using it to register config options.  However, with these 3
patches, I definitely don't feel comfortable breaking the amount of
projects these would break.  It also makes me think that maybe these
options should be in neutron-lib since they're consumed so widely. 
Anyway, I've come up with some possible options to deal with this, but
would like to hear others' opinions on this:

1) Let the patches merge and break those projects as a signal that
importing these shouldn't be done.  The affected projects can choose to
push fixes that continue importing the neutron config options or
defining their own config options.
2) Deprecate the old locations for some timeframe, and then remove
later.
3) Texas Three-Step: change the neutron patches to keep pointers in the
old locations to the new, and then push patches to the affected repos
with Depends-On directives.  Once all patches merge, push up one more
patch to neutron to remove the old location.
4) Abandon these reviews and do nothing.
5) Move these config options to neutron-lib so that they can be used by
any project.  This still requires doing one of the above options,
however.
6) Any others I can't think of?



[1] https://review.openstack.org/#/c/343045/
[2] http://codesearch.openstack.org/?q=from%20neutron.agent.common%20im
port%20config=nope==

[3] https://review.openstack.org/#/c/340228/
[4] http://codesearch.openstack.org/?q=neutron.plugins.ml2%20import%20c
onfig=nope==

[5] https://review.openstack.org/#/c/347867/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev