Re: [openstack-dev] [nova] config options help text improvement: current status

[Markus Zoeller]

If you have open changes in Gerrit for this task, please be aware
that the commit message must be updated to the Newton blueprint [1]:

Implements blueprint centralize-config-options-newton

New changes should use this line in the commit message too. This was
necessary to keep a clean track record of the work done in Mitaka.
I'll find two IRC meeting slots (Europe/Asia and US friendly) shortly
after the Mitaka release where we can discuss open question. I will
also attend the Newton summit, in case you want to reach out there.
The next weeks until the release are focused on creating a stable
and reliable product and preparation for the summit, so I have less
time to review these changes. But you can always contact me in IRC.

References:
[1] 
https://blueprints.launchpad.net/nova/+spec/centralize-config-options-newton

Regards, Markus Zoeller (markus_z)

Markus Zoeller/Germany/IBM@IBMDE wrote on 03/02/2016 06:45:45 PM:

> From: Markus Zoeller/Germany/IBM@IBMDE
> To: "OpenStack Development Mailing List" 
<openstack-dev@lists.openstack.org>
> Date: 03/02/2016 06:47 PM
> Subject: [openstack-dev] [nova] config options help text improvement: 
> current status
> 
> TL;DR: From ~600 nova specific config options are:
> ~140 at a central location with an improved help text
> ~220 options in open reviews (currently on hold)
> ~240 options todo
> 
> 
> Background
> ==
> Nova has a lot of config options. Most of them weren't well
> documented and without looking in the code you probably don't
> understand what they do. That's fine for us developers but the ops
> had more problems with the interface we provide for them [1]. After
> the Mitaka summit we came to the conclusion that this should be 
> improved, which is currently in progress with blueprint [2].
> 
> 
> Current Status
> ==
> After asking on the ML for help [3] the progress improved a lot. 
> The goal is clear now and we know how to achieve it. The organization 
> is done via [4] which also has a section of "odd config options". 
> This section is important for a later step when we want do deprecate 
> config options to get rid of unnecessary ones. 
> 
> As we reached the Mitaka-3 milestone we decided to put the effort [5] 
> on hold to stabilize the project and focus the review effort on bug 
> fixes. When the Newton cycle opens, we can continue the work. The 
> current result can be seen in the sample "nova.conf" file generated 
> after each commit [6]. The appendix at the end of this post shows an
> example.
> 
> All options we have will be treated that way and moved to a central
> location at "nova/conf/". That's the central location which hosts
> now the interface to the ops. It's easier to get an overview now.
> The appendix shows how the config options were spread at the beginning
> and how they are located now.
> 
> I initially thought that we have around 800 config options in Nova
> but I learned meanwhile that we import a lot from other libs, for 
> example from "oslo.db" and expose them as Nova options. We have around
> 600 Nova specific config options, and ~140 are already treaded like
> described above and ca. 220 are in the pipeline of open reviews.
> Which leaves us ~240 which are not looked at yet.
> 
> 
> Outlook
> ===
> The numbers of the beginning of this ML post make me believe that we
> can finish the work in the upcoming Newton cycle. "Finished" means
> here: 
> * all config options we provide to our ops have proper and usable docs
> * we have an understanding which options don't make sense anymore
> * we know which options should get stronger validation to reduce errors
> 
> I'm looking forward to it :)
> 
> 
> Thanks
> ==
> I'd like to thank all the people who are working on this and making
> this possible. A special thanks goes to Ed Leafe, Esra Celik and
> Stephen Finucane. They put a tremendous amount of work in it.
> 
> 
> References:
> ===
> [1] 
> http://lists.openstack.org/pipermail/openstack-operators/2016-January/
> 009301.html
> [2] 
https://blueprints.launchpad.net/nova/+spec/centralize-config-options
> [3] 
> 
http://lists.openstack.org/pipermail/openstack-dev/2015-December/081271.html

> [4] https://etherpad.openstack.org/p/config-options
> [5] Gerrit reviews for this topic: 
> https://review.openstack.org/#/q/status:open+project:openstack/nova
> +branch:master+topic:bp/centralize-config-options
> [6] The sample config file which gets generated after each commit:
> http://docs.openstack.org/developer/nova/sample_config.html
> 
> 
> Appendix
> 
> 
> Example of the help text improvement
> 

Re: [openstack-dev] [nova] config options help text improvement: current status

[Esra Celik]

> I'm looking forward to it :)

> Thanks
> ==
> I'd like to thank all the people who are working on this and making
> this possible. A special thanks goes to Ed Leafe, Esra Celik and
> Stephen Finucane. They put a tremendous amount of work in it.

Markus, it was a pleasure for me to work with you, you are extremely helpful at 
every stage. 
PS: I am also looking forward to commit some more work on this as soon as the 
Newton cycle opens :) 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Doug Hellmann]

Excerpts from Markus Zoeller's message of 2016-03-03 12:50:55 +0100:
> Doug Hellmann <d...@doughellmann.com> wrote on 03/02/2016 07:19:22 PM:
> 
> > From: Doug Hellmann <d...@doughellmann.com>
> > To: openstack-dev <openstack-dev@lists.openstack.org>
> > Date: 03/02/2016 07:20 PM
> > Subject: Re: [openstack-dev] [nova] config options help text 
> > improvement: current status
> > 
> > Excerpts from Markus Zoeller's message of 2016-03-02 18:45:45 +0100:
> > 
> > [a lot snipped]
> > 
> > > Appendix
> > > 
> > > 
> > > Example of the help text improvement
> > > ---
> > > As an example, compare the previous documentation of the scheduler 
> > > option "scheduler_tracks_instance_changes". 
> > > Before we started:
> > > 
> > > # Determines if the Scheduler tracks changes to instances to help 
> > > # with its filtering decisions. (boolean value)
> > > #scheduler_tracks_instance_changes = true
> > > 
> > > After the improvement:
> > > 
> > > # The scheduler may need information about the instances on a host 
> 
> > > # in order to evaluate its filters and weighers. The most common 
> > > # need for this information is for the (anti-)affinity filters, 
> > > # which need to choose a host based on the instances already 
> running
> > > # on a host.
> > > #
> > > # If the configured filters and weighers do not need this 
> information,
> > > # disabling this option will improve performance. It may also be 
> > > # disabled when the tracking overhead proves too heavy, although 
> > > # this will cause classes requiring host usage data to query the 
> > > # database on each request instead.
> > > #
> > > # This option is only used by the FilterScheduler and its 
> subclasses;
> > > # if you use a different scheduler, this option has no effect.
> > > #
> > > # * Services that use this:
> > > #
> > > # ``nova-scheduler``
> > > #
> > > # * Related options:
> > > #
> > > # None
> > > #  (boolean value)
> > > #scheduler_tracks_instance_changes = true
> > 
> > If, in the course of adding this information, you think it would be
> > useful for oslo.config or the config generator to provide a way to
> > expose or derive the information, let me know. We're going to be doing
> > some more work on the config generator to enable some automation in the
> > config reference guide maintained by the docs team, and this looks like
> > the sort of thing that might be useful to include in a discoverable way
> > (not just within the comment text for the options).
> > 
> > Doug
> 
> I assume you mean the information about the services which use the
> config option. We have indeed the concern that this is prone to be
> outdated easily. I was thinking if it would make sense to do an
> analysis of the module imports based on the nova services starter
> scripts [1] to derive that information but I couldn't dive into that.
> Long story short, if the config generator could derive that, that
> would be awesome.

The config generator can't derive it directly, but we could make it
smart enough to know that if it is given multiple input files it should
list all of the input files that result in a given input option being
loaded, and map that to a service name somehow. You still have to do the
analysis by hand and set up the different input files, though.

Doug

> 
> References:
> [1] https://git.openstack.org/cgit/openstack/nova/tree/nova/cmd
> 
> Regards, Markus Zoeller (markus_z)
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Markus Zoeller]

Markus Zoeller/Germany/IBM@IBMDE wrote on 03/03/2016 11:45:52 AM:

> From: Markus Zoeller/Germany/IBM@IBMDE
> To: "OpenStack Development Mailing List \(not for usage questions\)" 
> <openstack-dev@lists.openstack.org>
> Date: 03/03/2016 11:47 AM
> Subject: Re: [openstack-dev] [nova] config options help text 
> improvement: current status
> 
> Matt Riedemann <mrie...@linux.vnet.ibm.com> wrote on 03/03/2016 03:50:18 

> AM:
> 
> > From: Matt Riedemann <mrie...@linux.vnet.ibm.com>
> > To: openstack-dev@lists.openstack.org
> > Date: 03/03/2016 03:51 AM
> > Subject: Re: [openstack-dev] [nova] config options help text 
> > improvement: current status
> > 
> > 
> > 
> > On 3/2/2016 11:45 AM, Markus Zoeller wrote:
> > > TL;DR: From ~600 nova specific config options are:
> > >  ~140 at a central location with an improved help text
> > >  ~220 options in open reviews (currently on hold)
> > >  ~240 options todo
> > >
> > > [...]
> > >
> > >
> > > Regards, Markus Zoeller (markus_z)
> > >
> > >
> > 
> > Thanks, this looks a lot better.
> > 
> > On a side note, can we get someone to herd the cats around this bug?
> > 
> > https://bugs.launchpad.net/nova/+bug/1532228
> > 
> > There are a few duplicates and at least 3 patches up for the same 
thing, 
> 
> > but all three are incomplete. Basically all of the 
> > nova.virt.libvirt.volume.* options are missing from the generated 
sample 
> 
> > config. It would be nice if someone drove those 3 patches into one and 

> > resolved all of the missing libvirt volume options rather than a 
couple 
> > here and there.
> > 
> > -- 
> > 
> > Thanks,
> > 
> > Matt Riedemann
> 
> I found 3 open patches for that and combined it into [1]. My gerrit 
> query was: 
> project:openstack/nova 
> status:open 
> file:opts.py 
> NOT bp/centralize-config-options 
> Hope this helps.
> 
> References:
> [1] https://review.openstack.org/#/c/287685
> [2] https://youtu.be/Pk7yqlTMvp8
> 
> Regards, Markus Zoeller (markus_z)

Matt, I added a 4th one in the same area to the same patch. 
That patch replaces these 4 reviews:
https://review.openstack.org/#/c/266268/
https://review.openstack.org/#/c/265339/
https://review.openstack.org/#/c/276649/
https://review.openstack.org/#/c/265299/

Regards, Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Markus Zoeller]

Doug Hellmann <d...@doughellmann.com> wrote on 03/02/2016 07:19:22 PM:

> From: Doug Hellmann <d...@doughellmann.com>
> To: openstack-dev <openstack-dev@lists.openstack.org>
> Date: 03/02/2016 07:20 PM
> Subject: Re: [openstack-dev] [nova] config options help text 
> improvement: current status
> 
> Excerpts from Markus Zoeller's message of 2016-03-02 18:45:45 +0100:
> 
> [a lot snipped]
> 
> > Appendix
> > 
> > 
> > Example of the help text improvement
> > ---
> > As an example, compare the previous documentation of the scheduler 
> > option "scheduler_tracks_instance_changes". 
> > Before we started:
> > 
> > # Determines if the Scheduler tracks changes to instances to help 
> > # with its filtering decisions. (boolean value)
> > #scheduler_tracks_instance_changes = true
> > 
> > After the improvement:
> > 
> > # The scheduler may need information about the instances on a host 

> > # in order to evaluate its filters and weighers. The most common 
> > # need for this information is for the (anti-)affinity filters, 
> > # which need to choose a host based on the instances already 
running
> > # on a host.
> > #
> > # If the configured filters and weighers do not need this 
information,
> > # disabling this option will improve performance. It may also be 
> > # disabled when the tracking overhead proves too heavy, although 
> > # this will cause classes requiring host usage data to query the 
> > # database on each request instead.
> > #
> > # This option is only used by the FilterScheduler and its 
subclasses;
> > # if you use a different scheduler, this option has no effect.
> > #
> > # * Services that use this:
> > #
> > # ``nova-scheduler``
> > #
> > # * Related options:
> > #
> > # None
> > #  (boolean value)
> > #scheduler_tracks_instance_changes = true
> 
> If, in the course of adding this information, you think it would be
> useful for oslo.config or the config generator to provide a way to
> expose or derive the information, let me know. We're going to be doing
> some more work on the config generator to enable some automation in the
> config reference guide maintained by the docs team, and this looks like
> the sort of thing that might be useful to include in a discoverable way
> (not just within the comment text for the options).
> 
> Doug

I assume you mean the information about the services which use the
config option. We have indeed the concern that this is prone to be
outdated easily. I was thinking if it would make sense to do an
analysis of the module imports based on the nova services starter
scripts [1] to derive that information but I couldn't dive into that.
Long story short, if the config generator could derive that, that
would be awesome.

References:
[1] https://git.openstack.org/cgit/openstack/nova/tree/nova/cmd

Regards, Markus Zoeller (markus_z)



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Markus Zoeller]

Matt Riedemann <mrie...@linux.vnet.ibm.com> wrote on 03/03/2016 03:50:18 
AM:

> From: Matt Riedemann <mrie...@linux.vnet.ibm.com>
> To: openstack-dev@lists.openstack.org
> Date: 03/03/2016 03:51 AM
> Subject: Re: [openstack-dev] [nova] config options help text 
> improvement: current status
> 
> 
> 
> On 3/2/2016 11:45 AM, Markus Zoeller wrote:
> > TL;DR: From ~600 nova specific config options are:
> >  ~140 at a central location with an improved help text
> >  ~220 options in open reviews (currently on hold)
> >  ~240 options todo
> >
> > [...]
> >
> >
> > Regards, Markus Zoeller (markus_z)
> >
> >
> 
> Thanks, this looks a lot better.
> 
> On a side note, can we get someone to herd the cats around this bug?
> 
> https://bugs.launchpad.net/nova/+bug/1532228
> 
> There are a few duplicates and at least 3 patches up for the same thing, 

> but all three are incomplete. Basically all of the 
> nova.virt.libvirt.volume.* options are missing from the generated sample 

> config. It would be nice if someone drove those 3 patches into one and 
> resolved all of the missing libvirt volume options rather than a couple 
> here and there.
> 
> -- 
> 
> Thanks,
> 
> Matt Riedemann

I found 3 open patches for that and combined it into [1]. My gerrit 
query was: 
project:openstack/nova 
status:open 
file:opts.py 
NOT bp/centralize-config-options 
Hope this helps.

References:
[1] https://review.openstack.org/#/c/287685
[2] https://youtu.be/Pk7yqlTMvp8

Regards, Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Markus Zoeller]

Andreas Jaeger <a...@suse.com> wrote on 03/03/2016 09:31:44 AM:

> From: Andreas Jaeger <a...@suse.com>
> To: "OpenStack Development Mailing List (not for usage questions)" 
> <openstack-dev@lists.openstack.org>
> Date: 03/03/2016 09:43 AM
> Subject: Re: [openstack-dev] [nova] config options help text 
> improvement: current status
> 
> On 2016-03-02 19:11, Tim Bell wrote:
> > 
> > Great. Does this additional improved text also get into the 
> configuration guide documentation somehow ? 
> 
> It should, the reference part of configuration guide is autogenerated.
> 
> The current draft version for Mitaka is at
> 
http://docs.openstack.org/draft/config-reference/compute/config-options.html
,
> it was generated last on the 22nd of February and will be regenerated
> with Mitaka content.
> 
> Andreas

Yes, it's there, I see the change. Unfortunately the line breaks get lost.
I also tried to use restructured text syntax in the hope the manuals could
reuse it. Andreas, do you see a chance for that? For example the option
"base_url" in section "serial_console" [1].

References:
[1] 
https://github.com/openstack/nova/blob/307b74977456735d4dcebc2988421b5980a2a71e/nova/conf/serial_console.py#L73


Regards, Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Andreas Jaeger]

On 2016-03-02 19:11, Tim Bell wrote:
> 
> Great. Does this additional improved text also get into the configuration 
> guide documentation somehow ? 

It should, the reference part of configuration guide is autogenerated.

The current draft version for Mitaka is at
http://docs.openstack.org/draft/config-reference/compute/config-options.html,
it was generated last on the 22nd of February and will be regenerated
with Mitaka content.

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Matt Riedemann]

On 3/2/2016 11:45 AM, Markus Zoeller wrote:

TL;DR: From ~600 nova specific config options are:
 ~140 at a central location with an improved help text
 ~220 options in open reviews (currently on hold)
 ~240 options todo


Background
==
Nova has a lot of config options. Most of them weren't well
documented and without looking in the code you probably don't
understand what they do. That's fine for us developers but the ops
had more problems with the interface we provide for them [1]. After
the Mitaka summit we came to the conclusion that this should be
improved, which is currently in progress with blueprint [2].


Current Status
==
After asking on the ML for help [3] the progress improved a lot.
The goal is clear now and we know how to achieve it. The organization
is done via [4] which also has a section of "odd config options".
This section is important for a later step when we want do deprecate
config options to get rid of unnecessary ones.

As we reached the Mitaka-3 milestone we decided to put the effort [5]
on hold to stabilize the project and focus the review effort on bug
fixes. When the Newton cycle opens, we can continue the work. The
current result can be seen in the sample "nova.conf" file generated
after each commit [6]. The appendix at the end of this post shows an
example.

All options we have will be treated that way and moved to a central
location at "nova/conf/". That's the central location which hosts
now the interface to the ops. It's easier to get an overview now.
The appendix shows how the config options were spread at the beginning
and how they are located now.

I initially thought that we have around 800 config options in Nova
but I learned meanwhile that we import a lot from other libs, for
example from "oslo.db" and expose them as Nova options. We have around
600 Nova specific config options, and ~140 are already treaded like
described above and ca. 220 are in the pipeline of open reviews.
Which leaves us ~240 which are not looked at yet.


Outlook
===
The numbers of the beginning of this ML post make me believe that we
can finish the work in the upcoming Newton cycle. "Finished" means
here:
* all config options we provide to our ops have proper and usable docs
* we have an understanding which options don't make sense anymore
* we know which options should get stronger validation to reduce errors

I'm looking forward to it :)


Thanks
==
I'd like to thank all the people who are working on this and making
this possible. A special thanks goes to Ed Leafe, Esra Celik and
Stephen Finucane. They put a tremendous amount of work in it.


References:
===
[1]
http://lists.openstack.org/pipermail/openstack-operators/2016-January/009301.html
[2] https://blueprints.launchpad.net/nova/+spec/centralize-config-options
[3]
http://lists.openstack.org/pipermail/openstack-dev/2015-December/081271.html
[4] https://etherpad.openstack.org/p/config-options
[5] Gerrit reviews for this topic:
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/centralize-config-options
[6] The sample config file which gets generated after each commit:
 http://docs.openstack.org/developer/nova/sample_config.html


Appendix


Example of the help text improvement
---
As an example, compare the previous documentation of the scheduler
option "scheduler_tracks_instance_changes".
Before we started:

 # Determines if the Scheduler tracks changes to instances to help
 # with its filtering decisions. (boolean value)
 #scheduler_tracks_instance_changes = true

After the improvement:

 # The scheduler may need information about the instances on a host
 # in order to evaluate its filters and weighers. The most common
 # need for this information is for the (anti-)affinity filters,
 # which need to choose a host based on the instances already running
 # on a host.
 #
 # If the configured filters and weighers do not need this information,
 # disabling this option will improve performance. It may also be
 # disabled when the tracking overhead proves too heavy, although
 # this will cause classes requiring host usage data to query the
 # database on each request instead.
 #
 # This option is only used by the FilterScheduler and its subclasses;
 # if you use a different scheduler, this option has no effect.
 #
 # * Services that use this:
 #
 # ``nova-scheduler``
 #
 # * Related options:
 #
 # None
 #  (boolean value)
 #scheduler_tracks_instance_changes = true


The spread of config options in the tree

We started with this in November 2015. It's the Nova project tree and
the numbers behind the package name are the numbers of config options
declared in that package (config options declared in sub-packages are
not accumulated).

 

Re: [openstack-dev] [nova] config options help text improvement: current status

[Rochelle Grober]

Don't quote me on this, but the tool that generates the dev docs is the one the 
docs team for the config ref use to generate that document.

And they have been looped in on the upcoming improvements.

--Rocky

-Original Message-
From: Matthew Treinish [mailto:mtrein...@kortar.org] 
Sent: Wednesday, March 02, 2016 3:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova] config options help text improvement: 
current status

On Thu, Mar 03, 2016 at 10:24:28AM +1100, Tony Breeds wrote:
> On Wed, Mar 02, 2016 at 06:11:47PM +, Tim Bell wrote:
>  
> > Great. Does this additional improved text also get into the configuration 
> > guide documentation somehow ? 
> 
> It's certainly part of tox -egenconfig, I don't know about docs.o.o

The sample config file is generated (doing basically the same thing as the tox
job) for nova's devref:

http://docs.openstack.org/developer/nova/sample_config.html

-Matt Treinish

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Matthew Treinish]

On Thu, Mar 03, 2016 at 10:24:28AM +1100, Tony Breeds wrote:
> On Wed, Mar 02, 2016 at 06:11:47PM +, Tim Bell wrote:
>  
> > Great. Does this additional improved text also get into the configuration 
> > guide documentation somehow ? 
> 
> It's certainly part of tox -egenconfig, I don't know about docs.o.o

The sample config file is generated (doing basically the same thing as the tox
job) for nova's devref:

http://docs.openstack.org/developer/nova/sample_config.html

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Tony Breeds]

On Wed, Mar 02, 2016 at 06:11:47PM +, Tim Bell wrote:
 
> Great. Does this additional improved text also get into the configuration 
> guide documentation somehow ? 

It's certainly part of tox -egenconfig, I don't know about docs.o.o

Tony.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Doug Hellmann]

Excerpts from Markus Zoeller's message of 2016-03-02 18:45:45 +0100:

[a lot snipped]

> Appendix
> 
> 
> Example of the help text improvement
> ---
> As an example, compare the previous documentation of the scheduler 
> option "scheduler_tracks_instance_changes". 
> Before we started:
> 
> # Determines if the Scheduler tracks changes to instances to help 
> # with its filtering decisions. (boolean value)
> #scheduler_tracks_instance_changes = true
> 
> After the improvement:
> 
> # The scheduler may need information about the instances on a host 
> # in order to evaluate its filters and weighers. The most common 
> # need for this information is for the (anti-)affinity filters, 
> # which need to choose a host based on the instances already running
> # on a host.
> #
> # If the configured filters and weighers do not need this information,
> # disabling this option will improve performance. It may also be 
> # disabled when the tracking overhead proves too heavy, although 
> # this will cause classes requiring host usage data to query the 
> # database on each request instead.
> #
> # This option is only used by the FilterScheduler and its subclasses;
> # if you use a different scheduler, this option has no effect.
> #
> # * Services that use this:
> #
> # ``nova-scheduler``
> #
> # * Related options:
> #
> # None
> #  (boolean value)
> #scheduler_tracks_instance_changes = true

If, in the course of adding this information, you think it would be
useful for oslo.config or the config generator to provide a way to
expose or derive the information, let me know. We're going to be doing
some more work on the config generator to enable some automation in the
config reference guide maintained by the docs team, and this looks like
the sort of thing that might be useful to include in a discoverable way
(not just within the comment text for the options).

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] config options help text improvement: current status

[Tim Bell]

Great. Does this additional improved text also get into the configuration guide 
documentation somehow ? 



Tim

On 02/03/16 18:45, "Markus Zoeller"  wrote:

>TL;DR: From ~600 nova specific config options are:
>~140 at a central location with an improved help text
>~220 options in open reviews (currently on hold)
>~240 options todo
>
>
>Background
>==
>Nova has a lot of config options. Most of them weren't well
>documented and without looking in the code you probably don't
>understand what they do. That's fine for us developers but the ops
>had more problems with the interface we provide for them [1]. After
>the Mitaka summit we came to the conclusion that this should be 
>improved, which is currently in progress with blueprint [2].
>
>
>Current Status
>==
>After asking on the ML for help [3] the progress improved a lot. 
>The goal is clear now and we know how to achieve it. The organization 
>is done via [4] which also has a section of "odd config options". 
>This section is important for a later step when we want do deprecate 
>config options to get rid of unnecessary ones. 
>
>As we reached the Mitaka-3 milestone we decided to put the effort [5] 
>on hold to stabilize the project and focus the review effort on bug 
>fixes. When the Newton cycle opens, we can continue the work. The 
>current result can be seen in the sample "nova.conf" file generated 
>after each commit [6]. The appendix at the end of this post shows an
>example.
>
>All options we have will be treated that way and moved to a central
>location at "nova/conf/". That's the central location which hosts
>now the interface to the ops. It's easier to get an overview now.
>The appendix shows how the config options were spread at the beginning
>and how they are located now.
>
>I initially thought that we have around 800 config options in Nova
>but I learned meanwhile that we import a lot from other libs, for 
>example from "oslo.db" and expose them as Nova options. We have around
>600 Nova specific config options, and ~140 are already treaded like
>described above and ca. 220 are in the pipeline of open reviews.
>Which leaves us ~240 which are not looked at yet.
>
>
>Outlook
>===
>The numbers of the beginning of this ML post make me believe that we
>can finish the work in the upcoming Newton cycle. "Finished" means
>here: 
>* all config options we provide to our ops have proper and usable docs
>* we have an understanding which options don't make sense anymore
>* we know which options should get stronger validation to reduce errors
>
>I'm looking forward to it :)
>
>
>Thanks
>==
>I'd like to thank all the people who are working on this and making
>this possible. A special thanks goes to Ed Leafe, Esra Celik and
>Stephen Finucane. They put a tremendous amount of work in it.
>
>
>References:
>===
>[1] 
>http://lists.openstack.org/pipermail/openstack-operators/2016-January/009301.html
>[2] https://blueprints.launchpad.net/nova/+spec/centralize-config-options
>[3] 
>http://lists.openstack.org/pipermail/openstack-dev/2015-December/081271.html
>[4] https://etherpad.openstack.org/p/config-options
>[5] Gerrit reviews for this topic: 
>https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/centralize-config-options
>[6] The sample config file which gets generated after each commit:
>http://docs.openstack.org/developer/nova/sample_config.html
>
>
>Appendix
>
>
>Example of the help text improvement
>---
>As an example, compare the previous documentation of the scheduler 
>option "scheduler_tracks_instance_changes". 
>Before we started:
>
># Determines if the Scheduler tracks changes to instances to help 
># with its filtering decisions. (boolean value)
>#scheduler_tracks_instance_changes = true
>
>After the improvement:
>
># The scheduler may need information about the instances on a host 
># in order to evaluate its filters and weighers. The most common 
># need for this information is for the (anti-)affinity filters, 
># which need to choose a host based on the instances already running
># on a host.
>#
># If the configured filters and weighers do not need this information,
># disabling this option will improve performance. It may also be 
># disabled when the tracking overhead proves too heavy, although 
># this will cause classes requiring host usage data to query the 
># database on each request instead.
>#
># This option is only used by the FilterScheduler and its subclasses;
># if you use a different scheduler, this option has no effect.
>#
># * Services that use this:
>#
># ``nova-scheduler``
>#
># * Related options:
>#
># None
>#  (boolean value)
>#scheduler_tracks_instance_changes = true
>
>
>The spread of config options in the tree

[openstack-dev] [nova] config options help text improvement: current status

[Markus Zoeller]

TL;DR: From ~600 nova specific config options are:
~140 at a central location with an improved help text
~220 options in open reviews (currently on hold)
~240 options todo


Background
==
Nova has a lot of config options. Most of them weren't well
documented and without looking in the code you probably don't
understand what they do. That's fine for us developers but the ops
had more problems with the interface we provide for them [1]. After
the Mitaka summit we came to the conclusion that this should be 
improved, which is currently in progress with blueprint [2].


Current Status
==
After asking on the ML for help [3] the progress improved a lot. 
The goal is clear now and we know how to achieve it. The organization 
is done via [4] which also has a section of "odd config options". 
This section is important for a later step when we want do deprecate 
config options to get rid of unnecessary ones. 

As we reached the Mitaka-3 milestone we decided to put the effort [5] 
on hold to stabilize the project and focus the review effort on bug 
fixes. When the Newton cycle opens, we can continue the work. The 
current result can be seen in the sample "nova.conf" file generated 
after each commit [6]. The appendix at the end of this post shows an
example.

All options we have will be treated that way and moved to a central
location at "nova/conf/". That's the central location which hosts
now the interface to the ops. It's easier to get an overview now.
The appendix shows how the config options were spread at the beginning
and how they are located now.

I initially thought that we have around 800 config options in Nova
but I learned meanwhile that we import a lot from other libs, for 
example from "oslo.db" and expose them as Nova options. We have around
600 Nova specific config options, and ~140 are already treaded like
described above and ca. 220 are in the pipeline of open reviews.
Which leaves us ~240 which are not looked at yet.


Outlook
===
The numbers of the beginning of this ML post make me believe that we
can finish the work in the upcoming Newton cycle. "Finished" means
here: 
* all config options we provide to our ops have proper and usable docs
* we have an understanding which options don't make sense anymore
* we know which options should get stronger validation to reduce errors

I'm looking forward to it :)


Thanks
==
I'd like to thank all the people who are working on this and making
this possible. A special thanks goes to Ed Leafe, Esra Celik and
Stephen Finucane. They put a tremendous amount of work in it.


References:
===
[1] 
http://lists.openstack.org/pipermail/openstack-operators/2016-January/009301.html
[2] https://blueprints.launchpad.net/nova/+spec/centralize-config-options
[3] 
http://lists.openstack.org/pipermail/openstack-dev/2015-December/081271.html
[4] https://etherpad.openstack.org/p/config-options
[5] Gerrit reviews for this topic: 
https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/centralize-config-options
[6] The sample config file which gets generated after each commit:
http://docs.openstack.org/developer/nova/sample_config.html


Appendix


Example of the help text improvement
---
As an example, compare the previous documentation of the scheduler 
option "scheduler_tracks_instance_changes". 
Before we started:

# Determines if the Scheduler tracks changes to instances to help 
# with its filtering decisions. (boolean value)
#scheduler_tracks_instance_changes = true

After the improvement:

# The scheduler may need information about the instances on a host 
# in order to evaluate its filters and weighers. The most common 
# need for this information is for the (anti-)affinity filters, 
# which need to choose a host based on the instances already running
# on a host.
#
# If the configured filters and weighers do not need this information,
# disabling this option will improve performance. It may also be 
# disabled when the tracking overhead proves too heavy, although 
# this will cause classes requiring host usage data to query the 
# database on each request instead.
#
# This option is only used by the FilterScheduler and its subclasses;
# if you use a different scheduler, this option has no effect.
#
# * Services that use this:
#
# ``nova-scheduler``
#
# * Related options:
#
# None
#  (boolean value)
#scheduler_tracks_instance_changes = true


The spread of config options in the tree

We started with this in November 2015. It's the Nova project tree and 
the numbers behind the package name are the numbers of config options
declared in that package (config options declared in sub-packages are
not accumulated).

Based on:
commit 201090b0bcb 
Date: Thu Nov 19