Excerpts from Joshua Harlow's message of 2015-11-04 14:13:33 -0800:
> Agreed, it'd be nice to have an 'audit' of the various projects configs 
> and try to categorize which ones should be reloadable (and the priority 
> to make it reloadable) and which ones are service discovery configs (and 
> probably shouldn't be in config in the first place) and which ones are 
> nice to haves to be configurable... (like rpc_response_timeout).
> 
> The side-effects of making a few things configurable will likely cause a 
> whole bunch of other issues anyway (like how an application/library... 
> gets notified that a config has changed and it may need to re-adjust 
> itself to those new values which may include for example unloading a 
> driver, stopping a thread, starting a new driver...), so I'm thinking we 
> should start small and grow as needed (based on above prioritization).
> 
> Log levels are high on my known list.

Right, anything that is going to take significant application rework to
support should wait. The session identified a few relatively simple
options that help with debugging when they are changed, including
logging, and we should start with those.

Doug

> 
> Fox, Kevin M wrote:
> > Ah, I hadn't considered the rabbit_hosts (discovery) case. yeah, that would 
> > be a useful thing to be able to tweak live. I haven't needed that feature 
> > yet, but could see how that flexibility could come in handy.
> >
> > Thanks,
> > Kevin
> > ________________________________________
> > From: Joshua Harlow [harlo...@fastmail.com]
> > Sent: Wednesday, November 04, 2015 11:34 AM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [oslo][oslo.config][oslo.service] Dynamic 
> > Reconfiguration of OpenStack Services
> >
> > Along this line, thinks like the following are likely more changeable
> > (and my guess is operators would want to change them when things start
> > going badly), for example from a nova.conf that I have laying around...
> >
> > [DEFAULT]
> >
> > rabbit_hosts=...
> > rpc_response_timeout=...
> > default_notification_level=...
> > default_log_levels=...
> >
> > [glance]
> >
> > api_servers=...
> >
> > (and more)
> >
> > Some of those I think should have higher priority as being
> > reconfigurable, but I think operators should be asked what they think
> > would be useful and prioritize those.
> >
> > Some of those really are service discovery 'types' (rabbit_hosts,
> > glance/api_servers, keystone/api_servers) but fixing this is likely a
> > longer term goal (see conversations in keystone).
> >
> > Joshua Harlow wrote:
> >> gord chung wrote:
> >>> we actually had a solution implemented in Ceilometer to handle this[1].
> >>>
> >>> that said, based on the results of our survey[2], we found that most
> >>> operators *never* update configuration files after the initial setup and
> >>> if they did it was very rarely (monthly updates). the question related
> >>> to Ceilometer and its pipeline configuration file so the results might
> >>> be specific to Ceilometer. I think you should definitely query operators
> >>> before undertaking any work. the last thing you want to do is implement
> >>> a feature no one really needs/wants.
> >>>
> >>> [1]
> >>> http://specs.openstack.org/openstack/ceilometer-specs/specs/liberty/reload-file-based-pipeline-configuration.html
> >>>
> >>> [2]
> >>> http://lists.openstack.org/pipermail/openstack-dev/2015-September/075628.html
> >>>
> >> So my general though on the above is yes, definitely consult operators
> >> to see if they would use this, although if a feature doesn't exist and
> >> has never existed (say outside of ceilometer) then it's sort of hard to
> >> get an accurate survey result from a group of people that have never had
> >> the feature in the first place... Either way it should be done, just to
> >> get more knowledge...
> >>
> >> I know operators (at yahoo!) want to be able to dynamically change the
> >> logging level, and that's not a monthly task, but more of an 'as-needed'
> >> one that would be very helpful when things start going badly... So
> >> perhaps the set of reloadable configuration should start out small and
> >> not encompass all the things...
> >>
> >>> On 04/11/2015 10:00 AM, Marian Horban wrote:
> >>>> Hi guys,
> >>>>
> >>>> Unfortunately I haven't been on Tokio summit but I know that there was
> >>>> discussion about dynamic reloading of configuration.
> >>>> Etherpad refs:
> >>>> https://etherpad.openstack.org/p/mitaka-cross-project-dynamic-config-services,
> >>>>
> >>>>
> >>>> https://etherpad.openstack.org/p/mitaka-oslo-security-logging
> >>>>
> >>>> In this thread I want to discuss agreements reached on the summit and
> >>>> discuss
> >>>> implementation details.
> >>>>
> >>>> Some notes taken from etherpad and my remarks:
> >>>>
> >>>> 1. "Adding "mutable" parameter for each option."
> >>>> "Do we have an option mutable=True on CfgOpt? Yes"
> >>>> ---------------------------------------------------------
> >>>> As I understood 'mutable' parameter must indicate whether service
> >>>> contains
> >>>> code responsible for reloading of this option or not.
> >>>> And this parameter should be one of the arguments of cfg.Opt
> >>>> constructor.
> >>>> Problems:
> >>>> 1. Library's options.
> >>>> SSL options ca_file, cert_file, key_file taken from oslo.service library
> >>>> could be reloaded in nova-api so these options should be mutable...
> >>>> But for some projects that don't need SSL support reloading of SSL
> >>>> options
> >>>> doesn't make sense. For such projects this option should be non mutable.
> >>>> Problem is that oslo.service - single and there are many different
> >>>> projects
> >>>> which use it in different way.
> >>>> The same options could be mutable and non mutable in different contexts.
> >>>> 2. Support of config options on some platforms.
> >>>> Parameter "mutable" could be different for different platforms. Some
> >>>> options
> >>>> make sense only for specific platforms. If we mark such options as
> >>>> mutable
> >>>> it could be misleading on some platforms.
> >>>> 3. Dependency of options.
> >>>> There are many 'workers' options(osapi_compute_workers, ec2_workers,
> >>>> metadata_workers, workers). These options specify number of workers for
> >>>> OpenStack API services.
> >>>> If value of the 'workers' option is greater than '1' instance of
> >>>> ProcessLauncher is created otherwise instance of ServiceLauncher is
> >>>> created.
> >>>> When ProcessLauncher receives SIGHUP it reloads it own configuration,
> >>>> gracefully terminates children and respawns new children.
> >>>> This mechanism allows to reload many config options implicitly.
> >>>> But if value of the 'workers' option equals '1' instance of
> >>>> ServiceLauncher
> >>>> is created.
> >>>> ServiceLauncher starts everything in single process and in this case we
> >>>> don't have such implicit reloading.
> >>>>
> >>>> I think that mutability of options is a complicated feature and I
> >>>> think that
> >>>> adding of 'mutable' parameter into cfg.Opt constructor could just add
> >>>> mess.
> >>>>
> >>>> 2. "oslo.service catches SIGHUP and calls oslo.config"
> >>>> ---------------------------------------------------------
> >>>>  From my point of view every service should register list of hooks to
> >>>> reload
> >>>> config options. oslo.service should catch SIGHUP and call list of
> >>>> registered
> >>>> hooks one by one with specified order.
> >>>> Discussion of such implementation was started in ML:
> >>>> http://lists.openstack.org/pipermail/openstack-dev/2015-September/074558.html.
> >>>>
> >>>> Raw reviews:
> >>>> https://review.openstack.org/#/c/228892/,
> >>>> https://review.openstack.org/#/c/223668/.
> >>>>
> >>>> 3. "oslo.config is responsible to log changes which were ignored on
> >>>> SIGHUP"
> >>>> ---------------------------------------------------------
> >>>> Some config options could be changed using API(for example quotas)
> >>>> that's why
> >>>> oslo.config doesn't know actual configuration of service and can't log
> >>>> changes of configuration.
> >>>>
> >>>> Regards, Marian Horban
> >>>>
> >>>>
> >>>> __________________________________________________________________________
> >>>>
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >>>>
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>> --
> >>> gord
> >>>
> >>> __________________________________________________________________________
> >>>
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to