[openstack-dev] [oslo][oslo.config] Reloading configuration of service
> Excerpts from Josh's message: >> So a few 'event' like constructs/libraries that I know about: >> >> http://docs.openstack.org/developer/taskflow/types.html#taskflow.types.notifier.Notifier >> >> >> I'd be happy to extract that and move to somewhere else if needed, it >> provides basic event/pub/sub kind of activities for taskflow (in-memory, >> not over rpc...) I've investigated several event libraries...And chose taskflow because first of all it fits all our requirements and it is already used in openstack. > Excerpts from Doug's message >> We probably want the ability to have multiple callbacks. There are >> already a lot of libraries available on PyPI for handling "events" like >> this, so maybe we can pick one of those that is well maintained and >> integrate it with oslo.service? I've created raw review in oslo.service https://review.openstack.org/#/c/228892/ . I've used taskflow library(as Josh proposed). By default I added one handler that reloads global configuration. What do you think about such implementation? Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo][oslo.config] Reloading configuration of service
Hi Doug, > Rather than building hooks into oslo.config, why don't we build them > into the thing that is catching the signal. That way the app can do lots > of things in response to a signal, and one of them might be reloading > the configuration. Hm... Yes... It is really stupid idea to put reloading hook into oslo.config. I'll move that hook mechanism into oslo.service. oslo.service is responsible for catching/handling signals. Is it enough to have one callback function? Or should I must add ability to register many different callback functions? What is your point of view? Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo][oslo.config] Reloading configuration of service
Hi Josh, > Sounds like a useful idea if projects can plug-in themselves into the > reloading process. I definitely think there needs to be a way for > services to plug-in to this, although I'm not quite sure it will be > sufficient at the current time though. > > An example of why: > > - > https://github.com/openstack/cinder/blob/stable/kilo/cinder/volume/__init__.py#L24 > (unless this module is purged from python and reloaded it will likely > not reload correctly). > > Likely these can all be easily fixed (I just don't know how many of > those exist in the various projects); but I guess we have to start > somewhere so getting the underlying code able to be reloaded is a first > step of likely many. Each of openstack component should contain code responsible for reloading such objects. What objects will be reloaded? It depends of inspire and desire of developers/users. Writing such code is a second step. Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo][oslo.config] Reloading configuration of service
Hi guys, I would like to talk about reloading config during reloading service. Now we have ability to reload config of service with SIGHUP signal. Right now SIGHUP causes just calling conf.reload_config_files(). As result configuration is updated, but services don't know about it, there is no way to notify them. I've created review https://review.openstack.org/#/c/213062/ to allow to execute service's code on reloading config event. Possible usage can be https://review.openstack.org/#/c/223668/. Any ideas or suggestions __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack][oslo.service] Manage of openstack services by ProcessLauncher
Most of openstack services use ProcessLauncher(located in oslo.services) to run services, fork new worker processes, reload configuration, etc. Initialization of services in master process usually contains opening of sockets, so that socket will be inherited in child processes. Then master(parent) process spawns(with fork call) children. Communication between master process and children is implemented with signals, for example when master process wants to shutdown children it sends SIGTERM signal to children, to reload children config master process sends SIGHUP signal. I would like to discuss three things: 1. Behaviour of reloading configuration in children processes. 2. Additional way to control of master process. 3. Communication between master and children processes. 1. Behaviour of reloading configuration in children processes. Now we can reload processes by sending SIGHUP to master process. Master process reloads its own configuration and sends SIGHUP signal to children. When child process receives SIGHUP signal it loads config file, stops and starts service. During stopping-starting service new config options usually don't applied because there should be written a lot of code to manage cofiguration changes. rpodolyaka expressed idea to shutdown children during reloading configuration and respawn new workers. This approach frees us of implementing a huge amount of service-related code for reloading configuration of specific objects. Apache and NGINX uses the same reloading approach: gracefully stop children and start new child instances. 2. Additional way to control of master process. Right now we can control ProcessLauncher by sending signals to master process. It is the only way to manage it. The problem is that such approach is not platform independent. We already faced an issue: Windows doesn't support SIGHUP signal, so part of functionality is inaccessible in Windows :(. Usually process containers like ProcessLauncher could be managed by CLI too. What do you think about creating listening interface for incoming commands? 3. Сommunication between master and children processes. Master process uses signals to control children. Since many signals are not supported on some platforms I suggest to replace signal mechanism with pipes. Each of children will listen to input commands on one side of pipe and master process will write commands on the other side of pipe. Any idea? __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack][nova] Streamlining of config options in nova
Hi all, During development process in nova I faced with an issue related with config options. Now we have lists of config options and registering options mixed with source code in regular files. From one side it can be convenient: to have module-encapsulated config options. But problems appear when we need to use some config option in different modules/packages. If some option is registered in module X and module X imports module Y for some reasons... and in one day we need to import this option in module Y we will get exception NoSuchOptError on import_opt in module Y. Because of circular dependency. To resolve it we can move registering of this option in Y module(in the inappropriate place) or use other tricks. I offer to create file options.py in each package and move all package's config options and registration code there. Such approach allows us to import any option in any place of nova without problems. Implementations of this refactoring can be done piece by piece where piece is one package. What is your opinion about this idea? Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] apache wsgi application support
On Tue, 16 Jun 2015, Sean Dague wrote: I'd expect nova to be running on http://localhost/compute not http://localhost:8774 when running under wsgi. That's going to probably interestingly break a lot of weird assumptions by different projects, but that's part of the reason for doing this exercise. Things should be using the service catalog, and when they aren't, we need to figure it out. Absolutely agree. I think that we need to get these things sorted before any further progression here. Volunteers welcomed to help get us there. You can count on me. Marian __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
