- I can write a method in base test to start local executor. I will do that as a separate bp. - After the engine is made standalone, the API will communicate to the engine and the engine to the executor via the oslo.messaging transport. This means that for the "local" option, we need to start all three components (API, engine, and executor) on the same process. If the long term goal as you stated above is to use separate launchers for these components, this means that the API launcher needs to duplicate all the logic to launch the engine and the executor. Hence, my proposal here is to move the logic to launch the components into a common module and either have a single generic launch script that launch specific components based on the CLI options or have separate launch scripts that reference the appropriate launch function from the common module. - The RPC client/server in oslo.messaging do not determine the transport. The transport is determine via oslo.config and then given explicitly to the RPC client/server. https://github.com/stackforge/mistral/blob/master/mistral/engine/scalable/engine.py#L31and https://github.com/stackforge/mistral/blob/master/mistral/cmd/task_executor.py#L63are examples for the client and server respectively. The in process Queue is instantiated within this transport object from the fake driver. For the "local" option, all three components need to share the same transport in order to have the Queue in scope. Thus, we will need some method to have this transport object visible to all three components and hence my proposal to use a global variable and a factory method.
On Tue, Mar 11, 2014 at 10:34 PM, Renat Akhmerov <rakhme...@mirantis.com>wrote: > > On 12 Mar 2014, at 06:37, W Chan <m4d.co...@gmail.com> wrote: > > Here're the proposed changes. > 1) Rewrite the launch script to be more generic which contains option to > launch all components (i.e. API, engine, executor) on the same process but > over separate threads or launch each individually. > > > You mentioned test_executor.py so I think it would make sense first to > refactor the code in there related with acquiring transport and launching > executor. My suggestions are: > > - In test base class (mistral.tests.base.BaseTest) create the new > method *start_local_executor()* that would deal with getting a fake > driver inside and all that stuff. This would be enough for tests where we > need to run engine and check something. start_local_executor() can be just > a part of setUp() method for such tests. > - As for the launch script I have the following thoughts: > - Long-term launch scripts should be different for all API, engine > and executor. Now API and engine start within the same process but it's > just a temporary solution. > - Launch script for engine (which is the same as API's for now) > should have an option *--use-local-executor* to be able to run an > executor along with engine itself within the same process. > > > 2) Move transport to a global variables, similar to global _engine and > then shared by the different component. > > > Not sure why we need it. Can you please explain more detailed here? The > better way would be to initialize engine and executor with transport when > we create them. If our current structure doesn't allow this easily we > should discuss it and change it. > > In mistral.engine.engine.py we now have: > > def load_engine(): > global _engine > module_name = cfg.CONF.engine.engine > module = importutils.import_module(module_name) > _engine = module.get_engine() > > As an option we could have the code that loads engine in engine launch > script (once we decouple it from API process) so that when we call > get_engine() we could pass in all needed configuration parameters like > transport. > > 3) Modified the engine and the executor to use a factory method to get the > global transport > > > If we made a decision on #2 we won't need it. > > > A side note: when we discuss things like that I really miss DI container :) > > Renat Akhmerov > @ Mirantis Inc. > > > > _______________________________________________ > OpenStack-dev mailing list > OpenStackemail@example.com > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
_______________________________________________ OpenStack-dev mailing list OpenStackfirstname.lastname@example.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev