Looks good. Thanks, Winson! Renat, What do you think?
On Wed, Feb 26, 2014 at 10:00 AM, W Chan <[email protected]> wrote: > The following link is the google doc of the proposed engine/executor > message flow architecture. > https://drive.google.com/file/d/0B4TqA9lkW12PZ2dJVFRsS0pGdEU/edit?usp=sharing > > The diagram on the right is the scalable engine where one or more engine > sends requests over a transport to one or more executors. The executor > client, transport, and executor server follows the RPC client/server > design > pattern<https://github.com/openstack/oslo.messaging/tree/master/oslo/messaging/rpc>in > oslo.messaging. > > The diagram represents the local engine. In reality, it's following the > same RPC client/server design pattern. The only difference is that it'll > be configured to use a > fake<https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/impl_fake.py>RPC > backend driver. The fake driver uses in process > queues <http://docs.python.org/2/library/queue.html#module-Queue> shared > between a pair of engine and executor. > > The following are the stepwise changes I will make. > 1) Keep the local and scalable engine structure intact. Create the > Executor Client at ./mistral/engine/scalable/executor/client.py. Create > the Executor Server at ./mistral/engine/scalable/executor/service.py and > implement the task operations under > ./mistral/engine/scalable/executor/executor.py. Delete > ./mistral/engine/scalable/executor/executor.py. Modify the launcher > ./mistral/cmd/task_executor.py. Modify ./mistral/engine/scalable/engine.py > to use the Executor Client instead of sending the message directly to > rabbit via pika. The sum of this is the atomic change that keeps existing > structure and without breaking the code. > 2) Remove the local engine. > https://blueprints.launchpad.net/mistral/+spec/mistral-inproc-executor > 3) Implement versioning for the engine. > https://blueprints.launchpad.net/mistral/+spec/mistral-engine-versioning > 4) Port abstract engine to use oslo.messaging and implement the engine > client, engine server, and modify the API layer to consume the engine > client. > https://blueprints.launchpad.net/mistral/+spec/mistral-engine-standalone-process > . > > Winson > > > On Mon, Feb 24, 2014 at 8:07 PM, Renat Akhmerov <[email protected]>wrote: > >> >> On 25 Feb 2014, at 02:21, W Chan <[email protected]> wrote: >> >> Renat, >> >> Regarding your comments on change https://review.openstack.org/#/c/75609/, >> I don't think the port to oslo.messaging is just a swap from pika to >> oslo.messaging. OpenStack services as I understand is usually implemented >> as an RPC client/server over a messaging transport. Sync vs async calls >> are done via the RPC client call and cast respectively. The messaging >> transport is abstracted and concrete implementation is done via >> drivers/plugins. So the architecture of the executor if ported to >> oslo.messaging needs to include a client, a server, and a transport. The >> consumer (in this case the mistral engine) instantiates an instance of the >> client for the executor, makes the method call to handle task, the client >> then sends the request over the transport to the server. The server picks >> up the request from the exchange and processes the request. If cast >> (async), the client side returns immediately. If call (sync), the client >> side waits for a response from the server over a reply_q (a unique queue >> for the session in the transport). Also, oslo.messaging allows versioning >> in the message. Major version change indicates API contract changes. Minor >> version indicates backend changes but with API compatibility. >> >> >> My main concern about this patch is not related with messaging >> infrastructure. I believe you know better than me how it should look like. >> I'm mostly concerned with the way of making changes you chose. From my >> perspective, it's much better to make atomic changes where every changes >> doesn't affect too much in existing architecture. So the first step could >> be to change pika to oslo.messaging with minimal structural changes without >> introducing versioning (could be just TODO comment saying that the >> framework allows it and we may want to use it in the future, to be decide), >> without getting rid of the current engine structure (local, scalable). Some >> of the things in the file structure and architecture came from the >> decisions made by many people and we need to be careful about changing them. >> >> >> So, where I'm headed with this change... I'm implementing the basic >> structure/scaffolding for the new executor service using oslo.messaging >> (default transport with rabbit). Since the whole change will take a few >> rounds, I don't want to disrupt any changes that the team is making at the >> moment and so I'm building the structure separately. I'm also adding >> versioning (v1) in the module structure to anticipate any versioning >> changes in the future. I expect the change request will lead to some >> discussion as we are doing here. I will migrate the core operations of the >> executor (handle_task, handle_task_error, do_task_action) to the server >> component when we agree on the architecture and switch the consumer >> (engine) to use the new RPC client for the executor instead of sending the >> message to the queue over pika. Also, the launcher for >> ./mistral/cmd/task_executor.py will change as well in subsequent round. An >> example launcher is here >> https://github.com/uhobawuhot/interceptor/blob/master/bin/interceptor-engine. >> The interceptor project here is what I use to research how oslo.messaging >> works. I hope this is clear. The blueprint only changes how the request >> and response are being transported. It shouldn't change how the executor >> currently works. >> >> >> Please create a document describing the approach you're pursuing here. I >> would expect to see the main goals you want to achieve upon completion. >> >> Finally, can you clarify the difference between local vs scalable engine? >> I personally do not prefer to explicitly name the engine scalable because >> this requirement should be in the engine by default and we do not need to >> explicitly state/separate that. But if this is a roadblock for the change, >> I can put the scalable structure back in the change to move this forward. >> >> >> Separation for local and scalable implementations appeared for historical >> reasons because from the beginning we didn't see how it all would look like >> and hence we tried different approaches to implement the engine. At some >> point we got 2 working versions: the one that didn't distribute anything >> (local) and another one that could distribute tasks over task executors via >> asynchronous HA transport (scalable). Later on we decided to leave them >> both since scalable is needed by the requirements and local might be useful >> for demonstration purposes and testing since it doesn't require RabbitMQ to >> be installed. So we decided to refactor both and make them work similarly >> except the way they run tasks. >> >> Thanks. >> >> Renat Akhmerov >> @Mirantis Inc. >> >> _______________________________________________ >> OpenStack-dev mailing list >> [email protected] >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Best Regards, Nikolay
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
