On 25 Feb 2014, at 02:21, W Chan <m4d.co...@gmail.com> wrote:

> Renat,
> Regarding your comments on change https://review.openstack.org/#/c/75609/, I 
> don't think the port to oslo.messaging is just a swap from pika to 
> oslo.messaging.  OpenStack services as I understand is usually implemented as 
> an RPC client/server over a messaging transport.  Sync vs async calls are 
> done via the RPC client call and cast respectively.  The messaging transport 
> is abstracted and concrete implementation is done via drivers/plugins.  So 
> the architecture of the executor if ported to oslo.messaging needs to include 
> a client, a server, and a transport.  The consumer (in this case the mistral 
> engine) instantiates an instance of the client for the executor, makes the 
> method call to handle task, the client then sends the request over the 
> transport to the server.  The server picks up the request from the exchange 
> and processes the request.  If cast (async), the client side returns 
> immediately.  If call (sync), the client side waits for a response from the 
> server over a reply_q (a unique queue for the session in the transport).  
> Also, oslo.messaging allows versioning in the message. Major version change 
> indicates API contract changes.  Minor version indicates backend changes but 
> with API compatibility.  

My main concern about this patch is not related with messaging infrastructure. 
I believe you know better than me how it should look like. I’m mostly concerned 
with the way of making changes you chose. From my perspective, it’s much better 
to make atomic changes where every changes doesn’t affect too much in existing 
architecture. So the first step could be to change pika to oslo.messaging with 
minimal structural changes without introducing versioning (could be just TODO 
comment saying that the framework allows it and we may want to use it in the 
future, to be decide), without getting rid of the current engine structure 
(local, scalable). Some of the things in the file structure and architecture 
came from the decisions made by many people and we need to be careful about 
changing them.

> So, where I'm headed with this change...  I'm implementing the basic 
> structure/scaffolding for the new executor service using oslo.messaging 
> (default transport with rabbit).  Since the whole change will take a few 
> rounds, I don't want to disrupt any changes that the team is making at the 
> moment and so I'm building the structure separately.  I'm also adding 
> versioning (v1) in the module structure to anticipate any versioning changes 
> in the future.   I expect the change request will lead to some discussion as 
> we are doing here.  I will migrate the core operations of the executor 
> (handle_task, handle_task_error, do_task_action) to the server component when 
> we agree on the architecture and switch the consumer (engine) to use the new 
> RPC client for the executor instead of sending the message to the queue over 
> pika.  Also, the launcher for ./mistral/cmd/task_executor.py will change as 
> well in subsequent round.  An example launcher is here 
> https://github.com/uhobawuhot/interceptor/blob/master/bin/interceptor-engine. 
>  The interceptor project here is what I use to research how oslo.messaging 
> works.  I hope this is clear. The blueprint only changes how the request and 
> response are being transported.  It shouldn't change how the executor 
> currently works.

Please create a document describing the approach you’re pursuing here. I would 
expect to see the main goals you want to achieve upon completion.

> Finally, can you clarify the difference between local vs scalable engine?  I 
> personally do not prefer to explicitly name the engine scalable because this 
> requirement should be in the engine by default and we do not need to 
> explicitly state/separate that.  But if this is a roadblock for the change, I 
> can put the scalable structure back in the change to move this forward.

Separation for local and scalable implementations appeared for historical 
reasons because from the beginning we didn’t see how it all would look like and 
hence we tried different approaches to implement the engine. At some point we 
got 2 working versions: the one that didn’t distribute anything (local) and 
another one that could distribute tasks over task executors via asynchronous HA 
transport (scalable). Later on we decided to leave them both since scalable is 
needed by the requirements and local might be useful for demonstration purposes 
and testing since it doesn’t require RabbitMQ to be installed. So we decided to 
refactor both and make them work similarly except the way they run tasks.


Renat Akhmerov
@Mirantis Inc.
OpenStack-dev mailing list

Reply via email to