Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
Hi Sahid, There is a work-in-progress work to change how concurrency is handled in openstack processes: Replace eventlet with ??? https://review.openstack.org/164035 The latest updates can be found in etherpads from the OpenStack Vancouver Summit: - https://etherpad.openstack.org/p/liberty-cross-project-managing-concurrency - https://etherpad.openstack.org/p/YVR-oslo-asyncio No choice was made at Vancouver. In short, we should continue to investigate the different options (asyncio/trollius, threads, etc.) to get more information to be able to take a decision. Victor - Original Message - From: Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com To: openstack-dev@lists.openstack.org Sent: Thursday, May 7, 2015 11:34:51 AM Subject: [openstack-dev] [nova][oslo] RPC Asynchronous Communication Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. future = rpcapi.invoke_long_process() ... do something else here ... result = future.get_response() 2. We can use the benefice of all of the work previously done with the Conductor and so by updating the framework Objects and Indirection Api we should take advantage of async operations to the database. MyObject = MyClassObject.get_async() ... do something else here ... MyObject.wait() MyObject.foo = bar MyObject.save_async() ... do something else here ... MyObject.wait() All of this is to illustrate and have to be discussed. I guess the first job needs to come from Oslo Messaging so the question is to know the feeling here and then from Nova since it will be the primary consumer of this feature. https://blueprints.launchpad.net/nova/+spec/asynchronous-communication Thanks, s. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
On Thu, May 21, 2015 at 8:13 AM, Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com wrote: On Fri, May 08, 2015 at 09:13:59AM -0400, Doug Hellmann wrote: Excerpts from Joe Gordon's message of 2015-05-07 17:43:06 -0700: On May 7, 2015 2:37 AM, Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com wrote: Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. Isn't this what the use of green threads (and eventlet) is supposed to solve. Assuming my understanding is correct, and we can fix any issues without adding async oslo.messaging, then adding yet another async pattern seems like a bad thing. The aim is to be not library specific and avoid to add different and custom patterns on the base code each time we want something not blocking. We can let the well experimented community working in olso messaging to maintain that part and as Doug said oslo can use different executors so we can avoid the requirement of a specific library. I see no problem with nova depending on a specific async library. I am not keen on adding yet another column to our support matrix. Yes, this is what the various executors in the messaging library do, including the eventlet-based executor we use by default. Where are you seeing nova block on RPC calls? In Nova we use the indirection api to make call to the database by the conductor through RPCs. By the solution presented we can create async operations to read and write from the database. Olekssi asks me to give an example I will reply to him. s. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
On Fri, May 08, 2015 at 09:13:59AM -0400, Doug Hellmann wrote: Excerpts from Joe Gordon's message of 2015-05-07 17:43:06 -0700: On May 7, 2015 2:37 AM, Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com wrote: Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. Isn't this what the use of green threads (and eventlet) is supposed to solve. Assuming my understanding is correct, and we can fix any issues without adding async oslo.messaging, then adding yet another async pattern seems like a bad thing. The aim is to be not library specific and avoid to add different and custom patterns on the base code each time we want something not blocking. We can let the well experimented community working in olso messaging to maintain that part and as Doug said oslo can use different executors so we can avoid the requirement of a specific library. Yes, this is what the various executors in the messaging library do, including the eventlet-based executor we use by default. Where are you seeing nova block on RPC calls? In Nova we use the indirection api to make call to the database by the conductor through RPCs. By the solution presented we can create async operations to read and write from the database. Olekssi asks me to give an example I will reply to him. s. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
Excerpts from Joe Gordon's message of 2015-05-07 17:43:06 -0700: On May 7, 2015 2:37 AM, Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com wrote: Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. Isn't this what the use of green threads (and eventlet) is supposed to solve. Assuming my understanding is correct, and we can fix any issues without adding async oslo.messaging, then adding yet another async pattern seems like a bad thing. Yes, this is what the various executors in the messaging library do, including the eventlet-based executor we use by default. Where are you seeing nova block on RPC calls? Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][oslo] RPC Asynchronous Communication
Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. future = rpcapi.invoke_long_process() ... do something else here ... result = future.get_response() 2. We can use the benefice of all of the work previously done with the Conductor and so by updating the framework Objects and Indirection Api we should take advantage of async operations to the database. MyObject = MyClassObject.get_async() ... do something else here ... MyObject.wait() MyObject.foo = bar MyObject.save_async() ... do something else here ... MyObject.wait() All of this is to illustrate and have to be discussed. I guess the first job needs to come from Oslo Messaging so the question is to know the feeling here and then from Nova since it will be the primary consumer of this feature. https://blueprints.launchpad.net/nova/+spec/asynchronous-communication Thanks, s. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
On May 7, 2015 2:37 AM, Sahid Orentino Ferdjaoui sahid.ferdja...@redhat.com wrote: Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. Isn't this what the use of green threads (and eventlet) is supposed to solve. Assuming my understanding is correct, and we can fix any issues without adding async oslo.messaging, then adding yet another async pattern seems like a bad thing. future = rpcapi.invoke_long_process() ... do something else here ... result = future.get_response() 2. We can use the benefice of all of the work previously done with the Conductor and so by updating the framework Objects and Indirection Api we should take advantage of async operations to the database. MyObject = MyClassObject.get_async() ... do something else here ... MyObject.wait() MyObject.foo = bar MyObject.save_async() ... do something else here ... MyObject.wait() All of this is to illustrate and have to be discussed. I guess the first job needs to come from Oslo Messaging so the question is to know the feeling here and then from Nova since it will be the primary consumer of this feature. https://blueprints.launchpad.net/nova/+spec/asynchronous-communication Thanks, s. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
Hi, I generally like the idea of async CALL. Is there a place in Nova (or other services) where the new CALL may be applied to see advantage? Thanks, Oleksii Zamiatin 07.05.15 12:34, Sahid Orentino Ferdjaoui пишет: Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. future = rpcapi.invoke_long_process() ... do something else here ... result = future.get_response() 2. We can use the benefice of all of the work previously done with the Conductor and so by updating the framework Objects and Indirection Api we should take advantage of async operations to the database. MyObject = MyClassObject.get_async() ... do something else here ... MyObject.wait() MyObject.foo = bar MyObject.save_async() ... do something else here ... MyObject.wait() All of this is to illustrate and have to be discussed. I guess the first job needs to come from Oslo Messaging so the question is to know the feeling here and then from Nova since it will be the primary consumer of this feature. https://blueprints.launchpad.net/nova/+spec/asynchronous-communication Thanks, s. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova][oslo] RPC Asynchronous Communication
I'd really like this idea, async call will definitely improve overall performance for cloud control system. In nova (and other components) there are some slow tasks which handle resource with long time running, which makes new tasks get a huge delay before getting served, especially for the high concurrent request (e.g. provisioning hundreds VMs) with high delay operation for the resource handling case. To really archive the result of improving system overall performance, I think the biggest challenge is it must makes all operations cross components to be asynchronous in the handling pipeline, a synchronous operation in the call path will makes the workflow still be synchronous, the system still need to wait this synchronous operation to be finish and it will makes delay/waiting keep there, and this kind of synchronous operation are very familiar around the resource handling case for now. thanks, zhiyan On Thu, May 7, 2015 at 6:05 PM, ozamiatin ozamia...@mirantis.com wrote: Hi, I generally like the idea of async CALL. Is there a place in Nova (or other services) where the new CALL may be applied to see advantage? Thanks, Oleksii Zamiatin 07.05.15 12:34, Sahid Orentino Ferdjaoui пишет: Hi, The primary point of this expected discussion around asynchronous communication is to optimize performance by reducing latency. For instance the design used in Nova and probably other projects let able to operate ascynchronous operations from two way. 1. When communicate between inter-services 2. When communicate to the database 1 and 2 are close since they use the same API but I prefer to keep a difference here since the high level layer is not the same. From Oslo Messaging point of view we currently have two methods to invoke an RPC: Cast and Call: The first one is not bloking and will invoke a RPC without to wait any response while the second will block the process and wait for the response. The aim is to add new method which will return without to block the process an object let's call it Future which will provide some basic methods to wait and get a response at any time. The benefice from Nova will comes on a higher level: 1. When communicate between services it will be not necessary to block the process and use this free time to execute some other computations. future = rpcapi.invoke_long_process() ... do something else here ... result = future.get_response() 2. We can use the benefice of all of the work previously done with the Conductor and so by updating the framework Objects and Indirection Api we should take advantage of async operations to the database. MyObject = MyClassObject.get_async() ... do something else here ... MyObject.wait() MyObject.foo = bar MyObject.save_async() ... do something else here ... MyObject.wait() All of this is to illustrate and have to be discussed. I guess the first job needs to come from Oslo Messaging so the question is to know the feeling here and then from Nova since it will be the primary consumer of this feature. https://blueprints.launchpad.net/nova/+spec/asynchronous-communication Thanks, s. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
