Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Both of you, thanks for your insights. Greatly appreciated. Le 04/03/2016 09:25, Cheng, Yingxin a écrit : Hi, First of all, many delayed thanks to Jay Pipes's benchmarking framework, learnt a lot from it : ) Other comments inline. On Friday, March 4, 2016 8:42 AM Jay Pipes wrote: Hi again,

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Hi, First of all, many delayed thanks to Jay Pipes's benchmarking framework, learnt a lot from it : ) Other comments inline. On Friday, March 4, 2016 8:42 AM Jay Pipes wrote: > Hi again, Yingxin, sorry for the delayed response... been traveling. > Comments inline :) > > On 03/01/2016 12:34

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Hi again, Yingxin, sorry for the delayed response... been traveling. Comments inline :) On 03/01/2016 12:34 AM, Cheng, Yingxin wrote: Hi, I have simulated the distributed resource management with the incremental update model based on Jay's benchmarking framework:

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Le 02/03/2016 09:15, Cheng, Yingxin a écrit : On Tuesday, March 1, 2016 7:29 PM, John Garbutt wrote On 1 March 2016 at 08:34, Cheng, Yingxin wrote: Hi, I have simulated the distributed resource management with the incremental update

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On Tuesday, March 1, 2016 7:29 PM, John Garbutt wrote > On 1 March 2016 at 08:34, Cheng, Yingxin wrote: > > Hi, > > > > I have simulated the distributed resource management with the incremental > update model based on Jay's benchmarking

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 1 March 2016 at 08:34, Cheng, Yingxin wrote: > Hi, > > I have simulated the distributed resource management with the incremental > update model based on Jay's benchmarking framework: > https://github.com/cyx1231st/placement-bench/tree/shared-state-demonstration. >

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Hi, I have simulated the distributed resource management with the incremental update model based on Jay's benchmarking framework: https://github.com/cyx1231st/placement-bench/tree/shared-state-demonstration. The complete result lies at http://paste.openstack.org/show/488677/. It's ran by a VM

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/24/2016 05:56 AM, Chris Dent wrote: >> I'd actually also be interested if this has a potential to reduce the >> demand on the message bus. I've been investigating this for a while, >> and I >> found that RabbitMQ will happily consume 5 high end CPU cores on a >> single box >> just to serve

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Sorry for the late reply. On 22 February 2016 at 18:45, John Garbutt wrote: > On 21 February 2016 at 13:51, Cheng, Yingxin wrote: > > On 19 February 2016 at 5:58, John Garbutt wrote: > >> On 17 February 2016 at 17:52, Clint Byrum wrote: > >> > Excerpts

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On Tue, 23 Feb 2016, Clint Byrum wrote: I'd actually also be interested if this has a potential to reduce the demand on the message bus. I've been investigating this for a while, and I found that RabbitMQ will happily consume 5 high end CPU cores on a single box just to serve the needs of 1000

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Le 24/02/2016 01:10, Jay Pipes a écrit : On 02/22/2016 04:23 AM, Sylvain Bauza wrote: I won't argue against performance here. You made a very nice PoC for testing scaling DB writes within a single python process and I trust your findings. While I would be naturally preferring some

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

dnesday, February 24, 2016 8:11 AM > To: Sylvain Bauza <sba...@redhat.com>; OpenStack Development Mailing List > (not for usage questions) <openstack-dev@lists.openstack.org>; Cheng, Yingxin > <yingxin.ch...@intel.com> > Subject: Re: [openstack-dev] [nova] A prototype i

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Excerpts from Jay Pipes's message of 2016-02-23 16:10:46 -0800: > On 02/22/2016 04:23 AM, Sylvain Bauza wrote: > > I won't argue against performance here. You made a very nice PoC for > > testing scaling DB writes within a single python process and I trust > > your findings. While I would be

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/22/2016 04:23 AM, Sylvain Bauza wrote: I won't argue against performance here. You made a very nice PoC for testing scaling DB writes within a single python process and I trust your findings. While I would be naturally preferring some shared-nothing approach that can horizontally scale,

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/23/2016 06:03 PM, Chris Friesen wrote: On 02/21/2016 01:56 PM, Jay Pipes wrote: Yingxin, sorry for the delay in responding to this thread. My comments inline. On 02/17/2016 12:45 AM, Cheng, Yingxin wrote: To better illustrate the differences between shared-state, resource-provider and

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/23/2016 06:12 PM, Chris Friesen wrote: On 02/22/2016 03:23 AM, Sylvain Bauza wrote: See, I don't think that a compute node unable to start a request is an 'abnormal case'. There are many reasons why a request can't be honored by the compute node : - for example, the scheduler doesn't

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/22/2016 03:23 AM, Sylvain Bauza wrote: See, I don't think that a compute node unable to start a request is an 'abnormal case'. There are many reasons why a request can't be honored by the compute node : - for example, the scheduler doesn't own all the compute resources and thus can

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/21/2016 01:56 PM, Jay Pipes wrote: Yingxin, sorry for the delay in responding to this thread. My comments inline. On 02/17/2016 12:45 AM, Cheng, Yingxin wrote: To better illustrate the differences between shared-state, resource-provider and legacy scheduler, I’ve drew 3 simplified

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 21 February 2016 at 13:51, Cheng, Yingxin wrote: > On 19 February 2016 at 5:58, John Garbutt wrote: >> On 17 February 2016 at 17:52, Clint Byrum wrote: >> > Excerpts from Cheng, Yingxin's message of 2016-02-14 21:21:28 -0800: >> Long term, I see a

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

y 15, 2016 9:48 PM *To:* OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> *Subject:* Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler" Le 15/02/2016 10:48, Cheng, Yingxin a écrit :

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On Sun, 21 Feb 2016, Jay Pipes wrote: I don't see how the shared-state scheduler is getting the most accurate resource view. It is only in extreme circumstances that the resource-provider scheduler's view of the resources in a system (all of which is stored without caching in the database)

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

/#/c/271823/ Regards, -Yingxin *From:*Sylvain Bauza [mailto:sba...@redhat.com] *Sent:* Monday, February 15, 2016 9:48 PM *To:* OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> *Subject:* Re: [openstack-dev] [nova] A prototype implementation towa

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 19 February 2016 at 5:58, John Garbutt wrote: > On 17 February 2016 at 17:52, Clint Byrum wrote: > > Excerpts from Cheng, Yingxin's message of 2016-02-14 21:21:28 -0800: > >> Hi, > >> > >> I've uploaded a prototype https://review.openstack.org/#/c/280047/ to > >> testify its

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 17 February 2016 at 17:52, Clint Byrum wrote: > Excerpts from Cheng, Yingxin's message of 2016-02-14 21:21:28 -0800: >> Hi, >> >> I've uploaded a prototype https://review.openstack.org/#/c/280047/ to >> testify its design goals in accuracy, performance, reliability and >>

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/17/2016 06:59 AM, Chris Dent wrote: The advantage of a centralized datastore for that information is that it provides administrative control (e.g. reserving resources for other needs) and visibility. That level of command and control seems to be something people really want

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On 02/15/2016 09:27 AM, Sylvain Bauza wrote: > > > Le 15/02/2016 06:21, Cheng, Yingxin a écrit : >> >> Hi, >> >> >> >> I’ve uploaded a prototype https://review.openstack.org/#/c/280047/ >> to testify its design goals >> in accuracy, performance,

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

> -Original Message- > From: Clint Byrum [mailto:cl...@fewbar.com] > Sent: Thursday, February 18, 2016 1:53 AM > To: openstack-dev <openstack-dev@lists.openstack.org> > Subject: Re: [openstack-dev] [nova] A prototype implementation towards the > "shared state

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Excerpts from Cheng, Yingxin's message of 2016-02-14 21:21:28 -0800: > Hi, > > I've uploaded a prototype https://review.openstack.org/#/c/280047/ to testify > its design goals in accuracy, performance, reliability and compatibility > improvements. It will also be an Austin Summit Session if

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On Wed, 17 February 2016, Sylvain Bauza wrote (sorry, quoting off-context, but I feel it's a side point, not the main discussion) Le 17/02/2016 16:40, Cheng, Yingxin a écrit : IMHO, the authority to allocate resources is not limited by compute nodes, but also include network service, storage

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

(sorry, quoting off-context, but I feel it's a side point, not the main discussion) Le 17/02/2016 16:40, Cheng, Yingxin a écrit : IMHO, the authority to allocate resources is not limited by compute nodes, but also include network service, storage service or all other services which have

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On Wed, 17 February 2016, Sylvain Bauza wrote Le 17/02/2016 12:59, Chris Dent a écrit : On Wed, 17 Feb 2016, Cheng, Yingxin wrote: To better illustrate the differences between shared-state, resource- provider and legacy scheduler, I've drew 3 simplified pictures [1] in emphasizing the location

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Le 17/02/2016 12:59, Chris Dent a écrit : On Wed, 17 Feb 2016, Cheng, Yingxin wrote: To better illustrate the differences between shared-state, resource- provider and legacy scheduler, I've drew 3 simplified pictures [1] in emphasizing the location of resource view, the location of claim and

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

On Wed, 17 Feb 2016, Cheng, Yingxin wrote: To better illustrate the differences between shared-state, resource- provider and legacy scheduler, I've drew 3 simplified pictures [1] in emphasizing the location of resource view, the location of claim and resource consumption, and the resource

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

ng List (not for usage questions) <openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler" Le 15/02/2016 10:48, Cheng, Yingxin a écrit : Thanks Sylvain, 1. The below ideas will be extended

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 02/15/2016 03:27 AM, Sylvain Bauza wrote: > - can we have the feature optional for operators One thing that concerns me is the lesson learned from simply having a compute node's instance information sent and persisted in memory. That was

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

) <openstack-dev@lists.openstack.org> *Subject:* Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler" Le 15/02/2016 06:21, Cheng, Yingxin a écrit : Hi, I’ve uploaded a prototype https://review.openstack.org/#/c/28

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

a...@redhat.com] Sent: Monday, February 15, 2016 5:28 PM To: OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org> Subject: Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler" Le 15/02/2016 06

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Le 15/02/2016 06:21, Cheng, Yingxin a écrit : Hi, I’ve uploaded a prototype https://review.openstack.org/#/c/280047/ to testify its design goals in accuracy, performance, reliability and compatibility improvements. It will also be an Austin

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

ris Pavlovic [mailto:bo...@pavlovic.me] > *Sent:* Monday, February 15, 2016 1:59 PM > *To:* OpenStack Development Mailing List (not for usage questions) < > openstack-dev@lists.openstack.org> > *Subject:* Re: [openstack-dev] [nova] A prototype implementation towards > the "

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

org> Subject: Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler" Yingxin, This looks quite similar to the work of this bp: https://blueprints.launchpad.net/nova/+spec/no-db-scheduler It's really nice that somebody is still trying to push sched

Re: [openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Yingxin, This looks quite similar to the work of this bp: https://blueprints.launchpad.net/nova/+spec/no-db-scheduler It's really nice that somebody is still trying to push scheduler refactoring in this way. Thanks. Best regards, Boris Pavlovic On Sun, Feb 14, 2016 at 9:21 PM, Cheng, Yingxin

[openstack-dev] [nova] A prototype implementation towards the "shared state scheduler"

Hi, I've uploaded a prototype https://review.openstack.org/#/c/280047/ to testify its design goals in accuracy, performance, reliability and compatibility improvements. It will also be an Austin Summit Session if elected: