Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-10 Thread Attila Fazekas
- Original Message - From: Jay Pipes jaypi...@gmail.com To: openstack-dev@lists.openstack.org Sent: Wednesday, March 4, 2015 9:22:43 PM Subject: Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler On 03/04/2015 01:51 AM, Attila Fazekas wrote

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-10 Thread Nikola Đipanov
On 03/06/2015 03:19 PM, Attila Fazekas wrote: Looks like we need some kind of _per compute node_ mutex in the critical section, multiple scheduler MAY be able to schedule to two compute node at same time, but not for scheduling to the same compute node. If we don't want to introduce

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-10 Thread Attila Fazekas
- Original Message - From: Nikola Đipanov ndipa...@redhat.com To: openstack-dev@lists.openstack.org Sent: Tuesday, March 10, 2015 10:53:01 AM Subject: Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler On 03/06/2015 03:19 PM, Attila Fazekas

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-10 Thread Attila Fazekas
supported in nova-scheduler - Original Message - From: Nikola Đipanov ndipa...@redhat.com To: openstack-dev@lists.openstack.org Sent: Tuesday, March 10, 2015 10:53:01 AM Subject: Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler On 03

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-06 Thread Attila Fazekas
@lists.openstack.org Sent: Friday, March 6, 2015 10:29:52 AM Subject: Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler On 03/06/2015 01:56 AM, Rui Chen wrote: Thank you very much for in-depth discussion about this topic, @Nikola and @Sylvain. I agree

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-06 Thread Attila Fazekas
scheduling.. - Original Message - From: Nikola Đipanov ndipa...@redhat.com To: openstack-dev@lists.openstack.org Sent: Friday, March 6, 2015 10:29:52 AM Subject: Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler On 03/06/2015 01:56 AM, Rui

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-06 Thread Mehdi Abaakouk
Hi, just some oslo.messaging thoughts about having multiple nova-scheduler processes (can also apply to any other daemon acting as rpc server), nova-scheduler use service.Service.create() to create a rpc server, that one is identified by a 'topic' and a 'server' (the

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-06 Thread Nikola Đipanov
On 03/06/2015 01:56 AM, Rui Chen wrote: Thank you very much for in-depth discussion about this topic, @Nikola and @Sylvain. I agree that we should solve the technical debt firstly, and then make the scheduler better. That was not necessarily my point. I would be happy to see work on how

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-05 Thread Rui Chen
Thank you very much for in-depth discussion about this topic, @Nikola and @Sylvain. I agree that we should solve the technical debt firstly, and then make the scheduler better. Best Regards. 2015-03-05 21:12 GMT+08:00 Sylvain Bauza sba...@redhat.com: Le 05/03/2015 13:00, Nikola Đipanov a

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-05 Thread Sylvain Bauza
Le 05/03/2015 08:54, Rui Chen a écrit : We will face the same issue in multiple nova-scheduler process case, like Sylvain say, right? Two processes/workers can actually consume two distinct resources on the same HostState. No. The problem I mentioned was related to having multiple

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-05 Thread Rui Chen
My BP aims is launching multiple nova-scheduler processes on a host, like nova-conductor. If we run multiple nova-scheduler services on separate hosts, that will work, forking the multiple nova-scheduler child processes on a host that will work too? Different child processes had different

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-05 Thread Nikola Đipanov
On 03/04/2015 09:23 AM, Sylvain Bauza wrote: Le 04/03/2015 04:51, Rui Chen a écrit : Hi all, I want to make it easy to launch a bunch of scheduler processes on a host, multiple scheduler workers will make use of multiple processors of host and enhance the performance of nova-scheduler. I

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-05 Thread Sylvain Bauza
Le 05/03/2015 13:00, Nikola Đipanov a écrit : On 03/04/2015 09:23 AM, Sylvain Bauza wrote: Le 04/03/2015 04:51, Rui Chen a écrit : Hi all, I want to make it easy to launch a bunch of scheduler processes on a host, multiple scheduler workers will make use of multiple processors of host and

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Mike Bayer
Attila Fazekas afaze...@redhat.com wrote: Hi, I wonder what is the planned future of the scheduling. The scheduler does a lot of high field number query, which is CPU expensive when you are using sqlalchemy-orm. Does anyone tried to switch those operations to sqlalchemy-core ? An

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Jay Pipes
On 03/04/2015 01:51 AM, Attila Fazekas wrote: Hi, I wonder what is the planned future of the scheduling. The scheduler does a lot of high field number query, which is CPU expensive when you are using sqlalchemy-orm. Does anyone tried to switch those operations to sqlalchemy-core ? Actually,

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Rui Chen
Looks like it's a complicated problem, and nova-scheduler can't scale-out horizontally in active/active mode. Maybe we should illustrate the problem in the HA docs. http://docs.openstack.org/high-availability-guide/content/_schedulers.html Thanks for everybody's attention. 2015-03-05 5:38

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Rui Chen
We will face the same issue in multiple nova-scheduler process case, like Sylvain say, right? Two processes/workers can actually consume two distinct resources on the same HostState. 2015-03-05 13:26 GMT+08:00 Alex Xu sou...@gmail.com: Rui, you still can run multiple nova-scheduler process

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Alex Xu
Rui, you still can run multiple nova-scheduler process now. 2015-03-05 10:55 GMT+08:00 Rui Chen chenrui.m...@gmail.com: Looks like it's a complicated problem, and nova-scheduler can't scale-out horizontally in active/active mode. Maybe we should illustrate the problem in the HA docs.

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Attila Fazekas
Hi, I wonder what is the planned future of the scheduling. The scheduler does a lot of high field number query, which is CPU expensive when you are using sqlalchemy-orm. Does anyone tried to switch those operations to sqlalchemy-core ? The scheduler does lot of thing in the application, like

Re: [openstack-dev] [nova] blueprint about multiple workers supported in nova-scheduler

2015-03-04 Thread Sylvain Bauza
Le 04/03/2015 04:51, Rui Chen a écrit : Hi all, I want to make it easy to launch a bunch of scheduler processes on a host, multiple scheduler workers will make use of multiple processors of host and enhance the performance of nova-scheduler. I had registered a blueprint and commit a patch