Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Sylvain Bauza]

2014-04-25 1:38 GMT+02:00 Jay Lau :

> Seems http://summit.openstack.org/cfp/details/262 can cover this? Thanks.
>
>
Well, that will depend on the number of sessions we could get. This #262
proposal was agreed to be merged with #140 if no enough slots.



>
> 2014-04-25 5:09 GMT+08:00 Sylvain Bauza :
>
>
>> Le 24 avr. 2014 19:20, "Henrique Truta"  a
>> écrit :
>>
>> >
>> > Donald,
>> >
>> > By "selection", I think Jenny means identifying whether and which
>> active VM should be migrated, once the current Nova scheduler only deals
>> with the VM in the momment of its creation or with a specific user input.
>> >
>>
>> As Don said, we're beginning the process to spin-off the scheduler by
>> defining a line in the sand in between the sched and other Nova bits.
>>
>> That's the first step before the real fork which will lead to a separate
>> project standing by its own, Gantt. Having that said, the scope of Gantt is
>> currently still subject to discussion, which will happen during a Summit
>> design session.
>>
>> Regarding the need to have a dynamic scheduler acting also on
>> notifications and not only on boot requests, that could be seen either as
>> part to Gantt, or a separate service which would send request to Gantt for
>> selecting destinations. IMHO, with respect to the timeline, I would like to
>> see the finale endpoint going to Gantt, with a temporary Stackforge project
>> if necessary.
>>
>> Again, IMHO, this feature request deserves its own session proposal for
>> the Summit. As the deadline has passed for submissions, we need to know if
>> that has been proposed yet so we could add it as subject of interest for
>> Gantt in an etherpad.
>>
>> Thanks,
>> -Sylvain
>>
>> >
>> > 2014-04-24 12:08 GMT-03:00 Dugger, Donald D > >:
>> >
>> >> Jenny-
>> >>
>> >>
>> >>
>> >> You should look at the `Propose Scheduler Library blueprint’:
>> >>
>> >>
>> >>
>> >> https://review.openstack.org/#/c/82133/9
>> >>
>> >>
>> >>
>> >> This BP is to create a client library for making calls to the
>> scheduler.  If you base your work upon this library then you shouldn’t need
>> to care about whether the Core Scheduler is the Nova integrated scheduler
>> or the Gantt separated scheduler, the library will call `a` scheduler as
>> appropriate.
>> >>
>> >>
>> >>
>> >> Having said that, I’m not sure I understand the distinction you are
>> seeing between `selection’ and `placement’.  The current scheduler filters
>> all hosts based upon filters (the selection part) and then the weighting
>> function finds the best node to host the VM (the placement part).  Seems to
>> me the current scheduler does both of those tasks.  (We can argue the
>> effectiveness/efficiency of the current implementation but I think it’s
>> functionally complete.)
>> >>
>> >>
>> >>
>> >> Also, have you proposed a session at the Juno summit on your proposal
>> for dynamic scheduling, seems like it would be appropriate.
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> Don Dugger
>> >>
>> >> "Censeo Toto nos in Kansa esse decisse." - D. Gale
>> >>
>> >> Ph: 303/443-3786
>> >>
>> >>
>> >>
>> >> From: Jiangying (Jenny) [mailto:jenny.jiangy...@huawei.com]
>> >> Sent: Thursday, April 24, 2014 3:36 AM
>> >> To: OpenStack Development Mailing List (not for usage questions)
>> >> Subject: Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic
>> scheduling
>> >>
>> >>
>> >>
>> >> Hi,
>> >>
>> >> We have checked that gantt now just made a synced up copy of the code
>> in nova.
>> >>
>> >> We still think dynamic scheduling will be a benefit of the nova
>> scheduler (or gantt later). The main difference between static and dynamic
>> scheduling is that static scheduling is a vm placement problem, while
>> dynamic scheduling deals with both vm selection and vm placement.
>> >>
>> >>
>> >>
>> >> Our scheduling mechanism consists of three parts:
>> >>
>> >> 1.   Controller, which triggers the scheduling;
>> >>
>> >> 2.   Da

Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Jay Lau]

Seems http://summit.openstack.org/cfp/details/262 can cover this? Thanks.


2014-04-25 5:09 GMT+08:00 Sylvain Bauza :

>
> Le 24 avr. 2014 19:20, "Henrique Truta"  a
> écrit :
>
> >
> > Donald,
> >
> > By "selection", I think Jenny means identifying whether and which active
> VM should be migrated, once the current Nova scheduler only deals with the
> VM in the momment of its creation or with a specific user input.
> >
>
> As Don said, we're beginning the process to spin-off the scheduler by
> defining a line in the sand in between the sched and other Nova bits.
>
> That's the first step before the real fork which will lead to a separate
> project standing by its own, Gantt. Having that said, the scope of Gantt is
> currently still subject to discussion, which will happen during a Summit
> design session.
>
> Regarding the need to have a dynamic scheduler acting also on
> notifications and not only on boot requests, that could be seen either as
> part to Gantt, or a separate service which would send request to Gantt for
> selecting destinations. IMHO, with respect to the timeline, I would like to
> see the finale endpoint going to Gantt, with a temporary Stackforge project
> if necessary.
>
> Again, IMHO, this feature request deserves its own session proposal for
> the Summit. As the deadline has passed for submissions, we need to know if
> that has been proposed yet so we could add it as subject of interest for
> Gantt in an etherpad.
>
> Thanks,
> -Sylvain
>
> >
> > 2014-04-24 12:08 GMT-03:00 Dugger, Donald D :
> >
> >> Jenny-
> >>
> >>
> >>
> >> You should look at the `Propose Scheduler Library blueprint’:
> >>
> >>
> >>
> >> https://review.openstack.org/#/c/82133/9
> >>
> >>
> >>
> >> This BP is to create a client library for making calls to the
> scheduler.  If you base your work upon this library then you shouldn’t need
> to care about whether the Core Scheduler is the Nova integrated scheduler
> or the Gantt separated scheduler, the library will call `a` scheduler as
> appropriate.
> >>
> >>
> >>
> >> Having said that, I’m not sure I understand the distinction you are
> seeing between `selection’ and `placement’.  The current scheduler filters
> all hosts based upon filters (the selection part) and then the weighting
> function finds the best node to host the VM (the placement part).  Seems to
> me the current scheduler does both of those tasks.  (We can argue the
> effectiveness/efficiency of the current implementation but I think it’s
> functionally complete.)
> >>
> >>
> >>
> >> Also, have you proposed a session at the Juno summit on your proposal
> for dynamic scheduling, seems like it would be appropriate.
> >>
> >>
> >>
> >> --
> >>
> >> Don Dugger
> >>
> >> "Censeo Toto nos in Kansa esse decisse." - D. Gale
> >>
> >> Ph: 303/443-3786
> >>
> >>
> >>
> >> From: Jiangying (Jenny) [mailto:jenny.jiangy...@huawei.com]
> >> Sent: Thursday, April 24, 2014 3:36 AM
> >> To: OpenStack Development Mailing List (not for usage questions)
> >> Subject: Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic
> scheduling
> >>
> >>
> >>
> >> Hi,
> >>
> >> We have checked that gantt now just made a synced up copy of the code
> in nova.
> >>
> >> We still think dynamic scheduling will be a benefit of the nova
> scheduler (or gantt later). The main difference between static and dynamic
> scheduling is that static scheduling is a vm placement problem, while
> dynamic scheduling deals with both vm selection and vm placement.
> >>
> >>
> >>
> >> Our scheduling mechanism consists of three parts:
> >>
> >> 1.   Controller, which triggers the scheduling;
> >>
> >> 2.   Data Collector, which collects the resource usage and topo for
> scheduling;
> >>
> >> 3.   Core Scheduler, which decides how to schedule the vms;
> >>
> >>
> >>
> >> We prefer to reuse the nova scheduler as the Core Scheduler, in order
> to avoid the possible inconsistent between static scheduling and dynamic
> scheduling. The vm selection function will be added into nova scheduler.
> For Data Collector, we expect to get the performance data from ceilometer
> and topo from nova.
> >>
> >>
> >>
> >> There is still one question that where

Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Sylvain Bauza]

Le 24 avr. 2014 19:20, "Henrique Truta"  a
écrit :
>
> Donald,
>
> By "selection", I think Jenny means identifying whether and which active
VM should be migrated, once the current Nova scheduler only deals with the
VM in the momment of its creation or with a specific user input.
>

As Don said, we're beginning the process to spin-off the scheduler by
defining a line in the sand in between the sched and other Nova bits.

That's the first step before the real fork which will lead to a separate
project standing by its own, Gantt. Having that said, the scope of Gantt is
currently still subject to discussion, which will happen during a Summit
design session.

Regarding the need to have a dynamic scheduler acting also on notifications
and not only on boot requests, that could be seen either as part to Gantt,
or a separate service which would send request to Gantt for selecting
destinations. IMHO, with respect to the timeline, I would like to see the
finale endpoint going to Gantt, with a temporary Stackforge project if
necessary.

Again, IMHO, this feature request deserves its own session proposal for the
Summit. As the deadline has passed for submissions, we need to know if that
has been proposed yet so we could add it as subject of interest for Gantt
in an etherpad.

Thanks,
-Sylvain

>
> 2014-04-24 12:08 GMT-03:00 Dugger, Donald D :
>
>> Jenny-
>>
>>
>>
>> You should look at the `Propose Scheduler Library blueprint’:
>>
>>
>>
>> https://review.openstack.org/#/c/82133/9
>>
>>
>>
>> This BP is to create a client library for making calls to the
scheduler.  If you base your work upon this library then you shouldn’t need
to care about whether the Core Scheduler is the Nova integrated scheduler
or the Gantt separated scheduler, the library will call `a` scheduler as
appropriate.
>>
>>
>>
>> Having said that, I’m not sure I understand the distinction you are
seeing between `selection’ and `placement’.  The current scheduler filters
all hosts based upon filters (the selection part) and then the weighting
function finds the best node to host the VM (the placement part).  Seems to
me the current scheduler does both of those tasks.  (We can argue the
effectiveness/efficiency of the current implementation but I think it’s
functionally complete.)
>>
>>
>>
>> Also, have you proposed a session at the Juno summit on your proposal
for dynamic scheduling, seems like it would be appropriate.
>>
>>
>>
>> --
>>
>> Don Dugger
>>
>> "Censeo Toto nos in Kansa esse decisse." - D. Gale
>>
>> Ph: 303/443-3786
>>
>>
>>
>> From: Jiangying (Jenny) [mailto:jenny.jiangy...@huawei.com]
>> Sent: Thursday, April 24, 2014 3:36 AM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling
>>
>>
>>
>> Hi,
>>
>> We have checked that gantt now just made a synced up copy of the code in
nova.
>>
>> We still think dynamic scheduling will be a benefit of the nova
scheduler (or gantt later). The main difference between static and dynamic
scheduling is that static scheduling is a vm placement problem, while
dynamic scheduling deals with both vm selection and vm placement.
>>
>>
>>
>> Our scheduling mechanism consists of three parts:
>>
>> 1.   Controller, which triggers the scheduling;
>>
>> 2.   Data Collector, which collects the resource usage and topo for
scheduling;
>>
>> 3.   Core Scheduler, which decides how to schedule the vms;
>>
>>
>>
>> We prefer to reuse the nova scheduler as the Core Scheduler, in order to
avoid the possible inconsistent between static scheduling and dynamic
scheduling. The vm selection function will be added into nova scheduler.
For Data Collector, we expect to get the performance data from ceilometer
and topo from nova.
>>
>>
>>
>> There is still one question that where the controller should be
implemented?
>>
>> We regard implementing the controller in nova scheduler as the first
choice. And we also consider extending ceilometer.(Ie. When ceilometer
discovers an overload host, an alarm can be reported and it can trigger a
vm evacuate.)
>>
>>
>>
>> Do you have any comments?
>>
>>
>>
>> Jenny
>>
>>
>>
>> 发件人: Henrique Truta [mailto:henriquecostatr...@gmail.com]
>> 发送时间: 2014年4月12日 1:00
>> 收件人: OpenStack Development Mailing List (not for usage questions)
>> 主题: Re: [openstack-dev] [nova] Dynamic scheduling
>>
>>
>>
>> Is there anyone currently working on Neat/G

Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Dugger, Donald D]

If we’re talking about automatic migration then yes, that is a very interesting 
subject that I hope we can address but there’s still lots of things that need 
to be done for initial scheduling.  There are many other issues with automatic 
migration (hysteresis, migration cost, policy decisions come to mind 
immediately) that have to be considered.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Henrique Truta [mailto:henriquecostatr...@gmail.com]
Sent: Thursday, April 24, 2014 11:16 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

Donald,
By "selection", I think Jenny means identifying whether and which active VM 
should be migrated, once the current Nova scheduler only deals with the VM in 
the momment of its creation or with a specific user input.

2014-04-24 12:08 GMT-03:00 Dugger, Donald D 
mailto:donald.d.dug...@intel.com>>:
Jenny-

You should look at the `Propose Scheduler Library blueprint’:

https://review.openstack.org/#/c/82133/9

This BP is to create a client library for making calls to the scheduler.  If 
you base your work upon this library then you shouldn’t need to care about 
whether the Core Scheduler is the Nova integrated scheduler or the Gantt 
separated scheduler, the library will call `a` scheduler as appropriate.

Having said that, I’m not sure I understand the distinction you are seeing 
between `selection’ and `placement’.  The current scheduler filters all hosts 
based upon filters (the selection part) and then the weighting function finds 
the best node to host the VM (the placement part).  Seems to me the current 
scheduler does both of those tasks.  (We can argue the effectiveness/efficiency 
of the current implementation but I think it’s functionally complete.)

Also, have you proposed a session at the Juno summit on your proposal for 
dynamic scheduling, seems like it would be appropriate.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Jiangying (Jenny) 
[mailto:jenny.jiangy...@huawei.com<mailto:jenny.jiangy...@huawei.com>]
Sent: Thursday, April 24, 2014 3:36 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

Hi,
We have checked that gantt now just made a synced up copy of the code in nova.
We still think dynamic scheduling will be a benefit of the nova scheduler (or 
gantt later). The main difference between static and dynamic scheduling is that 
static scheduling is a vm placement problem, while dynamic scheduling deals 
with both vm selection and vm placement.

Our scheduling mechanism consists of three parts:
1.   Controller, which triggers the scheduling;
2.   Data Collector, which collects the resource usage and topo for scheduling;
3.   Core Scheduler, which decides how to schedule the vms;

We prefer to reuse the nova scheduler as the Core Scheduler, in order to avoid 
the possible inconsistent between static scheduling and dynamic scheduling. The 
vm selection function will be added into nova scheduler. For Data Collector, we 
expect to get the performance data from ceilometer and topo from nova.

There is still one question that where the controller should be implemented?
We regard implementing the controller in nova scheduler as the first choice. 
And we also consider extending ceilometer.(Ie. When ceilometer discovers an 
overload host, an alarm can be reported and it can trigger a vm evacuate.)

Do you have any comments?

Jenny

发件人: Henrique Truta [mailto:henriquecostatr...@gmail.com]
发送时间: 2014年4月12日 1:00
收件人: OpenStack Development Mailing List (not for usage questions)
主题: Re: [openstack-dev] [nova] Dynamic scheduling

Is there anyone currently working on Neat/Gantt projects? I'd like to 
contribute to them, as well.

2014-04-11 11:37 GMT-03:00 Andrew Laski 
mailto:andrew.la...@rackspace.com>>:
On 04/10/14 at 11:33pm, Oleg Gelbukh wrote:
Andrew,

Thank you for clarification!


On Thu, Apr 10, 2014 at 3:47 PM, Andrew Laski 
mailto:andrew.la...@rackspace.com>>wrote:


The scheduler as it currently exists is a placement engine.  There is
sufficient complexity in the scheduler with just that responsibility so I
would prefer to see anything that's making runtime decisions separated out.
 Perhaps it could just be another service within the scheduler project once
it's broken out, but I think it will be beneficial to have a clear
distinction between placement decisions and runtime monitoring.


Do you think that auto-scaling could be considered another facet of this
'runtime monitoring' functionality? Now it is a combination of Heat and
Ceilometer. Does it worth moving to hypothetical runtime mobility service
as well?

Auto-scaling is certainly a facet of runtime monitoring.  But auto-scaling 
perfo

Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Henrique Truta]

Donald,

By "selection", I think Jenny means identifying whether and which active VM
should be migrated, once the current Nova scheduler only deals with the VM
in the momment of its creation or with a specific user input.


2014-04-24 12:08 GMT-03:00 Dugger, Donald D :

>  Jenny-
>
>
>
> You should look at the `Propose Scheduler Library blueprint’:
>
>
>
> https://review.openstack.org/#/c/82133/9
>
>
>
> This BP is to create a client library for making calls to the scheduler.
> If you base your work upon this library then you shouldn’t need to care
> about whether the Core Scheduler is the Nova integrated scheduler or the
> Gantt separated scheduler, the library will call `a` scheduler as
> appropriate.
>
>
>
> Having said that, I’m not sure I understand the distinction you are seeing
> between `selection’ and `placement’.  The current scheduler filters all
> hosts based upon filters (the selection part) and then the weighting
> function finds the best node to host the VM (the placement part).  Seems to
> me the current scheduler does both of those tasks.  (We can argue the
> effectiveness/efficiency of the current implementation but I think it’s
> functionally complete.)
>
>
>
> Also, have you proposed a session at the Juno summit on your proposal for
> dynamic scheduling, seems like it would be appropriate.
>
>
>
> --
>
> Don Dugger
>
> "Censeo Toto nos in Kansa esse decisse." - D. Gale
>
> Ph: 303/443-3786
>
>
>
> *From:* Jiangying (Jenny) [mailto:jenny.jiangy...@huawei.com]
> *Sent:* Thursday, April 24, 2014 3:36 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic
> scheduling
>
>
>
> Hi,
>
> We have checked that gantt now just made a synced up copy of the code in
> nova.
>
> We still think dynamic scheduling will be a benefit of the nova scheduler
> (or gantt later). The main difference between static and dynamic scheduling
> is that static scheduling is a vm placement problem, while dynamic
> scheduling deals with both vm selection and vm placement.
>
>
>
> Our scheduling mechanism consists of three parts:
>
> 1.   Controller, which triggers the scheduling;
>
> 2.   Data Collector, which collects the resource usage and topo for
> scheduling;
>
> 3.   Core Scheduler, which decides how to schedule the vms;
>
>
>
> We prefer to reuse the nova scheduler as the Core Scheduler, in order to
> avoid the possible inconsistent between static scheduling and dynamic
> scheduling. The vm selection function will be added into nova scheduler.
> For Data Collector, we expect to get the performance data from ceilometer
> and topo from nova.
>
>
>
> There is still one question that where the controller should be
> implemented?
>
> We regard implementing the controller in nova scheduler as the first
> choice. And we also consider extending ceilometer.(Ie. When ceilometer
> discovers an overload host, an alarm can be reported and it can trigger a
> vm evacuate.)
>
>
>
> Do you have any comments?
>
>
>
> Jenny
>
>
>
> *发件人**:* Henrique Truta 
> [mailto:henriquecostatr...@gmail.com]
>
> *发送时间:* 2014年4月12日 1:00
> *收件人:* OpenStack Development Mailing List (not for usage questions)
> *主题:* Re: [openstack-dev] [nova] Dynamic scheduling
>
>
>
> Is there anyone currently working on Neat/Gantt projects? I'd like to
> contribute to them, as well.
>
>
>
> 2014-04-11 11:37 GMT-03:00 Andrew Laski :
>
> On 04/10/14 at 11:33pm, Oleg Gelbukh wrote:
>
> Andrew,
>
> Thank you for clarification!
>
>
> On Thu, Apr 10, 2014 at 3:47 PM, Andrew Laski  >wrote:
>
>
>
> The scheduler as it currently exists is a placement engine.  There is
> sufficient complexity in the scheduler with just that responsibility so I
> would prefer to see anything that's making runtime decisions separated out.
>  Perhaps it could just be another service within the scheduler project once
> it's broken out, but I think it will be beneficial to have a clear
> distinction between placement decisions and runtime monitoring.
>
>
>
> Do you think that auto-scaling could be considered another facet of this
> 'runtime monitoring' functionality? Now it is a combination of Heat and
> Ceilometer. Does it worth moving to hypothetical runtime mobility service
> as well?
>
>
>
> Auto-scaling is certainly a facet of runtime monitoring.  But auto-scaling
> performs actions based on a set of user defined rules and is very visible
> while the enhancements proposed below are intended to benefit d

Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Dugger, Donald D]

Jenny-

You should look at the `Propose Scheduler Library blueprint’:

https://review.openstack.org/#/c/82133/9

This BP is to create a client library for making calls to the scheduler.  If 
you base your work upon this library then you shouldn’t need to care about 
whether the Core Scheduler is the Nova integrated scheduler or the Gantt 
separated scheduler, the library will call `a` scheduler as appropriate.

Having said that, I’m not sure I understand the distinction you are seeing 
between `selection’ and `placement’.  The current scheduler filters all hosts 
based upon filters (the selection part) and then the weighting function finds 
the best node to host the VM (the placement part).  Seems to me the current 
scheduler does both of those tasks.  (We can argue the effectiveness/efficiency 
of the current implementation but I think it’s functionally complete.)

Also, have you proposed a session at the Juno summit on your proposal for 
dynamic scheduling, seems like it would be appropriate.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

From: Jiangying (Jenny) [mailto:jenny.jiangy...@huawei.com]
Sent: Thursday, April 24, 2014 3:36 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

Hi,
We have checked that gantt now just made a synced up copy of the code in nova.
We still think dynamic scheduling will be a benefit of the nova scheduler (or 
gantt later). The main difference between static and dynamic scheduling is that 
static scheduling is a vm placement problem, while dynamic scheduling deals 
with both vm selection and vm placement.

Our scheduling mechanism consists of three parts:
1.   Controller, which triggers the scheduling;
2.   Data Collector, which collects the resource usage and topo for scheduling;
3.   Core Scheduler, which decides how to schedule the vms;

We prefer to reuse the nova scheduler as the Core Scheduler, in order to avoid 
the possible inconsistent between static scheduling and dynamic scheduling. The 
vm selection function will be added into nova scheduler. For Data Collector, we 
expect to get the performance data from ceilometer and topo from nova.

There is still one question that where the controller should be implemented?
We regard implementing the controller in nova scheduler as the first choice. 
And we also consider extending ceilometer.(Ie. When ceilometer discovers an 
overload host, an alarm can be reported and it can trigger a vm evacuate.)

Do you have any comments?

Jenny

发件人: Henrique Truta [mailto:henriquecostatr...@gmail.com]
发送时间: 2014年4月12日 1:00
收件人: OpenStack Development Mailing List (not for usage questions)
主题: Re: [openstack-dev] [nova] Dynamic scheduling

Is there anyone currently working on Neat/Gantt projects? I'd like to 
contribute to them, as well.

2014-04-11 11:37 GMT-03:00 Andrew Laski 
mailto:andrew.la...@rackspace.com>>:
On 04/10/14 at 11:33pm, Oleg Gelbukh wrote:
Andrew,

Thank you for clarification!


On Thu, Apr 10, 2014 at 3:47 PM, Andrew Laski 
mailto:andrew.la...@rackspace.com>>wrote:


The scheduler as it currently exists is a placement engine.  There is
sufficient complexity in the scheduler with just that responsibility so I
would prefer to see anything that's making runtime decisions separated out.
 Perhaps it could just be another service within the scheduler project once
it's broken out, but I think it will be beneficial to have a clear
distinction between placement decisions and runtime monitoring.


Do you think that auto-scaling could be considered another facet of this
'runtime monitoring' functionality? Now it is a combination of Heat and
Ceilometer. Does it worth moving to hypothetical runtime mobility service
as well?

Auto-scaling is certainly a facet of runtime monitoring.  But auto-scaling 
performs actions based on a set of user defined rules and is very visible while 
the enhancements proposed below are intended to benefit deployers and be very 
invisible to users.  So the set of allowable actions is very constrained 
compared to what auto-scaling can do.
In my opinion what's being proposed doesn't seem to fit cleanly into any 
existing service, so perhaps it could start as a standalone entity.  Then once 
there's something that can be used and demoed a proper place might suggest 
itself, or it might make sense to keep it separate.


--
Best regards,
Oleg Gelbukh

--
Best regards,
Oleg Gelbukh


On Wed, Apr 9, 2014 at 7:47 PM, Jay Lau 
mailto:jay.lau@gmail.com>> wrote:

 @Oleg, Till now, I'm not sure the target of Gantt, is it for initial
placement policy or run time policy or both, can you help clarify?

@Henrique, not sure if you know IBM PRS (Platform Resource Scheduler)
[1],
we have finished the "dynamic scheduler" in our Icehouse version (PRS
2.2),
it has exactly the same feature as 

Re: [openstack-dev] [nova][ceilometer][gantt] Dynamic scheduling

[Jiangying (Jenny)]

Hi,
We have checked that gantt now just made a synced up copy of the code in nova.
We still think dynamic scheduling will be a benefit of the nova scheduler (or 
gantt later). The main difference between static and dynamic scheduling is that 
static scheduling is a vm placement problem, while dynamic scheduling deals 
with both vm selection and vm placement.

Our scheduling mechanism consists of three parts:
1.   Controller, which triggers the scheduling;
2.   Data Collector, which collects the resource usage and topo for scheduling;
3.   Core Scheduler, which decides how to schedule the vms;

We prefer to reuse the nova scheduler as the Core Scheduler, in order to avoid 
the possible inconsistent between static scheduling and dynamic scheduling. The 
vm selection function will be added into nova scheduler. For Data Collector, we 
expect to get the performance data from ceilometer and topo from nova.

There is still one question that where the controller should be implemented?
We regard implementing the controller in nova scheduler as the first choice. 
And we also consider extending ceilometer.(Ie. When ceilometer discovers an 
overload host, an alarm can be reported and it can trigger a vm evacuate.)

Do you have any comments?

Jenny

发件人: Henrique Truta [mailto:henriquecostatr...@gmail.com]
发送时间: 2014年4月12日 1:00
收件人: OpenStack Development Mailing List (not for usage questions)
主题: Re: [openstack-dev] [nova] Dynamic scheduling

Is there anyone currently working on Neat/Gantt projects? I'd like to 
contribute to them, as well.

2014-04-11 11:37 GMT-03:00 Andrew Laski 
mailto:andrew.la...@rackspace.com>>:
On 04/10/14 at 11:33pm, Oleg Gelbukh wrote:
Andrew,

Thank you for clarification!


On Thu, Apr 10, 2014 at 3:47 PM, Andrew Laski 
mailto:andrew.la...@rackspace.com>>wrote:


The scheduler as it currently exists is a placement engine.  There is
sufficient complexity in the scheduler with just that responsibility so I
would prefer to see anything that's making runtime decisions separated out.
 Perhaps it could just be another service within the scheduler project once
it's broken out, but I think it will be beneficial to have a clear
distinction between placement decisions and runtime monitoring.


Do you think that auto-scaling could be considered another facet of this
'runtime monitoring' functionality? Now it is a combination of Heat and
Ceilometer. Does it worth moving to hypothetical runtime mobility service
as well?

Auto-scaling is certainly a facet of runtime monitoring.  But auto-scaling 
performs actions based on a set of user defined rules and is very visible while 
the enhancements proposed below are intended to benefit deployers and be very 
invisible to users.  So the set of allowable actions is very constrained 
compared to what auto-scaling can do.
In my opinion what's being proposed doesn't seem to fit cleanly into any 
existing service, so perhaps it could start as a standalone entity.  Then once 
there's something that can be used and demoed a proper place might suggest 
itself, or it might make sense to keep it separate.



--
Best regards,
Oleg Gelbukh



--
Best regards,
Oleg Gelbukh


On Wed, Apr 9, 2014 at 7:47 PM, Jay Lau 
mailto:jay.lau@gmail.com>> wrote:

 @Oleg, Till now, I'm not sure the target of Gantt, is it for initial
placement policy or run time policy or both, can you help clarify?

@Henrique, not sure if you know IBM PRS (Platform Resource Scheduler)
[1],
we have finished the "dynamic scheduler" in our Icehouse version (PRS
2.2),
it has exactly the same feature as your described, we are planning a live
demo for this feature in Atlanta Summit. I'm also writing some document
for
run time policy which will cover more run time policies for OpenStack,
but
not finished yet. (My shame for the slow progress). The related blueprint
is [2], you can also get some discussion from [3]

[1]
http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=
AN&subtype=CA&htmlfid=897/ENUS213-590&appname=USN
[2]
https://blueprints.launchpad.net/nova/+spec/resource-
optimization-service
[3] http://markmail.org/~jaylau/OpenStack-DRS

Thanks.


2014-04-09 23:21 GMT+08:00 Oleg Gelbukh 
mailto:ogelb...@mirantis.com>>:

Henrique,

You should check out Gantt project [1], it could be exactly the place to
implement such features. It is a generic cross-project Scheduler as a
Service forked from Nova recently.

[1] https://github.com/openstack/gantt

--
Best regards,
Oleg Gelbukh
Mirantis Labs


On Wed, Apr 9, 2014 at 6:41 PM, Henrique Truta <
henriquecostatr...@gmail.com> wrote:

 Hello, everyone!

I am currently a graduate student and member of a group of contributors
to OpenStack. We believe that a dynamic scheduler could improve the
efficiency of an OpenStack cloud, either by rebalancing nodes to
maximize
performance or to minimize the number of active hosts, in order to
minimize
energy costs. Therefore, we would like to propose a dynamic scheduling
mechanism to