Re: [openstack-dev] [Nova] [Climate] Reservation service called Climate

2013-11-13 Thread Sylvain Bauza

Hi Andrew,

Le 12/11/2013 21:50, Andrew Laski a écrit :

On 11/06/13 at 09:47am, Sylvain Bauza wrote:

Hi,

During the Design session 
https://etherpad.openstack.org/p/NovaIcehouse-Instance-Group-API we 
discussed the fact that this is not the role of Nova for doing atomic 
reservations in order to ensure the user needs will be met.


We discussed that it's not Nova's role to do atomic reservations of 
groups of resources.  But two phase commit came up a number of times 
for single instance scheduling/reservations which would allow 
reservations of groups to occur with some outside coordination.




Indeed you're right, the consensus was about nested InstanceGroups, 
where it was said it should be done outside Nova.


I raised the point (and sorry for my bad accent, was stressy) that 
we're already trying to provide a reservation system for Openstack, 
called Climate (a Stackforge project).
I would really like to discuss with you all, Nova community, about 
the reservation system and ensure that we, at Climate, are on the 
good path.


I think Climate has some interesting potential around capacity 
planning and would like to see it integrate well with Nova.  But my 
understanding of both proposals makes me think that the reservation 
system that Climate wants to provide isn't the best solution for the 
Instance Group work.  Instance Groups, or the longer term idea of 
Resource Groups should be workable with two phase commit since it's 
concerned about what can be placed at that moment.  Climate also 
considers future placements which is very cool but leads(potentially) 
to different design considerations and trade-offs.


Well, probably I jumped to the conclusion without waiting to see what 
InstanceGroups will actually be.
Let's follow the discussion on the InstanceGroups, and just keep in mind 
that Climate could help.
My main concern is just making sure we won't duplicate work on 
reservations themselves, that's it.


Climate is a quite new Stackforge project with little visibility, that's 
why I spoke about it.





One thing that isn't clear to me from the etherpad 
(https://etherpad.openstack.org/p/NovaIcehouse-ClimateInteractions) 
and the discussion is what API behaviour in Nova would be ideal for 
Climate?  You've listed questions about pclouds and host aggregates 
and other things that Nova provides, but those seem like 
implementation details to me.  What kinds of actions could Nova 
provide that would allow Climate to function well?




Well, the issue we had is that we spent the 10 mins speaking about how 
Climate could ensure that the leases will be honored, and consequently I 
ran out of time speaking about the interactions by themselves.


Let me just clarify how we can do the math : Climate will provide an 
Host reservation Admin API allowing administrators to dedicate 
nova-computes to Climate (ie. putting them in an host aggregate called 
freepool with no ways for getting them elected by the Nova scheduler 
on a regular nova boot command).


We'll then keep a backlog of reservations mapped to hosts and 
consequently we will be sure that on an atomic fashion, any hosts 
selected won't already have VMs running on them.



That said, let's go back to your question and what Nova features would 
be nice for Climate :


 1/ as said, there is no clear consensus on about Pclouds or AZs. I was 
at the Pcloud design session, and I saw that Phil is not planning having 
them delivered in Icehouse. As a consequence, we will begin our 
implementation by using AZs, that should match our needs. Anyway, we'll 
provide an interface for managing what we call Reservation Pools, and 
we will keep compatibility in between Pclouds and AZs in it. One side 
note is that we aren't using Host Flavors yet but rather an explicit 
dictionary of capabilities, that should be modified for exposing flavors 
instead, but not in the first Climate release.


 2/ So as to get host capabilities and as the resource tracker is not 
extensible, we made a design choice to implement an Hosts DB model 
having same attributes as in Nova (duplicate work, ergh.) with an 
associated table called HostExtraCapabilities with key/value pairs 
(see [1]). That's not perfect. The best way would maybe to plug into 
Nova-conductor for getting Hosts capabilities but that doesn't match yet 
our needs. Comments welcome.


 3/ In order to elect the rights hosts from the freepool (the host 
aggregate where hosts are dedicated to Climate) to the Reservation pool, 
we will implement our own scheduler. Also, duplicate work. It would be 
great ideally if the scheduler could have its own API endpoints so we 
could ask Nova provide me X hosts with these capabilities and put them 
in this AZ. Comments welcome too.


[1] : 
https://review.openstack.org/#/c/49363/10/climate/db/sqlalchemy/models.py



Thanks,
-Sylvain



Climate is planning to reserve both virtual instances and physical 
hosts, but for the discussion we had, only physical hosts usecase is 

Re: [openstack-dev] [Nova] [Climate] Reservation service called Climate

2013-11-07 Thread Dina Belova
Guys, there are Slideshare slides if it is more comfortable to use them for
you:

Slideshare Climate
Slideshttp://www.slideshare.net/dbelova/climate-project-28026918

Thanks!


On Wed, Nov 6, 2013 at 5:47 PM, Sylvain Bauza sylvain.ba...@bull.netwrote:

  Hi,

 During the Design session
 https://etherpad.openstack.org/p/NovaIcehouse-Instance-Group-API we
 discussed the fact that this is not the role of Nova for doing atomic
 reservations in order to ensure the user needs will be met.

 I raised the point (and sorry for my bad accent, was stressy) that we're
 already trying to provide a reservation system for Openstack, called
 Climate (a Stackforge project).
 I would really like to discuss with you all, Nova community, about the
 reservation system and ensure that we, at Climate, are on the good path.

 Climate is planning to reserve both virtual instances and physical hosts,
 but for the discussion we had, only physical hosts usecase is relevant.

 We had an unconference session today at 2pm, I can share you the slides :


 https://docs.google.com/presentation/d/1BJGmtzGees6tg_Np7JuKFtuLGiCaguVYD8hYJ2eVKAc/edit#slide=id.p

 (please focus on slides 11-14, they're talking on the design for host
 reservations)

 All the code is located on Stackforge, but please note the most important
 part of physical host reservations is still under review there :

 https://review.openstack.org/#/q/project:+stackforge/climate+status:open,n,z

 (We're missing reviewers, by the way !)


 I'm open to discuss and waiting your thoughts,
 -Sylvain


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 

Best regards,

Dina Belova

Software Engineer

Mirantis Inc.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev