Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-02 Thread Mike Spreitzer
FYI, I have refined my pictures at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz-GQQ1bRVgBpJdstpu0lH_TONw6g to hopefully make it clearer that I agree with the sentiment that holistic infrastructure

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Clint Byrum
Excerpts from Georgy Okrokvertskhov's message of 2013-09-30 11:44:26 -0700: Hi, I am working on the OpenStack project Murano which actually had to solve the same problem with software level orchestration. Right now Murano has a DSL language which allows you to define a workflow for a complex

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Robert Collins
On 1 October 2013 19:31, Clint Byrum cl...@fewbar.com wrote: TripleO meets at 2000 UTC every Tuesday. 1900UTC. :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Sylvain Bauza
Hi Mike and Zane, Le 27/09/2013 15:58, Mike Spreitzer a écrit : Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: Your diagrams clearly show scheduling happening in a separate stage to (infrastructure) orchestration, which is to say that at the point where resources are

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-10-01 Thread Thomas Spatzier
Clint Byrum cl...@fewbar.com wrote on 01.10.2013 08:31:44 - Excerpt: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 01.10.2013 08:33 Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-30 Thread Georgy Okrokvertskhov
Hi, I am working on the OpenStack project Murano which actually had to solve the same problem with software level orchestration. Right now Murano has a DSL language which allows you to define a workflow for a complex service deployment. Murano uses Heat for infrastructure managements and actually

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-27 Thread Zane Bitter
On 25/09/13 07:03, Mike Spreitzer wrote: Zane wrote: To take the first example, wouldn't your holistic scheduler effectively have to reserve a compute instance and some directly attached block storage prior to actually creating them? Have you considered Climate rather than Heat as an

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-27 Thread Mike Spreitzer
Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: Your diagrams clearly show scheduling happening in a separate stage to (infrastructure) orchestration, which is to say that at the point where resources are scheduled, their actual creation is in the *future*. I am not a

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-27 Thread Mike Spreitzer
Sorry, I was a bit too hasty in writing the last part of my last message; I forgot to qualify software orchestration to indicate I am speaking only of its preparatory phase. I should have written: Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: ... If I understood your

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-27 Thread Debojyoti Dutta
Hi Mike We understand your point, and, academically, we agree to a large extent. We aware of the optimization results for holistic placement etc (and how any realistic formulation is hard). When we look at the scheduler today, it does one thing at a time because it was meant to be pragmatic when

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-26 Thread Steven Hardy
On Wed, Sep 25, 2013 at 11:04:54PM +0200, Thomas Spatzier wrote: Excerpt from Clint's mail on 25.09.2013 22:23:07: I think we already have some summit suggestions for discussing HOT, it would be good to come prepared with some visions for the future of HOT so that we can hash these

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Clint Byrum
Excerpts from Mike Spreitzer's message of 2013-09-24 22:03:21 -0700: Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Thomas Spatzier
Clint Byrum cl...@fewbar.com wrote on 25.09.2013 08:46:57: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 25.09.2013 08:48 Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Mike Spreitzer
Clint wrote: There is a third stealth-objective that CFN has caused to linger in Heat. That is packaging cloud applications. By allowing the 100% concrete CFN template to stand alone, users can ship the template. IMO this marrying of software assembly, config, and orchestration is a

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Clint Byrum
Excerpts from Thomas Spatzier's message of 2013-09-25 00:59:44 -0700: Clint Byrum cl...@fewbar.com wrote on 25.09.2013 08:46:57: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 25.09.2013 08:48 Subject: Re: [openstack-dev] [heat] [scheduler

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-25 Thread Mike Spreitzer
Debo, Yathi: I have read https://docs.google.com/document/d/1IiPI0sfaWb1bdYiMWzAAx0HYR6UqzOan_Utgml5W1HI/edit?pli=1 and most of the referenced materials, and I have a couple of big-picture questions. That document talks about making Nova call out to something that makes the sort of smart

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-25 Thread Thomas Spatzier
Excerpt from Clint's mail on 25.09.2013 22:23:07: I think we already have some summit suggestions for discussing HOT, it would be good to come prepared with some visions for the future of HOT so that we can hash these things out, so I'd like to see this discussion continue. Absolutely! Can

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-24 Thread Zane Bitter
On 24/09/13 05:31, Mike Spreitzer wrote: I was not trying to raise issues of geographic dispersion and other higher level structures, I think the issues I am trying to raise are relevant even without them. This is not to deny the importance, or relevance, of higher levels of structure. But I

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-24 Thread Debojyoti Dutta
Joining the party late :) I think there have been a lot of interesting ideas around wholistic scheduling over the last few summits. However seems there is no clear agreement on 1) where it should be specified and implemented 2) what the specifications look like - VRT, policies, templates etc etc

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)

2013-09-24 Thread Mike Spreitzer
Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and added a companion picture at

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Zane Bitter
On 15/09/13 09:19, Mike Spreitzer wrote: But first I must admit that I am still a newbie to OpenStack, and still am missing some important clues. One thing that mystifies me is this: I see essentially the same thing, which I have generally taken to calling holistic scheduling, discussed in two

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Keith Bray
I think this picture is relevant to Heat context: https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o 0R9U/edit As more and more types of compute (containers, VMs, bare metal) and other resources (geographically dispersed) become available from the cloud with boarder

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Clint Byrum
Excerpts from Keith Bray's message of 2013-09-23 12:22:16 -0700: I think this picture is relevant to Heat context: https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o 0R9U/edit As more and more types of compute (containers, VMs, bare metal) and other resources

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Mike Spreitzer
I was not trying to raise issues of geographic dispersion and other higher level structures, I think the issues I am trying to raise are relevant even without them. This is not to deny the importance, or relevance, of higher levels of structure. But I would like to first respond to the

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-23 Thread Mike Spreitzer
Someone earlier asked for greater clarity about infrastructure orchestration, so here is my view. I see two main issues: (1) deciding the order in which to do things, and (2) doing them in an acceptable order. That's an oversimplified wording because, in general, some parallelism is

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-20 Thread Mike Spreitzer
I have written a new outline of my thoughts, you can find it at https://docs.google.com/document/d/1RV_kN2Io4dotxZREGEks9DM0Ih_trFZ-PipVDdzxq_E It is intended to stand up better to independent study. However, it is still just an outline. I am still learning about stuff going on in OpenStack,

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-20 Thread Christopher Armstrong
Hi Mike, I have a *slightly* better idea of the kind of stuff you're talking about, but I think it would really help if you could include some concrete real-world use cases and describe why a holistic scheduler inside of Heat is necessary for solving them. On Fri, Sep 20, 2013 at 2:13 AM, Mike

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-18 Thread Stein, Manuel (Manuel)
] Sent: Dienstag, 17. September 2013 07:00 To: OpenStack Development Mailing List Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse I have written a brief document, with pictures. See https://docs.google.com/document/d/1hQQGHId-z1A5LOipnBXFhsU3VAMQdSe-UXvL4VPY4ps

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-18 Thread Mike Spreitzer
Manuel, and others: I am sorry, in the rush at the end of the scheduler meeting a critical fact flew from my mind: the material I distributed beforehand was intended as something I could reference during discussion in the meeting, I did not expect it to fully stand on its own. Indeed, you have

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-17 Thread Mike Spreitzer
Fixed, sorry. From: Gary Kotton gkot...@vmware.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org, Date: 09/17/2013 03:26 AM Subject:Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse Hi, The document is locked. Thanks

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-16 Thread Mike Spreitzer
I have written a brief document, with pictures. See https://docs.google.com/document/d/1hQQGHId-z1A5LOipnBXFhsU3VAMQdSe-UXvL4VPY4ps Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org

[openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-15 Thread Mike Spreitzer
I've read up on recent goings-on in the scheduler subgroup, and have some thoughts to contribute. But first I must admit that I am still a newbie to OpenStack, and still am missing some important clues. One thing that mystifies me is this: I see essentially the same thing, which I have

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-15 Thread Gary Kotton
, September 15, 2013 10:19 AM To: OpenStack Development Mailing List openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Subject: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse I've read up on recent goings-on in the scheduler subgroup, and have some

Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse

2013-09-15 Thread Mike Spreitzer
From: Gary Kotton gkot...@vmware.com ... Can you please join us at the up and coming scheduler meeting. That will give you a chance to bring up the idea's and discuss them with a larger audience. I will do so on Sep 17. Later meetings still TBD. Regards,