Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
FYI, I have refined my pictures at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz-GQQ1bRVgBpJdstpu0lH_TONw6g to hopefully make it clearer that I agree with the sentiment that holistic infrastructure scheduling should not be part of heat but is closely related, and to make a graphical illustration of why I prefer the ordering of functionality that I do (the boundary between software and infrastructure issues gets less squiggly). Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Excerpts from Georgy Okrokvertskhov's message of 2013-09-30 11:44:26 -0700: Hi, I am working on the OpenStack project Murano which actually had to solve the same problem with software level orchestration. Right now Murano has a DSL language which allows you to define a workflow for a complex service deployment. Murano uses Heat for infrastructure managements and actually there is a part of DSL language which allows you to generate Heat template for deployment. This is a native to OpenStack project written in Python following all OpenStack community rules. Before creating Murano we evaluated different software orchestrators like SaltStack, Chef and Puppet+mcollective. All of them have capabilities for software management but all of them are not native to OpenStack. I think it will be quite reasonable to have something under full control of OpenStack community then use something which is not native (even in programming language) to OpenStack. Here is a link to the project overview: https://wiki.openstack.org/wiki/Murano/ProjectOverview Right now Murano is concentrated on Windows services management but we also working on Linux Agent to allow Linux software configuration too. Hi! We've written some very basic tools to do server configuration for the OpenStack on OpenStack (TripleO) Deployment program. Hopefully we can avert you having to do any duplicate work and join forces. Note that configuring software and servers is not one job. The tools we have right now: os-collect-config - agent to collect data from config sources and trigger commands on changes. [1] os-refresh-config - run scripts to manage state during config changes (run-parts but more structured) [2] os-apply-config - write config files [3] [1] http://pypi.python.org/pypi/os-collect-config [2] http://pypi.python.org/pypi/os-refresh-config [3] http://pypi.python.org/pypi/os-apply-config We do not have a tool to do run-time software installation, because we are working on an image based deployment method (thus building images with diskimage-builder). IMO, there are so many good tools already written that get this job done, doing one just for the sake of it being OpenStack native is a low priority. However, a minimal thing is needed for Heat users so they can use it to install those better tools for ongoing run-time configuration. cfn-init is actually pretty good. Its only crime other than being born of Amazon is that it also does a few other jobs, namely file writing and service management. Anyway, before you run off and write an agent, I hope you will take a look at os-collect-config and considering using it. For the command to run, I recommend os-refresh-config as you can have it run a progression of config tools. For what to run in the configuration step of os-refresh-config, cfn-init would work, however there is a blueprint for a native interface that might be a bit different here: https://blueprints.launchpad.net/heat/+spec/native-tools-bootstrap-config When do you have a meeting for HOT software configuration discussion? I think we can add value here for Heat as we have already required components for software orchestration with full integration with OpenStack and Keystone in particular. Heat meets at 2000 UTC every Wednesday. TripleO meets at 2000 UTC every Tuesday. Hope to see you there! ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
On 1 October 2013 19:31, Clint Byrum cl...@fewbar.com wrote: TripleO meets at 2000 UTC every Tuesday. 1900UTC. :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Hi Mike and Zane, Le 27/09/2013 15:58, Mike Spreitzer a écrit : Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: Your diagrams clearly show scheduling happening in a separate stage to (infrastructure) orchestration, which is to say that at the point where resources are scheduled, their actual creation is in the *future*. I am not a Climate expert, but it seems to me that they have a near-identical problem to solve: how do they integrate with Heat such that somebody who has reserved resources in the past can actually create them (a) as part of a Heat stack or (b) as standalone resources, at the user's option. IMO OpenStack should solve this problem only once. If I understand correctly, what Climate adds to the party is planning allocations to happen at some specific time in the non-immediate future. A holistic infrastructure scheduler is planning allocations to happen just as soon as we can get the plans through the relevant code path, which is why I describe it as now. Climate is wide-scoped aiming to exclusively reserve any kind of resources by a certain time. This generic sentence doesn't mean Climate can't schedule things 'now': you can ask for an immediate lease (starting 'now') and youwill get the resources as of now. Climate team is actually split into two different teams, one focusing on hardware procurement and one focusing of virtual procurement. I can't speak on behalf of the 'Climate Virtual' team, but I would bet scheduling an Heat stack or aSavanna cluster will require some kind of holistic DSL, indeed. From the 'Climate Physical' POV, that could even be necessary, butyetunclear at the moment. -Sylvain If I understood your remarks correctly, we agree that there is no (known) reason that the scheduling has to occur in the middle of orchestration (which would have implied that it needed to be incorporated in some sense into Heat). If you agree that by orchestration you meant specifically infrastructure orchestration then we are agreed. If software orchestration is also in the picture then I also agree that holistic infrastructure scheduling does not *have to* go in between software orchestration and infrastructure orchestration --- but I think that's a pretty good place for it. Right, so what I'm saying is that if all those things are _stated_ in the input then there's no need to run the orchestration engine to find out what they'll be; they're already stated. Yep. Thanks, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Clint Byrum cl...@fewbar.com wrote on 01.10.2013 08:31:44 - Excerpt: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 01.10.2013 08:33 Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration) Excerpts from Georgy Okrokvertskhov's message of 2013-09-30 11:44:26 -0700: Hi, I am working on the OpenStack project Murano which actually had to solve the same problem with software level orchestration. Right now Murano has a DSL language which allows you to define a workflow for a complex service deployment. ... Hi! We've written some very basic tools to do server configuration for the OpenStack on OpenStack (TripleO) Deployment program. Hopefully we can avert you having to do any duplicate work and join forces. Note that configuring software and servers is not one job. The tools we have right now: os-collect-config - agent to collect data from config sources and trigger commands on changes. [1] os-refresh-config - run scripts to manage state during config changes (run-parts but more structured) [2] os-apply-config - write config files [3] [1] http://pypi.python.org/pypi/os-collect-config [2] http://pypi.python.org/pypi/os-refresh-config [3] http://pypi.python.org/pypi/os-apply-config We do not have a tool to do run-time software installation, because we are working on an image based deployment method (thus building images with diskimage-builder). IMO, there are so many good tools already written that get this job done, doing one just for the sake of it being OpenStack native is a low priority. However, a minimal thing is needed for Heat users so they can use it to install those better tools for ongoing run-time configuration. cfn-init is actually pretty good. Its only crime other than being born of Amazon is that it also does a few other jobs, namely file writing and service management. Right, there has been some discussion going on to find the right level of software orchestration to go into Heat. As Clint said, there are a couple of things out there already, like what the tripleO project has been doing. And there are proposals / discussions going on to see who users could include some level of software orchestration into HOT, e.g. https://wiki.openstack.org/wiki/Heat/Software-Configuration-Provider and how such constructs in HOT would align with assets already out there. So Georgy's item is another one in that direction and it would be good to find some common denominator. Anyway, before you run off and write an agent, I hope you will take a look at os-collect-config and considering using it. For the command to run, I recommend os-refresh-config as you can have it run a progression of config tools. For what to run in the configuration step of os-refresh-config, cfn-init would work, however there is a blueprint for a native interface that might be a bit different here: https://blueprints.launchpad.net/heat/+spec/native-tools-bootstrap-config When do you have a meeting for HOT software configuration discussion? I think we can add value here for Heat as we have already required components for software orchestration with full integration with OpenStack and Keystone in particular. Heat meets at 2000 UTC every Wednesday. TripleO meets at 2000 UTC every Tuesday. Hope to see you there! In addition, it looks like there will be some design sessions on that topic at the HK summit, so if you happen to be there that could be another good chance to talk. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Hi, I am working on the OpenStack project Murano which actually had to solve the same problem with software level orchestration. Right now Murano has a DSL language which allows you to define a workflow for a complex service deployment. Murano uses Heat for infrastructure managements and actually there is a part of DSL language which allows you to generate Heat template for deployment. This is a native to OpenStack project written in Python following all OpenStack community rules. Before creating Murano we evaluated different software orchestrators like SaltStack, Chef and Puppet+mcollective. All of them have capabilities for software management but all of them are not native to OpenStack. I think it will be quite reasonable to have something under full control of OpenStack community then use something which is not native (even in programming language) to OpenStack. Here is a link to the project overview: https://wiki.openstack.org/wiki/Murano/ProjectOverview Right now Murano is concentrated on Windows services management but we also working on Linux Agent to allow Linux software configuration too. When do you have a meeting for HOT software configuration discussion? I think we can add value here for Heat as we have already required components for software orchestration with full integration with OpenStack and Keystone in particular. Thanks Georgy On Fri, Sep 27, 2013 at 7:15 AM, Mike Spreitzer mspre...@us.ibm.com wrote: Sorry, I was a bit too hasty in writing the last part of my last message; I forgot to qualify software orchestration to indicate I am speaking only of its preparatory phase. I should have written: Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: ... If I understood your remarks correctly, we agree that there is no (known) reason that the scheduling has to occur in the middle of orchestration (which would have implied that it needed to be incorporated in some sense into Heat). If you agree that by orchestration you meant specifically infrastructure orchestration then we are agreed. If software orchestration preparation is also in the picture then I also agree that holistic infrastructure scheduling does not *have to* go in between software orchestration preparation and infrastructure orchestration --- but I think that's a pretty good place for it. Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Georgy Okrokvertskhov Technical Program Manager, Cloud and Infrastructure Services, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
On 25/09/13 07:03, Mike Spreitzer wrote: Zane wrote: To take the first example, wouldn't your holistic scheduler effectively have to reserve a compute instance and some directly attached block storage prior to actually creating them? Have you considered Climate rather than Heat as an integration point? I had not considered Climate. Based on recent ML traffic, I see that Climate is about scheduling into the future, whereas I am only trying to talk about scheduling for the present. OTOH, perhaps you are concerned about concurrency issues. I am too. Doing a better job on that is a big part of the revision my group is working on now. I think it can be done. I plan to post a pointer to some details soon. Your diagrams clearly show scheduling happening in a separate stage to (infrastructure) orchestration, which is to say that at the point where resources are scheduled, their actual creation is in the *future*. I am not a Climate expert, but it seems to me that they have a near-identical problem to solve: how do they integrate with Heat such that somebody who has reserved resources in the past can actually create them (a) as part of a Heat stack or (b) as standalone resources, at the user's option. IMO OpenStack should solve this problem only once. Perhaps the concern is about competition between two managers trying to manage the same resources. I think that is (a) something that can not be completely avoided and (b) impossible to do well. My preference is to focus on one manager, and make sure it tolerates surprises in a way that is not terrible. Even without competing managers, bugs and other unexpected failures will cause nasty surprises. Zane later wrote: As proposed, the software configs contain directives like 'hosted_on: server_name'. (I don't know that I'm a huge fan of this design, but I don't think the exact details are relevant in this context.) There's no non-trivial processing in the preparatory stage of software orchestration that would require it to be performed before scheduling could occur. I hope I have addressed that with my remarks above about software orchestration. If I understood your remarks correctly, we agree that there is no (known) reason that the scheduling has to occur in the middle of orchestration (which would have implied that it needed to be incorporated in some sense into Heat). Zane also wrote: Let's make sure we distinguish between doing holistic scheduling, which requires a priori knowledge of the resources to be created, and automatic scheduling, which requires psychic knowledge of the user's mind. (Did the user want to optimise for performance or availability? How would you infer that from the template?) One reason I favor holistic infrastructure scheduling is that I want its input to be richer than today's CFN templates. Like Debo, I think the input can contain the kind of information that would otherwise require mind-reading. My group has been working examples involving multiple levels of anti-co-location statements, network reachability and proximity statements, disk exclusivity statements, and statements about the presence of licensed products. Right, so what I'm saying is that if all those things are _stated_ in the input then there's no need to run the orchestration engine to find out what they'll be; they're already stated. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: Your diagrams clearly show scheduling happening in a separate stage to (infrastructure) orchestration, which is to say that at the point where resources are scheduled, their actual creation is in the *future*. I am not a Climate expert, but it seems to me that they have a near-identical problem to solve: how do they integrate with Heat such that somebody who has reserved resources in the past can actually create them (a) as part of a Heat stack or (b) as standalone resources, at the user's option. IMO OpenStack should solve this problem only once. If I understand correctly, what Climate adds to the party is planning allocations to happen at some specific time in the non-immediate future. A holistic infrastructure scheduler is planning allocations to happen just as soon as we can get the plans through the relevant code path, which is why I describe it as now. If I understood your remarks correctly, we agree that there is no (known) reason that the scheduling has to occur in the middle of orchestration (which would have implied that it needed to be incorporated in some sense into Heat). If you agree that by orchestration you meant specifically infrastructure orchestration then we are agreed. If software orchestration is also in the picture then I also agree that holistic infrastructure scheduling does not *have to* go in between software orchestration and infrastructure orchestration --- but I think that's a pretty good place for it. Right, so what I'm saying is that if all those things are _stated_ in the input then there's no need to run the orchestration engine to find out what they'll be; they're already stated. Yep. Thanks, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Sorry, I was a bit too hasty in writing the last part of my last message; I forgot to qualify software orchestration to indicate I am speaking only of its preparatory phase. I should have written: Zane Bitter zbit...@redhat.com wrote on 09/27/2013 08:24:49 AM: ... If I understood your remarks correctly, we agree that there is no (known) reason that the scheduling has to occur in the middle of orchestration (which would have implied that it needed to be incorporated in some sense into Heat). If you agree that by orchestration you meant specifically infrastructure orchestration then we are agreed. If software orchestration preparation is also in the picture then I also agree that holistic infrastructure scheduling does not *have to* go in between software orchestration preparation and infrastructure orchestration --- but I think that's a pretty good place for it. Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
On Wed, Sep 25, 2013 at 11:04:54PM +0200, Thomas Spatzier wrote: Excerpt from Clint's mail on 25.09.2013 22:23:07: I think we already have some summit suggestions for discussing HOT, it would be good to come prepared with some visions for the future of HOT so that we can hash these things out, so I'd like to see this discussion continue. Absolutely! Can those involved in the discussion check if this seems to be covered in one of the session proposal me or others posted recently, and if not raise another proposal? This is a good one to have. There is already a general HOT Discussion proposal: http://summit.openstack.org/cfp/details/78 I'd encourage everyone with HOT functionality they'd like to discuss to raise a blueprint, with a linked wiki page (or etherpad), then link the BP as a comment to that session proposal. That way we can hopefully focus the session when discussing the HOT roadmap and plans for Icehouse. As in Portland, I expect we'll need breakout sessions in addition to this, but we can organize that with those interested during the summit. Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Excerpts from Mike Spreitzer's message of 2013-09-24 22:03:21 -0700: Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and added a companion picture at https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz-GQQ1bRVgBpJdstpu0lH_TONw6g that shows an alternative. One of the things I see going on is discussion about better techniques for software orchestration than are supported in plain CFN. Plain CFN allows any script you want in userdata, and prescription of certain additional setup elsewhere in cfn metadata. But it is all mixed together and very concrete. I think many contributors would like to see something with more abstraction boundaries, not only within one template but also the ability to have modular sources. Yes please. Orchestrate things, don't configure them. That is what configuration tools are for. There is a third stealth-objective that CFN has caused to linger in Heat. That is packaging cloud applications. By allowing the 100% concrete CFN template to stand alone, users can ship the template. IMO this marrying of software assembly, config, and orchestration is a concern unto itself, and best left outside of the core infrastructure orchestration system. I work closely with some colleagues who have a particular software orchestration technology they call Weaver. It takes as input for one deployment not a single monolithic template but rather a collection of modules. Like higher level constructs in programming languages, these have some independence and can be re-used in various combinations and ways. Weaver has a compiler that weaves together the given modules to form a monolithic model. In fact, the input is a modular Ruby program, and the Weaver compiler is essentially running that Ruby program; this program produces the monolithic model as a side effect. Ruby is a pretty good language in which to embed a domain-specific language, and my colleagues have done this. The modular Weaver input mostly looks declarative, but you can use Ruby to reduce the verboseness of, e.g., repetitive stuff --- as well as plain old modularity with abstraction. We think the modular Weaver input is much more compact and better for human reading and writing than plain old CFN. This might not be obvious when you are doing the hello world example, but when you get to realistic examples it becomes clear. The Weaver input discusses infrastructure issues, in the rich way Debo and I have been advocating, as well as software. For this reason I describe it as an integrated model (integrating software and infrastructure issues). I hope for HOT to evolve to be similarly expressive to the monolithic integrated model produced by the Weaver compiler. Indeed, we're dealing with this very problem in TripleO right now. We need to be able to compose templates that vary slightly for various reasons. A ruby DSL is not something I think is ever going to happen in OpenStack. But python has its advantages for DSL as well. I have been trying to use clever tricks in yaml for a while, but perhaps we should just move to a client-side python DSL that pushes the compiled yaml/json templates into the engine. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Clint Byrum cl...@fewbar.com wrote on 25.09.2013 08:46:57: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 25.09.2013 08:48 Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration) Excerpts from Mike Spreitzer's message of 2013-09-24 22:03:21 -0700: Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at https://docs.google.com/drawings/d/ 1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and added a companion picture at https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz- GQQ1bRVgBpJdstpu0lH_TONw6g that shows an alternative. One of the things I see going on is discussion about better techniques for software orchestration than are supported in plain CFN. Plain CFN allows any script you want in userdata, and prescription of certain additional setup elsewhere in cfn metadata. But it is all mixed together and very concrete. I think many contributors would like to see something with more abstraction boundaries, not only within one template but also the ability to have modular sources. Yes please. Orchestrate things, don't configure them. That is what configuration tools are for. There is a third stealth-objective that CFN has caused to linger in Heat. That is packaging cloud applications. By allowing the 100% concrete CFN template to stand alone, users can ship the template. IMO this marrying of software assembly, config, and orchestration is a concern unto itself, and best left outside of the core infrastructure orchestration system. I work closely with some colleagues who have a particular software orchestration technology they call Weaver. It takes as input for one deployment not a single monolithic template but rather a collection of modules. Like higher level constructs in programming languages, these have some independence and can be re-used in various combinations and ways. Weaver has a compiler that weaves together the given modules to form a monolithic model. In fact, the input is a modular Ruby program, and the Weaver compiler is essentially running that Ruby program; this program produces the monolithic model as a side effect. Ruby is a pretty good language in which to embed a domain-specific language, and my colleagues have done this. The modular Weaver input mostly looks declarative, but you can use Ruby to reduce the verboseness of, e.g., repetitive stuff --- as well as plain old modularity with abstraction. We think the modular Weaver input is much more compact and better for human reading and writing than plain old CFN. This might not be obvious when you are doing the hello world example, but when you get to realistic examples it becomes clear. The Weaver input discusses infrastructure issues, in the rich way Debo and I have been advocating, as well as software. For this reason I describe it as an integrated model (integrating software and infrastructure issues). I hope for HOT to evolve to be similarly expressive to the monolithic integrated model produced by the Weaver compiler. I don't fully get this idea of HOT consuming a monolithic model produced by some compiler - be it Weaver or anything else. I thought the goal was to develop HOT in a way that users can actually write HOT, as opposed to having to use some compiler to produce some useful model. So wouldn't it make sense to make sure we add the right concepts to HOT to make sure we are able to express what we want to express and have things like composability, re-use, substitutability? Indeed, we're dealing with this very problem in TripleO right now. We need to be able to compose templates that vary slightly for various reasons. A ruby DSL is not something I think is ever going to happen in OpenStack. But python has its advantages for DSL as well. I have been trying to use clever tricks in yaml for a while, but perhaps we should just move to a client-side python DSL that pushes the compiled yaml/json templates into the engine. As said in my comment above, I would like to see us focusing on the agreement of one language - HOT - instead of yet another DSL. There are things out there that are well established (like chef or puppet), and HOT should be able to efficiently and intuitively use those things and orchestrate components built using those things. Anyway, this might be off the track that was originally discussed in this thread (i.e. holistic scheduling and so on) ... ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Clint wrote: There is a third stealth-objective that CFN has caused to linger in Heat. That is packaging cloud applications. By allowing the 100% concrete CFN template to stand alone, users can ship the template. IMO this marrying of software assembly, config, and orchestration is a concern unto itself, and best left outside of the core infrastructure orchestration system. I favor separation of concerns. I do not follow what you are suggesting about how to separate these particular concerns. Can you elaborate? Clint also wrote: A ruby DSL is not something I think is ever going to happen in OpenStack. Ruby is particularly good when the runtime scripting is done through chef or puppet, which are based on Ruby. For example, Weaver supports chef based scripting, and integrates in a convenient way. A distributed system does not all have to be written in the same language. Thomas wrote: I don't fully get this idea of HOT consuming a monolithic model produced by some compiler - be it Weaver or anything else. I thought the goal was to develop HOT in a way that users can actually write HOT, as opposed to having to use some compiler to produce some useful model. So wouldn't it make sense to make sure we add the right concepts to HOT to make sure we are able to express what we want to express and have things like composability, re-use, substitutability? I am generally suspicious of analogies, but let me offer one here. In the realm of programming languages, many have great features for modularity within one source file. These features are greatly appreciated and used. But that does not stop people from wanting to maintain sources factored into multiple files. Back to the world at hand, I do not see a conflict between (1) making a language for monoliths with sophisticated internal structure and (2) defining one or more languages for non-monolithic sources. Thomas wrote: As said in my comment above, I would like to see us focusing on the agreement of one language - HOT - instead of yet another DSL. There are things out there that are well established (like chef or puppet), and HOT should be able to efficiently and intuitively use those things and orchestrate components built using those things. Yes, it may be that our best tactic at this point is to allow multiple (2), some or all not defined through the OpenStack Foundation, while agreeing here on (1). Thomas wrote: Anyway, this might be off the track that was originally discussed in this thread (i.e. holistic scheduling and so on) ... We are engaged in a boundary-drawing and relationship-drawing exercise. I brought up this idea of a software orchestration compiler to show why I think the software orchestration preparation stage is best done earlier rather than later. Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Excerpts from Thomas Spatzier's message of 2013-09-25 00:59:44 -0700: Clint Byrum cl...@fewbar.com wrote on 25.09.2013 08:46:57: From: Clint Byrum cl...@fewbar.com To: openstack-dev openstack-dev@lists.openstack.org, Date: 25.09.2013 08:48 Subject: Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration) Excerpts from Mike Spreitzer's message of 2013-09-24 22:03:21 -0700: Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at https://docs.google.com/drawings/d/ 1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and added a companion picture at https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz- GQQ1bRVgBpJdstpu0lH_TONw6g that shows an alternative. One of the things I see going on is discussion about better techniques for software orchestration than are supported in plain CFN. Plain CFN allows any script you want in userdata, and prescription of certain additional setup elsewhere in cfn metadata. But it is all mixed together and very concrete. I think many contributors would like to see something with more abstraction boundaries, not only within one template but also the ability to have modular sources. Yes please. Orchestrate things, don't configure them. That is what configuration tools are for. There is a third stealth-objective that CFN has caused to linger in Heat. That is packaging cloud applications. By allowing the 100% concrete CFN template to stand alone, users can ship the template. IMO this marrying of software assembly, config, and orchestration is a concern unto itself, and best left outside of the core infrastructure orchestration system. I work closely with some colleagues who have a particular software orchestration technology they call Weaver. It takes as input for one deployment not a single monolithic template but rather a collection of modules. Like higher level constructs in programming languages, these have some independence and can be re-used in various combinations and ways. Weaver has a compiler that weaves together the given modules to form a monolithic model. In fact, the input is a modular Ruby program, and the Weaver compiler is essentially running that Ruby program; this program produces the monolithic model as a side effect. Ruby is a pretty good language in which to embed a domain-specific language, and my colleagues have done this. The modular Weaver input mostly looks declarative, but you can use Ruby to reduce the verboseness of, e.g., repetitive stuff --- as well as plain old modularity with abstraction. We think the modular Weaver input is much more compact and better for human reading and writing than plain old CFN. This might not be obvious when you are doing the hello world example, but when you get to realistic examples it becomes clear. The Weaver input discusses infrastructure issues, in the rich way Debo and I have been advocating, as well as software. For this reason I describe it as an integrated model (integrating software and infrastructure issues). I hope for HOT to evolve to be similarly expressive to the monolithic integrated model produced by the Weaver compiler. I don't fully get this idea of HOT consuming a monolithic model produced by some compiler - be it Weaver or anything else. I thought the goal was to develop HOT in a way that users can actually write HOT, as opposed to having to use some compiler to produce some useful model. So wouldn't it make sense to make sure we add the right concepts to HOT to make sure we are able to express what we want to express and have things like composability, re-use, substitutability? We saw this in the history of puppet in fact, where the DSL was always the problem when trying to make less-than-obvious components, and eventually puppet had to grow a full ruby dsl to avoid those mistakes and keep up with Chef's language-first approach. Indeed, we're dealing with this very problem in TripleO right now. We need to be able to compose templates that vary slightly for various reasons. A ruby DSL is not something I think is ever going to happen in OpenStack. But python has its advantages for DSL as well. I have been trying to use clever tricks in yaml for a while, but perhaps we should just move to a client-side python DSL that pushes the compiled yaml/json templates into the engine. As said in my comment above, I would like to see us focusing on the agreement of one language - HOT - instead of yet another DSL. There are things out there that are well established (like chef or puppet), and HOT should be able to efficiently and intuitively use those things and orchestrate components built using those things. Anyway, this might be off
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Excerpt from Clint's mail on 25.09.2013 22:23:07: I think we already have some summit suggestions for discussing HOT, it would be good to come prepared with some visions for the future of HOT so that we can hash these things out, so I'd like to see this discussion continue. Absolutely! Can those involved in the discussion check if this seems to be covered in one of the session proposal me or others posted recently, and if not raise another proposal? This is a good one to have. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] [scheduler] Bringing things together for Icehouse (now featuring software orchestration)
Let me elaborate a little on my thoughts about software orchestration, and respond to the recent mails from Zane and Debo. I have expanded my picture at https://docs.google.com/drawings/d/1Y_yyIpql5_cdC8116XrBHzn6GfP_g0NHTTG_W4o0R9U and added a companion picture at https://docs.google.com/drawings/d/1TCfNwzH_NBnx3bNz-GQQ1bRVgBpJdstpu0lH_TONw6g that shows an alternative. One of the things I see going on is discussion about better techniques for software orchestration than are supported in plain CFN. Plain CFN allows any script you want in userdata, and prescription of certain additional setup elsewhere in cfn metadata. But it is all mixed together and very concrete. I think many contributors would like to see something with more abstraction boundaries, not only within one template but also the ability to have modular sources. I work closely with some colleagues who have a particular software orchestration technology they call Weaver. It takes as input for one deployment not a single monolithic template but rather a collection of modules. Like higher level constructs in programming languages, these have some independence and can be re-used in various combinations and ways. Weaver has a compiler that weaves together the given modules to form a monolithic model. In fact, the input is a modular Ruby program, and the Weaver compiler is essentially running that Ruby program; this program produces the monolithic model as a side effect. Ruby is a pretty good language in which to embed a domain-specific language, and my colleagues have done this. The modular Weaver input mostly looks declarative, but you can use Ruby to reduce the verboseness of, e.g., repetitive stuff --- as well as plain old modularity with abstraction. We think the modular Weaver input is much more compact and better for human reading and writing than plain old CFN. This might not be obvious when you are doing the hello world example, but when you get to realistic examples it becomes clear. The Weaver input discusses infrastructure issues, in the rich way Debo and I have been advocating, as well as software. For this reason I describe it as an integrated model (integrating software and infrastructure issues). I hope for HOT to evolve to be similarly expressive to the monolithic integrated model produced by the Weaver compiler. In Weaver, as well as in some of the other software orchestration technologies being discussed, there is a need for some preparatory work before the infrastructure (e.g., VMs) is created. This preparatory stage begins the implementation of the software orchestration abstractions. Here is the translation from something more abstract into flat userdata and other cfn metadata. For Weaver, this stage also involves some stack-specific setup in a distinct coordination service. When the VMs finally run their userdata, the Weaver-generated scripts there use that pre-configured part of the coordination service to interact properly with each other. I think that, to a first-order approximation, the software orchestration preparatory stage commutes with holistic infrastructure scheduling. They address independent issues, and can be done in either order. That is why I have added a companion picture; the two pictures show the two orders. My claim of commutativity is limited, as I and colleagues have demonstrated only one of the two orderings; the other is just a matter of recent thought. There could be gotchas lurking in there. Between the two orderings, I have a preference for the one I first mentioned and have experience with actually running. It has the virtue of keeping related things closer together: the software orchestration compiler is next to the software orchestration preparatory stage, and the holistic infrastructure scheduling is next to the infrastructure orchestration. In response to Debo's remark about flexibility: I am happy to see an architecture that allows either ordering if it turns out that they are both viable and the community really wants that flexibility. I am not so sure we can totally give up on architecting where things go, but this level of flexibility I can understand and get behind (provided it works). Just as a LP solver is a general utility whose uses do not require architecting, I can imagine a higher level utility that solves abstract placement problems. Actually, this is not a matter of imagination. My group has been evolving such a thing for years. It is now based, as Debo recommends, on a very flexible and general optimization algorithm. But the plumbing between it and the rest of the system is significant; I would not expect many users to take on that magnitude of task. I do not really want to get into dogmatic fights over what gets labelled heat. I will leave the questions about which piece goes where in the OpenStack programs and projects to those more informed and anointed. What I am trying to
