Re: [openstack-dev] Incubation Request: Murano
Anne Gentle wrote: The reference list lives in the governance git repository: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml A bit of metadata I'd like added to the programs.yaml file is which release the project was in what status, integrated or incubated. Shall I propose a patch? Currently you have to look at italicizations on https://wiki.openstack.org/wiki/Programs to determine integrated/incubated but you can't really know what release. Yes, this has been requested in the past. Feel free to propose a patch. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
On 03/07/2014 01:53 AM, Thierry Carrez wrote: Steven Dake wrote: I'm a bit confused as well as to how a incubated project would be differentiated from a integrated project in one program. This may have already been discussed by the TC. For example, Red Hat doesn't officially support incubated projects, but we officially support (with our full sales/training/documentation/support/ plus a whole bunch of other Red Hat internalisms) Integrated projects. OpenStack vendors need a way to let customers know (through an upstream page?) what a project in a specific program's status is so we can appropriately set expectations with the community and customers. The reference list lives in the governance git repository: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml Thierry, Thanks that is a great reference. Regards -steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
On 03/07/2014 07:28 AM, Anne Gentle wrote: On Fri, Mar 7, 2014 at 2:53 AM, Thierry Carrez thie...@openstack.org mailto:thie...@openstack.org wrote: Steven Dake wrote: I'm a bit confused as well as to how a incubated project would be differentiated from a integrated project in one program. This may have already been discussed by the TC. For example, Red Hat doesn't officially support incubated projects, but we officially support (with our full sales/training/documentation/support/ plus a whole bunch of other Red Hat internalisms) Integrated projects. OpenStack vendors need a way to let customers know (through an upstream page?) what a project in a specific program's status is so we can appropriately set expectations with the community and customers. The reference list lives in the governance git repository: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml A bit of metadata I'd like added to the programs.yaml file is which release the project was in what status, integrated or incubated. Shall I propose a patch? Currently you have to look at italicizations on https://wiki.openstack.org/wiki/Programs to determine integrated/incubated but you can't really know what release. Anne Anne, I think from a vendor POV, this makes great sense for OpenStack vendors to prevent a the programs list says its integrated when it wasn't in the particular version of OpenStack being used. Regards -steve -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Steven Dake wrote: I'm a bit confused as well as to how a incubated project would be differentiated from a integrated project in one program. This may have already been discussed by the TC. For example, Red Hat doesn't officially support incubated projects, but we officially support (with our full sales/training/documentation/support/ plus a whole bunch of other Red Hat internalisms) Integrated projects. OpenStack vendors need a way to let customers know (through an upstream page?) what a project in a specific program's status is so we can appropriately set expectations with the community and customers. The reference list lives in the governance git repository: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
On Fri, Mar 7, 2014 at 2:53 AM, Thierry Carrez thie...@openstack.orgwrote: Steven Dake wrote: I'm a bit confused as well as to how a incubated project would be differentiated from a integrated project in one program. This may have already been discussed by the TC. For example, Red Hat doesn't officially support incubated projects, but we officially support (with our full sales/training/documentation/support/ plus a whole bunch of other Red Hat internalisms) Integrated projects. OpenStack vendors need a way to let customers know (through an upstream page?) what a project in a specific program's status is so we can appropriately set expectations with the community and customers. The reference list lives in the governance git repository: http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml A bit of metadata I'd like added to the programs.yaml file is which release the project was in what status, integrated or incubated. Shall I propose a patch? Currently you have to look at italicizations on https://wiki.openstack.org/wiki/Programs to determine integrated/incubated but you can't really know what release. Anne -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Steven Dake wrote: My general take is workflow would fit in the Orchestration program, but not be integrated into the heat repo specifically. It would be a different repo, managed by the same orchestration program just as we have heat-cfntools and other repositories. Figuring out how to handle the who is the core team of people responsible for program's individual repositories is the most difficult aspect of making such a merge. For example, I'd not desire a bunch of folks from Murano +2/+A heat specific repos until they understood the code base in detail, or atleast the broad architecture. I think the same think applies in reverse from the Murano perspective. Ideally folks that are core on a specific program would need to figure out how to learn how to broadly review each repo (meaning the heat devs would have to come up to speed on murano and murano devs would have to come up to speed on heat. Learning a new code base is a big commitment for an already overtaxed core team. Being in the same program means you share the same team and PTL, not necessarily that all projects under the program have the same core review team. So you could have different core reviewers for both (although I'd encourage the core for ones become core for the other, since it will facilitate behaving like a coherent team). You could also have a single core team with clear expectations set (do not approve changes for code you're not familiar with). -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
On 03/06/2014 03:15 AM, Thierry Carrez wrote: Steven Dake wrote: My general take is workflow would fit in the Orchestration program, but not be integrated into the heat repo specifically. It would be a different repo, managed by the same orchestration program just as we have heat-cfntools and other repositories. Figuring out how to handle the who is the core team of people responsible for program's individual repositories is the most difficult aspect of making such a merge. For example, I'd not desire a bunch of folks from Murano +2/+A heat specific repos until they understood the code base in detail, or atleast the broad architecture. I think the same think applies in reverse from the Murano perspective. Ideally folks that are core on a specific program would need to figure out how to learn how to broadly review each repo (meaning the heat devs would have to come up to speed on murano and murano devs would have to come up to speed on heat. Learning a new code base is a big commitment for an already overtaxed core team. Being in the same program means you share the same team and PTL, not necessarily that all projects under the program have the same core review team. So you could have different core reviewers for both (although I'd encourage the core for ones become core for the other, since it will facilitate behaving like a coherent team). You could also have a single core team with clear expectations set (do not approve changes for code you're not familiar with). This may be possible with jenkins permissions, but what I'd like to see is for a way for people familiar with each specific project to be graduated to core for that project. (eg heat or workflow). An implicit expectation do not approve doesn't totally fit, because at some point, we may want to give those folks the ability to approve via a core nomination (because they have met the core requirements) for either heat or workflow. WIthout a way of nominating for core for a specific project (within a specific program), the poor developer has no way to know when they have officially been recognized by the core team as an actual core member. I agree folks in one program need to behave as a coherent team for the Orchestration program to be successful, which means a big commitment from the existing orchestration program core members (currently heat-core) to come up to speed on the example workflow code base and community (and vice-versa). I'm a bit confused as well as to how a incubated project would be differentiated from a integrated project in one program. This may have already been discussed by the TC. For example, Red Hat doesn't officially support incubated projects, but we officially support (with our full sales/training/documentation/support/ plus a whole bunch of other Red Hat internalisms) Integrated projects. OpenStack vendors need a way to let customers know (through an upstream page?) what a project in a specific program's status is so we can appropriately set expectations with the community and customers. Regards -steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote on 05/03/2014 00:32:08: From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 05/03/2014 00:34 Subject: Re: [openstack-dev] Incubation Request: Murano Hi Thomas, Zane, Thank you for bringing TOSCA to the discussion. I think this is important topic as it will help to find better alignment or even future merge of Murano DSL and Heat templates. Murano DSL uses YAML representation too, so we can easily merge use constructions from Heat and probably any other YAML based TOSCA formats. I will be glad to join TOSCA TC. Is there any formal process for that? The first part is that your company must be a member of OASIS. If that is the case, I think you can simply go to the TC page [1] and click a button to join the TC. If your company is not yet a member, you could get in touch with the TC chairs Paul Lipton and Simon Moser and ask for the best next steps. We recently had people from GigaSpaces join the TC, and since they are also doing very TOSCA aligned implementation in Cloudify, their input will probably help a lot to advance TOSCA. I also would like to use this opportunity and start conversation with Heat team about Heat roadmap and feature set. As Thomas mentioned in his previous e-mail TOSCA topology story is quite covered by HOT. At the same time there are entities like Plans which are covered by Murano. We had discussion about bringing workflows to Heat engine before HK summit and it looks like that Heat team has no plans to bring workflows into Heat. That is actually why we mentioned Orchestration program as a potential place for Murano DSL as Heat+Murano together will cover everything which is defined by TOSCA. I remember the discussions about whether to bring workflows into Heat or not. My personal opinion is that workflows are probably out of the scope of Heat (i.e. everything but the derived orchestration flows the Heat engine implements). So there could well be a layer on-top of Heat that lets Heat deal with all topology-related declarative business and adds workflow-based orchestration around it. TOSCA could be a way to describe the respective overarching models and then hand the different processing tasks to the right engine to deal with it. I think TOSCA initiative can be a great place to collaborate. I think it will be possible then to use Simplified TOSCA format for Application descriptions as TOSCA is intended to provide such descriptions. Is there a team who are driving TOSCA implementation in OpenStack community? I feel that such team is necessary. We started to implement a TOSCA YAML to HOT converter and our team member Sahdev (IRC spzala) has recently submitted code for a new stackforge project [2]. This is very initial, but could be a point to collaborate. [1] https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca [2] https://github.com/stackforge/heat-translator Regards, Thomas Thanks Georgy On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Excerpt from Zane Bitter's message on 04/03/2014 23:16:21: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org Date: 04/03/2014 23:20 Subject: Re: [openstack-dev] Incubation Request: Murano On 04/03/14 00:04, Georgy Okrokvertskhov wrote: It so happens that the OASIS's TOSCA technical committee are working as we speak on a TOSCA Simple Profile that will hopefully make things easier to use and includes a YAML representation (the latter is great IMHO, but the key to being able to do it is the former). Work is still at a relatively early stage and in my experience they are very much open to input from implementers. Nice, I was probably also writing a mail with this information at about the same time :-) And yes, we are very much interested in feedback from implementers and open to suggestions. If we can find gaps and fill them with good proposals, now is the right time. I would strongly encourage you to get involved in this effort (by joining the TOSCA TC), and also to architect Murano in such a way that it can accept input in multiple formats (this is something we are making good progress toward in Heat). Ideally the DSL format for Murano+Heat should be a trivial translation away from the relevant parts of the YAML representation of TOSCA Simple Profile. Right, having a straight-forward translation would be really desirable. The way to get there can actually be two-fold: (1) any feedback we get from the Murano folks on the TOSCA simple profile and YAML can help us to make TOSCA capable of addressing the right use cases, and (2) on the other hand make sure the implementation goes in a direction that is in line with what TOSCA YAML will look like. cheers, Zane
Re: [openstack-dev] Incubation Request: Murano
Hi Stan, thanks for sharing your thoughts about Murano and relation to TOSCA. I have added a few comments below. From: Stan Lagun sla...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 05/03/2014 00:51 Subject: Re: [openstack-dev] Incubation Request: Murano Hi all, Completely agree with Zane. Collaboration with TOSCA TC is a way to go as Murano is very close to TOSCA. Like Murano = 0.9 * TOSCA + UI + OpenStack services integration. Let me share my thoughts on TOSCA as I read all TOSCA docs and I'm also the author of initial Murano DSL design proposal so I can probably compare them. We initially considered to just implement TOSCA before going with own DSL. There was no YAML TOSCA out there at that time, just XML version. So here's why we've wrote our own DSL: 1. TOSCA is very complex and verbose. Considering there is no production-ready tooling for TOSCA users would have to type all those tons of XML tags and namespaces and TOSCA XMLs are really hard to read and write. No one gonna do this, especially outside of Java- enterprise world Right, that's why we are doing the simple profile and YAML work now to overcome those adoption issues. 2. TOSCA has no workflow language. TOSCA draft states that the language is indeed needed and recommends using BPEL or BPMN for that matter. Right, the goal of TOSCA was not to define a new workflow language but to refer to existing ones. This does not mean, of course that other languages than BPEL or BPMN cannot be used. We still consider standardization of such a language out of scope of the TC, but if there is some widely adopted flow language being implemented, e.g. in the course of Murano, I could imagine that a community could use such a simpler language in an OpenStack environment. Ideally though, such a simpler workflow language would be translatable to a standard language like BPMN (or a subset of it) so those who have a real process engine can consume the flow descriptions. Earlier versions of Murano showed that some sort of workflow language (declarative, imperative whatever) if absolutely required for non-trivial cases. If you don't have workflow language then you have to hard-code a lot of knowledge into engine in Python. But the whole idea of AppCatalog was that users upload (share) their application templates that contain application-specific maintenance/ deployment code that is run in on common shared server (not in any particular VM) and thus capable of orchestrating all activities that are taking place on different VMs belonging to given application (for complex applications with typical enterprise SOA architecture). Besides VMs applications can talk to OpenStack services like Heat, Neutron, Trove and 3rd party services (DNS registration, NNTP, license activation service etc). Especially with the Heat so that application can have its VMs and other IaaS resources. There is a similar problem in Heat - you can express most of the basic things in HOT but once you need something really complex like accessing external API, custom load balancing or anything tricky you need to resort to Python and write custom resource plugin. And then you required to have root access to engine to install that plugin. This is not a solution for Murano as in Murano any user can upload application manifest at any time without affecting running system and without admin permissions. Now going back to TOSCA the problem with TOSCA workflows is they are not part of standard. There is no standardized way how BPEL would access TOSCA attributes and how 2 systems need to interact. This alone makes any 2 TOSCA implementations incompatible with each other rendering the whole idea of standard useless. It is not standard if there is no compatibility. We have been working on what we call a plan portability API that describes what APIs a TOSCA container has to support so that portable flows can access topology information. During the v1.0 time frame, though, we focused on the declarative part (i.e. the topology model). But, yes I agree that this part needs to be done so that also plans get portable. If you are having experience in this area, it would be great to collaborate and see if we can feed your input into the TOSCA effort. And again BPEL is heavy XML language that you don't want to have in OpenStack. Trust me, I spent significant time studying it. And if there is YAML version of TOSCA that is much more readable than XML one there is no such thing for BPEL. And I'm not aware of any adequate replacement for it I agree that BPEL and BPMN are very heavy and hard to use without tooling, so no objection on looking at a lightweight alternative in the OpenStack orchestration context. 3. It seems like nobody really using TOSCA. TOSCA standard defines exact TOSCA package format. TOSCA was designed so that people can share those packages (CSARs as TOSCA calls
Re: [openstack-dev] Incubation Request: Murano
Forgot to provide the email addresses of Paul and Simon in my last mail: paul.lip...@ca.com smo...@de.ibm.com Regards, Thomas From: Thomas Spatzier/Germany/IBM@IBMDE To: OpenStack Development Mailing List \(not for usage questions\) openstack-dev@lists.openstack.org Date: 05/03/2014 10:21 Subject: Re: [openstack-dev] Incubation Request: Murano Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote on 05/03/2014 00:32:08: From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 05/03/2014 00:34 Subject: Re: [openstack-dev] Incubation Request: Murano Hi Thomas, Zane, Thank you for bringing TOSCA to the discussion. I think this is important topic as it will help to find better alignment or even future merge of Murano DSL and Heat templates. Murano DSL uses YAML representation too, so we can easily merge use constructions from Heat and probably any other YAML based TOSCA formats. I will be glad to join TOSCA TC. Is there any formal process for that? The first part is that your company must be a member of OASIS. If that is the case, I think you can simply go to the TC page [1] and click a button to join the TC. If your company is not yet a member, you could get in touch with the TC chairs Paul Lipton and Simon Moser and ask for the best next steps. We recently had people from GigaSpaces join the TC, and since they are also doing very TOSCA aligned implementation in Cloudify, their input will probably help a lot to advance TOSCA. I also would like to use this opportunity and start conversation with Heat team about Heat roadmap and feature set. As Thomas mentioned in his previous e-mail TOSCA topology story is quite covered by HOT. At the same time there are entities like Plans which are covered by Murano. We had discussion about bringing workflows to Heat engine before HK summit and it looks like that Heat team has no plans to bring workflows into Heat. That is actually why we mentioned Orchestration program as a potential place for Murano DSL as Heat+Murano together will cover everything which is defined by TOSCA. I remember the discussions about whether to bring workflows into Heat or not. My personal opinion is that workflows are probably out of the scope of Heat (i.e. everything but the derived orchestration flows the Heat engine implements). So there could well be a layer on-top of Heat that lets Heat deal with all topology-related declarative business and adds workflow-based orchestration around it. TOSCA could be a way to describe the respective overarching models and then hand the different processing tasks to the right engine to deal with it. I think TOSCA initiative can be a great place to collaborate. I think it will be possible then to use Simplified TOSCA format for Application descriptions as TOSCA is intended to provide such descriptions. Is there a team who are driving TOSCA implementation in OpenStack community? I feel that such team is necessary. We started to implement a TOSCA YAML to HOT converter and our team member Sahdev (IRC spzala) has recently submitted code for a new stackforge project [2]. This is very initial, but could be a point to collaborate. [1] https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca [2] https://github.com/stackforge/heat-translator Regards, Thomas Thanks Georgy On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Excerpt from Zane Bitter's message on 04/03/2014 23:16:21: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org Date: 04/03/2014 23:20 Subject: Re: [openstack-dev] Incubation Request: Murano On 04/03/14 00:04, Georgy Okrokvertskhov wrote: It so happens that the OASIS's TOSCA technical committee are working as we speak on a TOSCA Simple Profile that will hopefully make things easier to use and includes a YAML representation (the latter is great IMHO, but the key to being able to do it is the former). Work is still at a relatively early stage and in my experience they are very much open to input from implementers. Nice, I was probably also writing a mail with this information at about the same time :-) And yes, we are very much interested in feedback from implementers and open to suggestions. If we can find gaps and fill them with good proposals, now is the right time. I would strongly encourage you to get involved in this effort (by joining the TOSCA TC), and also to architect Murano in such a way that it can accept input in multiple formats (this is something we are making good progress toward in Heat). Ideally the DSL format for Murano+Heat should be a trivial translation away from the relevant parts of the YAML representation of TOSCA Simple Profile. Right, having a straight
Re: [openstack-dev] Incubation Request: Murano
On 03/05/2014 02:16 AM, Thomas Spatzier wrote: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote on 05/03/2014 00:32:08: From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 05/03/2014 00:34 Subject: Re: [openstack-dev] Incubation Request: Murano Hi Thomas, Zane, Thank you for bringing TOSCA to the discussion. I think this is important topic as it will help to find better alignment or even future merge of Murano DSL and Heat templates. Murano DSL uses YAML representation too, so we can easily merge use constructions from Heat and probably any other YAML based TOSCA formats. I will be glad to join TOSCA TC. Is there any formal process for that? The first part is that your company must be a member of OASIS. If that is the case, I think you can simply go to the TC page [1] and click a button to join the TC. If your company is not yet a member, you could get in touch with the TC chairs Paul Lipton and Simon Moser and ask for the best next steps. We recently had people from GigaSpaces join the TC, and since they are also doing very TOSCA aligned implementation in Cloudify, their input will probably help a lot to advance TOSCA. I also would like to use this opportunity and start conversation with Heat team about Heat roadmap and feature set. As Thomas mentioned in his previous e-mail TOSCA topology story is quite covered by HOT. At the same time there are entities like Plans which are covered by Murano. We had discussion about bringing workflows to Heat engine before HK summit and it looks like that Heat team has no plans to bring workflows into Heat. That is actually why we mentioned Orchestration program as a potential place for Murano DSL as Heat+Murano together will cover everything which is defined by TOSCA. I remember the discussions about whether to bring workflows into Heat or not. My personal opinion is that workflows are probably out of the scope of Heat (i.e. everything but the derived orchestration flows the Heat engine implements). So there could well be a layer on-top of Heat that lets Heat deal with all topology-related declarative business and adds workflow-based orchestration around it. TOSCA could be a way to describe the respective overarching models and then hand the different processing tasks to the right engine to deal with it. My general take is workflow would fit in the Orchestration program, but not be integrated into the heat repo specifically. It would be a different repo, managed by the same orchestration program just as we have heat-cfntools and other repositories. Figuring out how to handle the who is the core team of people responsible for program's individual repositories is the most difficult aspect of making such a merge. For example, I'd not desire a bunch of folks from Murano +2/+A heat specific repos until they understood the code base in detail, or atleast the broad architecture. I think the same think applies in reverse from the Murano perspective. Ideally folks that are core on a specific program would need to figure out how to learn how to broadly review each repo (meaning the heat devs would have to come up to speed on murano and murano devs would have to come up to speed on heat. Learning a new code base is a big commitment for an already overtaxed core team. I believe expanding our scope in this way would require TC approval. The main reason I don't want workflow in the heat repo specifically is because it adds complication to Heat itself. We want Heat to be one nice tidy small set of code that does one thing really well. This makes it easy to improve, easy to deploy, and easy to learn! These reasons are why, for example, we are continuing to push the autoscaling implementation out of Heat and into a separate repository over the next 1 to 2 cycles This on the other hand won't be an expansion of scope of the Orchestration program, because we already do autoscaling, we just want to make it more consumable. Regards, -steve I think TOSCA initiative can be a great place to collaborate. I think it will be possible then to use Simplified TOSCA format for Application descriptions as TOSCA is intended to provide such descriptions. Is there a team who are driving TOSCA implementation in OpenStack community? I feel that such team is necessary. We started to implement a TOSCA YAML to HOT converter and our team member Sahdev (IRC spzala) has recently submitted code for a new stackforge project [2]. This is very initial, but could be a point to collaborate. [1] https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca [2] https://github.com/stackforge/heat-translator Regards, Thomas Thanks Georgy On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier thomas.spatz...@de.ibm.com wrote: Excerpt from Zane Bitter's message on 04/03/2014 23:16:21: From: Zane Bitter zbit...@redhat.com To: openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Hi Steve, Thank you for sharing your thoughts. I believe that is what we were trying to receive as a feedback form TC. The current definition of program actually suggest the scenario you described. A new project will appear under Orchestration umbrella. Let say there will be two project one is Heat and another is Workflow (no specific name here, probably some part of Murano). Program will have one PTL (current Heat PTL) and two separate code team for each project. That was our understanding of what we want. I am not sure that this was enough stressed out on TC meeting. There were no any intentions to add anything to Heat. Not at all. We just discussed a possibility to split current Murano App Catalog into two parts. Catalog part would go to Catalog program to land App Catalog code near Glance project and integrate them as Glance will store application packages for Murano App Catalog service. The second part of Murano related to environment processing (deployment, life cycle management, events) would go to Orchestration program as a new project like Murano workflows or Murano environment control or anything else. As I mentioned in one of the previous e-mails, we already discussed with the heat team workflows Heat before HK summit. We understand this very well that workflows will not fit Heat and we perfectly understand reasons why. I think that the good result of last TC was the official mandate to discuss alignment and integration between projects Glance, Heat, Murano and probably other projects. Right now we consider the following: 1) Continue discussion around Catalog program mission and how Murano App Catalog will fit into this program. 2) Start conversation with the Heat team in two directions: a) TOSCA and its implementation. How Murano can extend TOSCA and how TOSCA can help Murano to define an application package. Murano should reuse as much as possible from TOSCA to implement this open standard b) Define the alignment between Heat and Murano. How workflows can coexist with HOT. What will be the best way to develop both Heat and Workflows within Orchestration program. 3) Explore Application space for OpenStack. As Thierry mentioned on TC meeting, there are concerns that it is probably to early for OpenStack to make a new step up to the stack. Thanks, Georgy On Wed, Mar 5, 2014 at 7:47 PM, Steven Dake sd...@redhat.com wrote: On 03/05/2014 02:16 AM, Thomas Spatzier wrote: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote on 05/03/2014 00:32:08: From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Date: 05/03/2014 00:34 Subject: Re: [openstack-dev] Incubation Request: Murano Hi Thomas, Zane, Thank you for bringing TOSCA to the discussion. I think this is important topic as it will help to find better alignment or even future merge of Murano DSL and Heat templates. Murano DSL uses YAML representation too, so we can easily merge use constructions from Heat and probably any other YAML based TOSCA formats. I will be glad to join TOSCA TC. Is there any formal process for that? The first part is that your company must be a member of OASIS. If that is the case, I think you can simply go to the TC page [1] and click a button to join the TC. If your company is not yet a member, you could get in touch with the TC chairs Paul Lipton and Simon Moser and ask for the best next steps. We recently had people from GigaSpaces join the TC, and since they are also doing very TOSCA aligned implementation in Cloudify, their input will probably help a lot to advance TOSCA. I also would like to use this opportunity and start conversation with Heat team about Heat roadmap and feature set. As Thomas mentioned in his previous e-mail TOSCA topology story is quite covered by HOT. At the same time there are entities like Plans which are covered by Murano. We had discussion about bringing workflows to Heat engine before HK summit and it looks like that Heat team has no plans to bring workflows into Heat. That is actually why we mentioned Orchestration program as a potential place for Murano DSL as Heat+Murano together will cover everything which is defined by TOSCA. I remember the discussions about whether to bring workflows into Heat or not. My personal opinion is that workflows are probably out of the scope of Heat (i.e. everything but the derived orchestration flows the Heat engine implements). So there could well be a layer on-top of Heat that lets Heat deal with all topology-related declarative business and adds workflow-based orchestration around it. TOSCA could be a way to describe the respective overarching models and then hand the different processing tasks to the right engine to deal with it. My general take is workflow would fit in the Orchestration program, but not be integrated into the heat repo specifically. It would be a different repo
Re: [openstack-dev] Incubation Request: Murano
Hi, Here is an etherpad page with current Murano status http://etherpad.openstack.org/p/murano-incubation-status. Thanks Georgy On Mon, Mar 3, 2014 at 9:04 PM, Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote: Hi Zane, Thank you very much for this question. First of all let me highlight that Murano DSL was much inspired by TOSCA. We carefully read this standard before our movement to Murano DSL. TOSCA standard has a lot f of very well designed concepts and ideas which we reused in Murano. There is one obvious draw back of TOSCA - very heavy and verbose XML based syntax. Taking into account that OpenStack itself is clearly moving from XML based representations, it will be strange to bring this huge XML monster back on a higher level. Frankly, the current Murano workflows language is XML based and it is quite painful to write a workflows without any additional instrument like IDE. Now let me remind that TOSCA has a defined scope of its responsibility. There is a list of areas which are out of scope. For Murano it was important to see that the following items are out of TOSCA scope: Citations from [1]: ... 2. The definition of concrete plans, i.e. the definition of plans in any process modeling language like BPMN or BPEL. 3. The definition of a language for defining plans (i.e. a new process modeling language). ... Plans in TOSCA understanding is something similar to workflows. This is what we address by Murano workflow. Not let me go through TOSCA ideas and show how they are reflected in Murano. It will be a long peace of text so feel free to skip it. Taking this into account lets review what we have in Murano as an application package. Inside application package we have: 1. Application metadata which describes application, its relations and properties 2. Heat templates snippets 3. Scripts for deployment 4. Workflow definitions In TOSCA document in section 3.2.1 there are Service Templates introduced. These templates are declarative descriptions of services components and service Topologies. Service templates can be stored in catalog to be found and used by users. This service template description is abstracted from actual infrastructure implementation and each cloud provider maps this definition to actual cloud infrastructure. This is definitely a part which is already covered by Heat. The same section says the following: Making a concrete instance of a Topology Template can be done by running a corresponding Plan (so-called instantiating management plan, a.k.a. build plan). This build plan could be provided by the service developer who also creates the Service Template. This plan part which is out of scope of TOSCA is essentially what Murano adds as a part of application definition. Section 3.3 of TOSCA document introduces an new entity - artifacts. Artifact is a name for content which is needed for service deployment including (scripts, executables, binaries and images). That is why Murano has a metadata service to store artifacts as a part of application package. Moreover, Murano works with Glance team to move this metadata repository from Murano to Glance providing generic artifact repository which can be used not only by Murano but by any other services. Sections 3.4 and 3.5 explain the idea of Relationships with Compatibilities and Service Composition. Murano actually implements all these high level features. Application definition has a section with contract definitions. This contract syntax is not just a declaration of the relations and capabilities but also a way to make assertions and on-the fly type validation and conversion if needed. Section 10 reveals details of requirements. It explains that requirements can be complex: inherit each other and be abstract to define a broad set of required values. Like when service requires relation database it will require type=RDMS without assuming the actual DB implementation MySQL, PostgreSQL or MSSQL. In order to solve the problem of complex requirements, relations and service composition we introduced classes in our DSL. It was presented and discussed in this e-mail thread [3]. Murano DSL syntax allows application package writer to compose applications from existing classes by using inheritance and class properties with complex types like classes. It is possible to define a requirement with using abstract classes to define specific types of applications and services like databases, webservers and other. Using class inheritance Murano will be able to satisfy the requirement by specific object which proper parent class by checking the whole hierarchy of objects parent classes which can be abstract. I don't want to cover all entities defined in TOSCA as most important were discussed already. There are implementations of many TOSCA concepts in Murano, like class properties for TOSCA Properties, class methods for TOSCA Operations etc. TL;DR Summarizing
Re: [openstack-dev] Incubation Request: Murano
Hi all, I would like to pick up the TOSCA topic brought up by Zane in his mail below. TOSCA is in fact a standard that seems closely aligned with the concepts that Murano is implementing, so thanks Zane for bringing up this discussion. I saw Georgy's reply early today where he stated that Murano is actually heavily inspired by TOSCA, but Murano took a different path due to some drawbacks in TOSCA v1.0 (e.g. XML). I would like to point out, though, that we (TOSCA TC) are heavily working on fixing some of the usability issues that TOSCA v1.0 has. The most important point being that we are working on a YAML rendering, along with a simplified profile of TOSCA, which both shall make it easier and more attractive to use TOSCA. Much of this work has actually been inspired by the collaboration with the Heat community and the development of the HOT language. That said, I would really like the Murano folks to have a look at a current working draft of the TOSCA Simple Profile in YAML which can be found at [1]. It would be nice to get some feedback, and ideally we could even collaborate to see if we can come up with a common solution that fits everyone's needs. As far as topologies are concerned, we are trying to get TOSCA YAML and HOT well aligned so we can have an easy mapping. Sahdev from our team (IRC spzala) is actually working on a TOSCA YAML to HOT converter which he recently put on stackforge (initial version only). With Murano it would be interesting to see if we could collaborate on the plans side of TOSCA. Apart from pure DSL work, I think Murano has some other interesting items that are also interesting from a TOSCA perspective. For example, I read about a catalog that stores artifacts needed for app deployments. TOSCA also has the concept of artifacts, and we have a packaging format to transport a model and its associated artifacts. So if at some point we start thinking about importing such a TOSCA archive into a layer above today's Heat, the question is if we could use e.g. the Murano catalog for storing all content. All that said, I see some good opportunities for collaboration and it would be nice to find a common solution with good alignment between projects and to avoid duplicate efforts. BTW, Georgy, I am impressed how closely you looked at the TOSCA spec and the charter :-) [1] https://www.oasis-open.org/committees/document.php?document_id=52381wg_abbrev=tosca Greetings, Thomas Zane Bitter zbit...@redhat.com wrote on 04/03/2014 03:33:01: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org Date: 04/03/2014 03:32 Subject: Re: [openstack-dev] Incubation Request: Murano On 25/02/14 05:08, Thierry Carrez wrote: The second challenge is that we only started to explore the space of workload lifecycle management, with what looks like slightly overlapping solutions (Heat, Murano, Solum, and the openstack-compatible PaaS options out there), and it might be difficult, or too early, to pick a winning complementary set. I'd also like to add that there is already a codified OASIS standard (TOSCA) that covers application definition at what appears to be a similar level to Murano. Basically it's a more abstract version of what Heat does plus workflows for various parts of the lifecycle (e.g. backup). Heat and TOSCA folks have been working together since around the time of the Havana design summit with the aim of eventually getting a solution for launching TOSCA applications on OpenStack. Nothing is set in stone yet, but I would like to hear from the Murano folks how they are factoring compatibility with existing standards into their plans. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
On 04/03/14 00:04, Georgy Okrokvertskhov wrote: First of all let me highlight that Murano DSL was much inspired by TOSCA. We carefully read this standard before our movement to Murano DSL. TOSCA standard has a lot f of very well designed concepts and ideas which we reused in Murano. There is one obvious draw back of TOSCA - very heavy and verbose XML based syntax. Taking into account that OpenStack itself is clearly moving from XML based representations, it will be strange to bring this huge XML monster back on a higher level. Frankly, the current Murano workflows language is XML based and it is quite painful to write a workflows without any additional instrument like IDE. It so happens that the OASIS's TOSCA technical committee are working as we speak on a TOSCA Simple Profile that will hopefully make things easier to use and includes a YAML representation (the latter is great IMHO, but the key to being able to do it is the former). Work is still at a relatively early stage and in my experience they are very much open to input from implementers. I would strongly encourage you to get involved in this effort (by joining the TOSCA TC), and also to architect Murano in such a way that it can accept input in multiple formats (this is something we are making good progress toward in Heat). Ideally the DSL format for Murano+Heat should be a trivial translation away from the relevant parts of the YAML representation of TOSCA Simple Profile. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Excerpt from Zane Bitter's message on 04/03/2014 23:16:21: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org Date: 04/03/2014 23:20 Subject: Re: [openstack-dev] Incubation Request: Murano On 04/03/14 00:04, Georgy Okrokvertskhov wrote: It so happens that the OASIS's TOSCA technical committee are working as we speak on a TOSCA Simple Profile that will hopefully make things easier to use and includes a YAML representation (the latter is great IMHO, but the key to being able to do it is the former). Work is still at a relatively early stage and in my experience they are very much open to input from implementers. Nice, I was probably also writing a mail with this information at about the same time :-) And yes, we are very much interested in feedback from implementers and open to suggestions. If we can find gaps and fill them with good proposals, now is the right time. I would strongly encourage you to get involved in this effort (by joining the TOSCA TC), and also to architect Murano in such a way that it can accept input in multiple formats (this is something we are making good progress toward in Heat). Ideally the DSL format for Murano+Heat should be a trivial translation away from the relevant parts of the YAML representation of TOSCA Simple Profile. Right, having a straight-forward translation would be really desirable. The way to get there can actually be two-fold: (1) any feedback we get from the Murano folks on the TOSCA simple profile and YAML can help us to make TOSCA capable of addressing the right use cases, and (2) on the other hand make sure the implementation goes in a direction that is in line with what TOSCA YAML will look like. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Hi Thomas, Zane, Thank you for bringing TOSCA to the discussion. I think this is important topic as it will help to find better alignment or even future merge of Murano DSL and Heat templates. Murano DSL uses YAML representation too, so we can easily merge use constructions from Heat and probably any other YAML based TOSCA formats. I will be glad to join TOSCA TC. Is there any formal process for that? I also would like to use this opportunity and start conversation with Heat team about Heat roadmap and feature set. As Thomas mentioned in his previous e-mail TOSCA topology story is quite covered by HOT. At the same time there are entities like Plans which are covered by Murano. We had discussion about bringing workflows to Heat engine before HK summit and it looks like that Heat team has no plans to bring workflows into Heat. That is actually why we mentioned Orchestration program as a potential place for Murano DSL as Heat+Murano together will cover everything which is defined by TOSCA. I think TOSCA initiative can be a great place to collaborate. I think it will be possible then to use Simplified TOSCA format for Application descriptions as TOSCA is intended to provide such descriptions. Is there a team who are driving TOSCA implementation in OpenStack community? I feel that such team is necessary. Thanks Georgy On Tue, Mar 4, 2014 at 2:36 PM, Thomas Spatzier thomas.spatz...@de.ibm.comwrote: Excerpt from Zane Bitter's message on 04/03/2014 23:16:21: From: Zane Bitter zbit...@redhat.com To: openstack-dev@lists.openstack.org Date: 04/03/2014 23:20 Subject: Re: [openstack-dev] Incubation Request: Murano On 04/03/14 00:04, Georgy Okrokvertskhov wrote: It so happens that the OASIS's TOSCA technical committee are working as we speak on a TOSCA Simple Profile that will hopefully make things easier to use and includes a YAML representation (the latter is great IMHO, but the key to being able to do it is the former). Work is still at a relatively early stage and in my experience they are very much open to input from implementers. Nice, I was probably also writing a mail with this information at about the same time :-) And yes, we are very much interested in feedback from implementers and open to suggestions. If we can find gaps and fill them with good proposals, now is the right time. I would strongly encourage you to get involved in this effort (by joining the TOSCA TC), and also to architect Murano in such a way that it can accept input in multiple formats (this is something we are making good progress toward in Heat). Ideally the DSL format for Murano+Heat should be a trivial translation away from the relevant parts of the YAML representation of TOSCA Simple Profile. Right, having a straight-forward translation would be really desirable. The way to get there can actually be two-fold: (1) any feedback we get from the Murano folks on the TOSCA simple profile and YAML can help us to make TOSCA capable of addressing the right use cases, and (2) on the other hand make sure the implementation goes in a direction that is in line with what TOSCA YAML will look like. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Georgy Okrokvertskhov Architect, OpenStack Platform Products, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Hi all, Completely agree with Zane. Collaboration with TOSCA TC is a way to go as Murano is very close to TOSCA. Like Murano = 0.9 * TOSCA + UI + OpenStack services integration. Let me share my thoughts on TOSCA as I read all TOSCA docs and I'm also the author of initial Murano DSL design proposal so I can probably compare them. We initially considered to just implement TOSCA before going with own DSL. There was no YAML TOSCA out there at that time, just XML version. So here's why we've wrote our own DSL: 1. TOSCA is very complex and verbose. Considering there is no production-ready tooling for TOSCA users would have to type all those tons of XML tags and namespaces and TOSCA XMLs are really hard to read and write. No one gonna do this, especially outside of Java-enterprise world 2. TOSCA has no workflow language. TOSCA draft states that the language is indeed needed and recommends using BPEL or BPMN for that matter. Earlier versions of Murano showed that some sort of workflow language (declarative, imperative whatever) if absolutely required for non-trivial cases. If you don't have workflow language then you have to hard-code a lot of knowledge into engine in Python. But the whole idea of AppCatalog was that users upload (share) their application templates that contain application-specific maintenance/deployment code that is run in on common shared server (not in any particular VM) and thus capable of orchestrating all activities that are taking place on different VMs belonging to given application (for complex applications with typical enterprise SOA architecture). Besides VMs applications can talk to OpenStack services like Heat, Neutron, Trove and 3rd party services (DNS registration, NNTP, license activation service etc). Especially with the Heat so that application can have its VMs and other IaaS resources. There is a similar problem in Heat - you can express most of the basic things in HOT but once you need something really complex like accessing external API, custom load balancing or anything tricky you need to resort to Python and write custom resource plugin. And then you required to have root access to engine to install that plugin. This is not a solution for Murano as in Murano any user can upload application manifest at any time without affecting running system and without admin permissions. Now going back to TOSCA the problem with TOSCA workflows is they are not part of standard. There is no standardized way how BPEL would access TOSCA attributes and how 2 systems need to interact. This alone makes any 2 TOSCA implementations incompatible with each other rendering the whole idea of standard useless. It is not standard if there is no compatibility. And again BPEL is heavy XML language that you don't want to have in OpenStack. Trust me, I spent significant time studying it. And if there is YAML version of TOSCA that is much more readable than XML one there is no such thing for BPEL. And I'm not aware of any adequate replacement for it 3. It seems like nobody really using TOSCA. TOSCA standard defines exact TOSCA package format. TOSCA was designed so that people can share those packages (CSARs as TOSCA calls them) between various TOSCA implementations. I've tried to google those packages. It took me like a hour to find even most trivial CSAR example. And it was on OASIS site. 4. There is no reference TOSCA implementation. No test suite. There is no way to check your implementation is really TOSCA-compatible. And no one to ask questions 5. TOSCA is very immature. They didn't even made XML version used and already working on YAML version that is not compatible with current draft 6. TOSCA is too restrictive and verbose in some areas while leaving a lot of white spaces in others So we decided to go with our own DSL that would eliminate all the problems above. And I personally feel that Murano is how TOSCA should be looked like if it was designed for OpenStack. Murano is perfectly aligned with Heat and other OpenStack services and practices. It is very Python-like. It is easy to read and write once you learn the basics. It is more universal and less restrictive than TOSCA. You can do much more than you could ever do in TOSCA. And it is very extensible. I hope that Murano and its ideas will find a way into OpenStack community :) On Wed, Mar 5, 2014 at 2:16 AM, Zane Bitter zbit...@redhat.com wrote: On 04/03/14 00:04, Georgy Okrokvertskhov wrote: First of all let me highlight that Murano DSL was much inspired by TOSCA. We carefully read this standard before our movement to Murano DSL. TOSCA standard has a lot f of very well designed concepts and ideas which we reused in Murano. There is one obvious draw back of TOSCA - very heavy and verbose XML based syntax. Taking into account that OpenStack itself is clearly moving from XML based representations, it will be strange to bring this huge XML monster back on a higher level. Frankly, the current Murano workflows language is
Re: [openstack-dev] Incubation Request: Murano
On 25/02/14 05:08, Thierry Carrez wrote: The second challenge is that we only started to explore the space of workload lifecycle management, with what looks like slightly overlapping solutions (Heat, Murano, Solum, and the openstack-compatible PaaS options out there), and it might be difficult, or too early, to pick a winning complementary set. I'd also like to add that there is already a codified OASIS standard (TOSCA) that covers application definition at what appears to be a similar level to Murano. Basically it's a more abstract version of what Heat does plus workflows for various parts of the lifecycle (e.g. backup). Heat and TOSCA folks have been working together since around the time of the Havana design summit with the aim of eventually getting a solution for launching TOSCA applications on OpenStack. Nothing is set in stone yet, but I would like to hear from the Murano folks how they are factoring compatibility with existing standards into their plans. cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Hi Zane, Thank you very much for this question. First of all let me highlight that Murano DSL was much inspired by TOSCA. We carefully read this standard before our movement to Murano DSL. TOSCA standard has a lot f of very well designed concepts and ideas which we reused in Murano. There is one obvious draw back of TOSCA - very heavy and verbose XML based syntax. Taking into account that OpenStack itself is clearly moving from XML based representations, it will be strange to bring this huge XML monster back on a higher level. Frankly, the current Murano workflows language is XML based and it is quite painful to write a workflows without any additional instrument like IDE. Now let me remind that TOSCA has a defined scope of its responsibility. There is a list of areas which are out of scope. For Murano it was important to see that the following items are out of TOSCA scope: Citations from [1]: ... 2. The definition of concrete plans, i.e. the definition of plans in any process modeling language like BPMN or BPEL. 3. The definition of a language for defining plans (i.e. a new process modeling language). ... Plans in TOSCA understanding is something similar to workflows. This is what we address by Murano workflow. Not let me go through TOSCA ideas and show how they are reflected in Murano. It will be a long peace of text so feel free to skip it. Taking this into account lets review what we have in Murano as an application package. Inside application package we have: 1. Application metadata which describes application, its relations and properties 2. Heat templates snippets 3. Scripts for deployment 4. Workflow definitions In TOSCA document in section 3.2.1 there are Service Templates introduced. These templates are declarative descriptions of services components and service Topologies. Service templates can be stored in catalog to be found and used by users. This service template description is abstracted from actual infrastructure implementation and each cloud provider maps this definition to actual cloud infrastructure. This is definitely a part which is already covered by Heat. The same section says the following: Making a concrete instance of a Topology Template can be done by running a corresponding Plan (so-called instantiating management plan, a.k.a. build plan). This build plan could be provided by the service developer who also creates the Service Template. This plan part which is out of scope of TOSCA is essentially what Murano adds as a part of application definition. Section 3.3 of TOSCA document introduces an new entity - artifacts. Artifact is a name for content which is needed for service deployment including (scripts, executables, binaries and images). That is why Murano has a metadata service to store artifacts as a part of application package. Moreover, Murano works with Glance team to move this metadata repository from Murano to Glance providing generic artifact repository which can be used not only by Murano but by any other services. Sections 3.4 and 3.5 explain the idea of Relationships with Compatibilities and Service Composition. Murano actually implements all these high level features. Application definition has a section with contract definitions. This contract syntax is not just a declaration of the relations and capabilities but also a way to make assertions and on-the fly type validation and conversion if needed. Section 10 reveals details of requirements. It explains that requirements can be complex: inherit each other and be abstract to define a broad set of required values. Like when service requires relation database it will require type=RDMS without assuming the actual DB implementation MySQL, PostgreSQL or MSSQL. In order to solve the problem of complex requirements, relations and service composition we introduced classes in our DSL. It was presented and discussed in this e-mail thread [3]. Murano DSL syntax allows application package writer to compose applications from existing classes by using inheritance and class properties with complex types like classes. It is possible to define a requirement with using abstract classes to define specific types of applications and services like databases, webservers and other. Using class inheritance Murano will be able to satisfy the requirement by specific object which proper parent class by checking the whole hierarchy of objects parent classes which can be abstract. I don't want to cover all entities defined in TOSCA as most important were discussed already. There are implementations of many TOSCA concepts in Murano, like class properties for TOSCA Properties, class methods for TOSCA Operations etc. TL;DR Summarizing everything I wrote above, I can say that Murano is a lightweight implementation of TOSCA ideas without XML syntax and declarative parts which are covered by Heat. This is definitely not an implementation of TOSCA standard as is, but I am not sure that this is event necessary. If TOSCA standard will be
Re: [openstack-dev] Incubation Request: Murano
Georgy Okrokvertskhov wrote: [...] As you can see this is complicated topic with a number of possible solutions. What Murano team is seeking to achieve is to get feedback of community and TC on the most appropriate way to structure the governance model for the project. And that should make for an interesting discussion. Like you say it's a complicated topic, and we need a coherent and integrated solution in the end. I think there are two unique challenges here. The first is that Murano looks more like a complete solution (rather than an infrastructure piece), which makes it span features that we currently find in separate programs (user-facing catalog/discovery, workload lifecycle management). My point in my earlier email is that we could maybe bring that new functionality to OpenStack as a whole without necessarily introducing a new component, by breaking those features into the existing projects. Because in the end, OpenStack is not a collection of overlapping products, it's a set of complementary components. The second challenge is that we only started to explore the space of workload lifecycle management, with what looks like slightly overlapping solutions (Heat, Murano, Solum, and the openstack-compatible PaaS options out there), and it might be difficult, or too early, to pick a winning complementary set. -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Mark Washenberger wrote: Prior to this email, I was imagining that we would expand the Images program to go beyond storing just block device images, and into more structured items like whole Nova instance templates, Heat templates, and Murano packages. In this scheme, Glance would know everything there is to know about a resource--its type, format, location, size, and relationships to other resources--but it would not know or offer any links for how a resource is to be used. I'm a bit uncomfortable as well. Part of our role at the Technical Committee is to make sure additions do not overlap in scope and make sense as a whole. Murano seems to cover two functions. The first one is publishing, cataloging and discovering software stacks. The second one is to deploy those software stacks and potentially manage their lifecycle. In the OpenStack integrated release we already have Glance as a publication/catalog/discovery component and Heat as the workload orchestration end. Georgy clearly identified those two facets, since the incubation request lists those two programs as potential homes for Murano. The problem is, Orchestration doesn't care about the Catalog part of Murano, and Glance doesn't care about the Orchestration part of Murano. Murano spans the scope of two established programs. It's not different enough to really warrant its own program, and it's too monolithic to fit in our current landscape. I see two ways out: Murano can continue to live as a separate application that lives on top of OpenStack and consumes various OpenStack components. Or its functionality can be split and subsumed by Glance and Heat, with Murano developers pushing it there. There seems to be interest in both those programs to add features that Murano covers. The question is, could we replicate Murano's featureset completely in those existing components ? Or is there anything Murano-unique that wouldn't fit in existing projects ? -- Thierry Carrez (ttx) ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
On Mon, 2014-02-24 at 11:24 +0100, Thierry Carrez wrote: Mark Washenberger wrote: Prior to this email, I was imagining that we would expand the Images program to go beyond storing just block device images, and into more structured items like whole Nova instance templates, Heat templates, and Murano packages. In this scheme, Glance would know everything there is to know about a resource--its type, format, location, size, and relationships to other resources--but it would not know or offer any links for how a resource is to be used. I'm a bit uncomfortable as well. Part of our role at the Technical Committee is to make sure additions do not overlap in scope and make sense as a whole. Murano seems to cover two functions. The first one is publishing, cataloging and discovering software stacks. The second one is to deploy those software stacks and potentially manage their lifecycle. In the OpenStack integrated release we already have Glance as a publication/catalog/discovery component and Heat as the workload orchestration end. Georgy clearly identified those two facets, since the incubation request lists those two programs as potential homes for Murano. The problem is, Orchestration doesn't care about the Catalog part of Murano, and Glance doesn't care about the Orchestration part of Murano. Murano spans the scope of two established programs. It's not different enough to really warrant its own program, and it's too monolithic to fit in our current landscape. I see two ways out: Murano can continue to live as a separate application that lives on top of OpenStack and consumes various OpenStack components. Or its functionality can be split and subsumed by Glance and Heat, with Murano developers pushing it there. There seems to be interest in both those programs to add features that Murano covers. There is a third component: the UI pieces. This naturally would belong in the UX program. The question is, could we replicate Murano's featureset completely in those existing components ? Or is there anything Murano-unique that wouldn't fit in existing projects ? Outside of its innovative UX form-construction component, the biggest thing that makes Murano unique (IMO) is its use of flow control constructs in its DSL. If I'm not mistaken, the Heat community has made it clear that they do not intend to introduce flow control constructs into HOT, and so there would be this piece that would need to live outside of Heat, but still in the Orchestration program. So, I believe that there is still a compelling reason for Murano to exist as a separate project within the Orchestration program, with a mission to provide a higher level DSL for deployment of complex application topologies that includes flow control contructs. And then someone will ask well, isn't that partly what Solum is designed for?. And we're back to a similar discussion ;) Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Incubation Request: Murano
Hi Thierry, Let me clarify the situation with existing programs and projects overlap. First of all, I would like to separate questions about what program Murano as a project can fit and about any overlap with existing projects in the official programs. We position Application Catalog as a project that provides functionality for application publishing, distribution and management. What we were suggesting is that Murano as a project might fit in a Catalog program which technically does not exist yet, but this could possibly be one of the ways how current Image program might evolve. On the project level, I don't see any overlap with existing Glance project functionality. The Murano team is actively working with the Glance team to define a roadmap towards more generic metadata repository functionality for storing metadata not only images but other artifacts like application packages, Heat templates etc. Once this roadmap is realized, Murano would use a Glance repository for storing metadata. Let me also explain why we listed Orchestration program as a possible place for Murano. The Orchestration mission states that the goal is to create a human- and machine-accessible service for managing the entire lifecycle of infrastructure and applications within OpenStack clouds. An application catalog provides self-service capabilities for a cloud user to manage applications on top of the cloud. In this form, the mission of the Orchestration program can be applied to the Murano project. The fact that Murano fits the Orchestration mission does not mean that there is an overlap with existing projects in this program. Murano uses Heat to perform actual deployment. In this sense Murano does not deploy most things directly. Murano uses application definition to generate a Heat template from Heat template snippets. A good analogy here is the TripleO project which combines Heat templates based on desired OpenStack configuration and uses Heat to perform actual work. The key functionality in Murano is an application package definition. An application package consists of a UI definition, metadata to control its appearance in the Catalog, requirements which help Catalog to find a required or dependent applications, rules to control Heat template definitions from snippets and scripts which are part of application packages too. An essential requirement is to keep Murano project solid as it covers all aspects of working with application starting from UI appearance and ending with controlling a heat template-based deployment. I also wouldn't completely disregard an option to create a new program combining few projects to cover aspects of application management. As you can see this is complicated topic with a number of possible solutions. What Murano team is seeking to achieve is to get feedback of community and TC on the most appropriate way to structure the governance model for the project. Thanks Georgy On Mon, Feb 24, 2014 at 2:24 AM, Thierry Carrez thie...@openstack.org wrote: Mark Washenberger wrote: Prior to this email, I was imagining that we would expand the Images program to go beyond storing just block device images, and into more structured items like whole Nova instance templates, Heat templates, and Murano packages. In this scheme, Glance would know everything there is to know about a resource--its type, format, location, size, and relationships to other resources--but it would not know or offer any links for how a resource is to be used. I'm a bit uncomfortable as well. Part of our role at the Technical Committee is to make sure additions do not overlap in scope and make sense as a whole. Murano seems to cover two functions. The first one is publishing, cataloging and discovering software stacks. The second one is to deploy those software stacks and potentially manage their lifecycle. In the OpenStack integrated release we already have Glance as a publication/catalog/discovery component and Heat as the workload orchestration end. Georgy clearly identified those two facets, since the incubation request lists those two programs as potential homes for Murano. The problem is, Orchestration doesn't care about the Catalog part of Murano, and Glance doesn't care about the Orchestration part of Murano. Murano spans the scope of two established programs. It's not different enough to really warrant its own program, and it's too monolithic to fit in our current landscape. I see two ways out: Murano can continue to live as a separate application that lives on top of OpenStack and consumes various OpenStack components. Or its functionality can be split and subsumed by Glance and Heat, with Murano developers pushing it there. There seems to be interest in both those programs to add features that Murano covers. The question is, could we replicate Murano's featureset completely in those existing components ? Or is there anything Murano-unique that wouldn't fit in existing
Re: [openstack-dev] Incubation Request: Murano
Hi Georgy, Thanks for all your efforts putting this together. In the incubation request, one of the proposals is to include Murano under an expanded scope of the Images program, renaming it the Catalog program. I've been extremely pleased with the help of you and your colleagues in helping to define the broader role for Glance as a more general artifact repository. However, the proposal to bring all of Murano under the expanded Images program strains my current understanding of how Images needs to expand in scope. Prior to this email, I was imagining that we would expand the Images program to go beyond storing just block device images, and into more structured items like whole Nova instance templates, Heat templates, and Murano packages. In this scheme, Glance would know everything there is to know about a resource--its type, format, location, size, and relationships to other resources--but it would not know or offer any links for how a resource is to be used. For example, Glance would know the virtual size, the storage format, and all the data associated with a disk image. But it would not necessarily know anything about a user's ability to either boot that disk image in Nova or to populate a Cinder volume with the image data. I think you make a very good point, however. In an orchestrated view of the cloud, the most usable approach is to have links directly from a resource to the actions you can perform with the resource. In pseudocode, image.boot() rather than nova.boot(image). In this more expansive view of the Catalog, I think it would make sense to include Murano entirely as part of the Catalog program. However, this change seems to me to imply a significant architectural shift for OpenStack in general, and I'm just not quite comfortable endorsing it. I'm very eager to hear other opinions about this question--perhaps I am simply not understanding the advantages. In any case, I hope these notes help to frame the question of where Murano can best fit. Thanks again, markwash On Thu, Feb 20, 2014 at 10:35 AM, Georgy Okrokvertskhov gokrokvertsk...@mirantis.com wrote: All, Murano is the OpenStack Application Catalog service which has been developing on stackforge almost 11 months. Murano has been presented on HK summit on unconference track and now we would like to apply for incubation during Juno release. As the first step we would like to get feedback from TC on Murano readiness from OpenStack processes standpoint as well as open up conversation around mission and how it fits OpenStack ecosystem. Murano incubation request form is here: https://wiki.openstack.org/wiki/Murano/Incubation As a part of incubation request we are looking for an advice from TC on the governance model for Murano. Murano may potentially fit to the expanding scope of Image program, if it will be transformed to Catalog program. Also it potentially fits Orchestration program, and as a third option there might be a value in creation of a new standalone Application Catalog program. We have pros and cons analysis in Murano Incubation request form. Murano team has been working on Murano as a community project. All our code and bugs/specs are hosted at OpenStack Gerrit and Launchpad correspondingly. Unit tests and all pep8/hacking checks are run at OpenStack Jenkins and we have integration tests running at our own Jenkins server for each patch set. Murano also has all necessary scripts for devstack integration. We have been holding weekly IRC meetings for the last 7 months and discussing architectural questions there and in openstack-dev mailing lists as well. Murano related information is here: Launchpad: https://launchpad.net/murano Murano Wiki page: https://wiki.openstack.org/wiki/Murano Murano Documentation: https://wiki.openstack.org/wiki/Murano/Documentation Murano IRC channel: #murano With this we would like to start the process of incubation application review. Thanks Georgy -- Georgy Okrokvertskhov Architect, OpenStack Platform Products, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
