Re: [openstack-dev] [heat] non-trivial example - IBM Connections [and Murano]
From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com ... Thank you for sharing this. It looks pretty impressive. Could you, please some details about DSL syntax, if it is possible? I will respond briefly, and pass your request along to the people working on that. In the Weaver language there are distinct concepts for software configuration vs. things (such as VMs) that can host software configs. As in the current software config work, there are distinct concepts for defining a software configuration vs. applying one to some particular infrastructure. Weaver supposes that local configuration is described in Chef; Weaver adds a way of connecting chef variables across machines. So you see, there are a lot of similarities with the current work on software config, which I support. Weaver takes advantage of the power of Ruby metaprogramming to add pretty natural and concise constructs for modeling infrastructure and software config. The source does not look like it is one level off, it looks like it is describing a thing rather than describing how to build a model of the thing (even though that is what it is actually doing). Embedding in Ruby adds powerful and familiar support for abstraction and customized repetition in descriptions of distributed systems. This is going to be hard to do nicely in a language (regardless of whether it uses JSON or YAML syntax) that is primarily for data representation. One of the points of sharing this example was to show a system for which you would like such power. What is the unique problem that Murano is addressing? The current software config work can deploy multiple configs to a target. Supposing that the local config (the non-distributed base configuration management tool involved, which is open in the current software config work) is expressed using something sufficiently powerful (chef and bash are examples), the local config language can do abstraction and composition in the description of local configuration. Regards, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] non-trivial example - IBM Connections [and Murano]
Hi Mike, Thank you for clarification. I like your approach with Ruby and I think this is a right way to solve such tasks like DSL creation. In Murano we use Yaml and python just to avoid introduction of a whole new language like Ruby to OpenStack. As for software configurations in heat, we are eager to have it available for use. We use Heat in Murano and we want to pass as much as possible work to Heat engine. Murano itself is intended to be an Application Catalog for managing available application packages and focuses on UI and user experience rather then on deployment details. We still use DSL for several things, have something working while waiting for Heat implementations, and to have imperative workflow engine which is useful when you need to orchestrate complex workflows. The last part is very powerful when you need to have an explicit control on deployment sequence with conditional branches orchestration among several different instances. When Mistral will be available we plan to use its workflow engine for task orchestration. Again, thank you for sharing the work you are doing in IBM. This is very good feedback for OpenStack community and helps to understand how OpenStack components used in enterprise use cases. Thanks Georgy On Sat, Feb 8, 2014 at 10:52 AM, Mike Spreitzer mspre...@us.ibm.com wrote: From: Georgy Okrokvertskhov gokrokvertsk...@mirantis.com ... Thank you for sharing this. It looks pretty impressive. Could you, please some details about DSL syntax, if it is possible? I will respond briefly, and pass your request along to the people working on that. In the Weaver language there are distinct concepts for software configuration vs. things (such as VMs) that can host software configs. As in the current software config work, there are distinct concepts for defining a software configuration vs. applying one to some particular infrastructure. Weaver supposes that local configuration is described in Chef; Weaver adds a way of connecting chef variables across machines. So you see, there are a lot of similarities with the current work on software config, which I support. Weaver takes advantage of the power of Ruby metaprogramming to add pretty natural and concise constructs for modeling infrastructure and software config. The source does not look like it is one level off, it looks like it is describing a thing rather than describing how to build a model of the thing (even though that is what it is actually doing). Embedding in Ruby adds powerful and familiar support for abstraction and customized repetition in descriptions of distributed systems. This is going to be hard to do nicely in a language (regardless of whether it uses JSON or YAML syntax) that is primarily for data representation. One of the points of sharing this example was to show a system for which you would like such power. What is the unique problem that Murano is addressing? The current software config work can deploy multiple configs to a target. Supposing that the local config (the non-distributed base configuration management tool involved, which is open in the current software config work) is expressed using something sufficiently powerful (chef and bash are examples), the local config language can do abstraction and composition in the description of local configuration. Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Georgy Okrokvertskhov Architect, OpenStack Platform Products, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [heat] non-trivial example - IBM Connections
Hi Mike, Thank you for sharing this. It looks pretty impressive. Could you, please some details about DSL syntax, if it is possible? In our Murano project we are also solving the problem of Heat template generation. At his moment we are working on a new DSL for Murano to move from XMLs based DSL to simplified lightweight Yaml based DSL syntax. Here is an example how this new DSL looks like. https://github.com/istalker2/MuranoDsl/blob/master/meta/com.mirantis.murano.services.windows.activeDirectory.PrimaryController/manifest.yaml Using Murano DSL it is possible to combine multiple Applications defined by their manifests and then Murano engine will create Heat template for the whole environment by using Heat snippets available in Murano. Thanks Georgy On Thu, Feb 6, 2014 at 11:16 PM, Mike Spreitzer mspre...@us.ibm.com wrote: Thanks to work by several colleagues, I can share a non-trivial Heat template for a system that we have been using as an example to work. It is in the TAR archive here: https://drive.google.com/file/d/0BypF9OutGsW3Z2JqVTcxaW1BeXc/edit?usp=sharing Start at connections.yaml. Also in the archive are other files referenced from that template. The system described is a non-trival J2EE application, IBM Connections. It is a suite of collaboration applications, including things like wiki, file sharing, blogs, and directory of people. It is deployed into a system of WebSphere application servers. The servers are organized into four clusters of four servers each; each server is a VM with a single application server. The applications are partitioned among the four clusters. There is also a deployment manager, a VM and process used to manage the application servers. There is also a pair of HTTP servers --- the IBM Http Server (IHS), basically the Apache httpd. There is also a database server, running DB2. This system makes reference to an external (not deployed by the template) NFS server and an extenal LDAP server. The template describes both the VMs and the configuration of the software on them. The images used have the IBM software installed but not configured. This template (and the referenced files) were produced by automatic translation from sources expressed in Weaver into a Heat template that is ready to run. Weaver is a system for describing both infrastructure and software configuration (based on Chef). A Weaver description is a modular Ruby program that computes a model of the infrastructure and software; Weaver includes certain constructs tailored to this job, you may think of this as a DSL embedded in Ruby. The cross-machine dependencies are described abstractly in the source, connected to things in the Chef. Weaver uses an implementation that is different from the one being implemented now; Weaver's is based on communication through Zookeeper. You will see in the Heat the convolution of the user's input and Weaver's implementation (as Heat has no built in support for this mechanism). (I say these things not as a sales pitch, but to explain what you will see.) You will not be able to instantiate this template, as it has several references to external servers that are part of our environment, including those mentioned above. In fact, I have edited the external references to remove some private details. This template is presented as an example to look at. Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Georgy Okrokvertskhov Architect, OpenStack Platform Products, Mirantis http://www.mirantis.com Tel. +1 650 963 9828 Mob. +1 650 996 3284 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
