Hi Stan, Comments inline.
> Zane, > > I appreciate your explanations on Heat/HOT. This really makes sense. > I didn't mean to say that MuranoPL is better for Heat. Actually HOT is good > for Heat's mission. I completely acknowledge it. > I've tried to avoid comparison between languages and I'm sorry if it felt > that way. This is not productive as I don't offer you to replace HOT with > MuranoPL (although I believe that certain elements of MuranoPL syntax can be > contributed to HOT and be valuable addition there). Also people tend to > protect what they have developed and invested into and to be fair this is > what we did in this thread to great extent. > > What I'm trying to achieve is that you and the rest of Heat team understand > why it was designed the way it is. I don't feel that Murano can become > full-fledged member of OpenStack ecosystem without a bless from Heat team. > And it would be even better if we agree on certain design, join our efforts > and contribute to each other for sake of Orchestration program. Note that I feel that most people outside of the Murano project are against the idea of using a DSL. You should feel that it could block the integration in OpenStack. > I'm sorry for long mail texts written in not-so-good English and appreciate > you patience reading and answering them. > > Having said that let me step backward and explain our design decisions. > > Cloud administrators are usually technical guys that are capable of learning > HOT and writing YAML templates. They know exact configuration of their cloud > (what services are available, what is the version of OpenStack cloud is > running) and generally understands how OpenStack works. They also know about > software they intent to install. If such guy wants to install Drupal he > knows exactly that he needs HOT template describing Fedora VM with Apache + > PHP + MySQL + Drupal itself. It is not a problem for him to write such HOT > template. > > Note that such template would be designed for very particular configuration. > There are hundreds of combinations that may be used to install that Drupal - > use RHEL/Windows/etc instead of Fedora, use ngnix/IIS/etc instead of Apache, > use FastCGI instead of mod_php, PostgreSQL instead of MySQL. You may choose > to have all software on single VM or have one VM for database and another > for Drupal. There are also constraints to those combinations. For example > you cannot have Fedora + IIS on the same VM. You cannot have Apache and > Drupal on different VMs. > > So the HOT template represent fixed combination of those software components. > HOT may have input parameters like "username" or "dbImageName" but the > overall structure of template is fixed. You cannot have template that choses > whether to use Windows or Linux based on parameter value. You cannot write > HOT that accepts number of instances it allowed to create and then decide > what would be installed on each of them. This is just not needed for Heat > users. > > With Murano the picture looks the opposite. Typical Murano user is a guy who > bought account from cloud hosting vendor (cloud operator) and want to run > some software in the cloud. He may not even be aware that it is OpenStack. > He knows nothing about programming in general and Heat in particular. He > doesn't want to write YAMLs. He may not know how exactly Drupal is installed > and what components it consists of. So that's where I want to make a first stop. If your primary user is not a developer, there is no reason to introduce a DSL for security reasons. The provider can trust the code he writes, and there is no need to create a dedicated language. > So what he does is he goes to his cloud (Murano) dashboard, browses through > application catalog, finds Drupal and drags it onto his environment board > (think like Visio-style designer). He can stop at this point, click "deploy" > button and the system will deploy Drupal. In another words the system (or > maybe better to say cloud operator or application developer) decides what > set of components is going to be installed (like 1 Fedora VM for MySQL and 1 > CentOS VM for Apache-PHP-Drupal). But user may decide he wants to customize > his environment. He digs down and sees that Drupal requires database > instance and the default is MySQL. He clicks on a button to see what are > other options available for that role. > > In Heat HOT developer is the user. But in Murano those are completely > different roles. There are developers that write application definitions > (that is DSL code) and there are end users who compose environments from > those applications (components). Application developers may have nothing to > do with particular cloud their application deployed on. As for Drupal > application the developer knows that Drupal can be run with MySQL or > PostgreSQL. But there may be many compatible implementations of those DBMSes > - Galera MySQL, TroveMySQL, MMM MySQL etc. So to get a list of what > components can be placed in a database role Murano needs to look at all > applications in Application Catalog and find which of them are compatible > with MySQL and PostgreSQL so that user could choose what implementation is > better suits his needs (trade performance for high-availability etc.). > > User can go deeper and to decide that he wants that MySQL instance (this can > be 1 or more VMs depending on implementation) to be shared between Drupal > and another application in that environment (say WordPress). He can go even > deeper to VM level and decide that he wants to have WordPress, Drupal, > Apcache and slave MySQL node on one VM and MySQL master node on another. > > The ultimate goal is to create sort of construction kit for applications. > Cloud LEGO. And let users build environments of any complexity from small > components while providing reasonable default so that one-click deployment > would also be possible. And such system to be useful the design process need > to be guided and driven by UI. System must know what combination of > components are possible and what are not and not let user create Microsoft > IIS hosted on Fedora. Those goals are really nice and I'd like to have something like that in OpenStack. > This leads us to following design decisions: > > a. We need a DSL that would allow describing those small components > independently in such a way that the environment could be composed from > components written by different people that are not aware of each other You somewhat start by the end here. You haven't convinced us that you need a DSL. > b. Components need to have properties (parameters in Heat terms). Properties > may be of different types (strings, numbers, booleans etc.). There can be > constraints on property values ("not null", "length > 5"). We have those constraints in Heat, they don't require imperative constructs. > c. Components may reference (require, make use of) other components (Drupal > uses MySQL, hosted on Apache etc). There can be constraints for those > references. Present in HOTs. > d. Constraints for both properties and references can be *very* complex and > indirect. Like "IIS can be deployed on any VM who's image is known to be > Windows with version >= WinServer2003". Or "my app requires 10 to 20 Windows > instances that are all member of the same ActiveDirectory domain". Or "I > have 2 properties: minInstanceCount and maxInstanceCount and constraint > maxInstaceCount >= minInstanceCount". So we need a language to express > conditions of any complexity. We support pluggable constraints in Heat, that you can write in Python. > e. Both properties and references must be declared in a way that would make > possible for UI dashboard to discover their existence, ask user appropriate > input and validate the constraints on the fly. Possible as well. > f. Properties and especially references can have complex structure (lists, > maps, combination of them) rather than plain scalars. For example > ActiveDirectory consists of (e.g. references) *list* of domain controllers. > Constraints must respect that and be capable of validating things like > len(list) >= 2, all elements in list have some predicate true, there is at > least one element with desired set of properties etc. If not possible, HOT should be extended to support this. > g. Language need to have some sort mechanism to express polymorphism. E,g, > GaleraMySQL *is* MySQL (and thus can be used anywhere where "just MySQL" can > be). This also implies ability to have declare interfaces (class contracts) > and ability to implement those interface to that it would be possible that > two components are conform to the same interface. You don't need to have polymorphism for this. I'm mostly familiar with the mechanism used by Juju, where you *declare* an interface. It's basically metadata, and it's much simpler. > h. If we want components to be reusable and easy to write we need to have > ability to one component to inherit/extend another. Inheritance implies > ability to override and customize some methods > > i. We don't want to do deployment ourself. We don't intend to replace Heat, > Puppet etc. So we need a way to construct Heat templates that will do the > job. We need to map component's properties to > HOT parameters. This requires a language to express those bindings > > j. Because there can be lists of references (or properties) it should be > possible to express something like > for(domain_controller in self.controllers): > hot_template.insert(resources_for_particular_controller) > > or > > if self.deploy_on_windows: > generate_windows_hot_template() > else: > generate_linux_hot_template() > > or > > for i in range(self.instance_count): > hot_template.insert(generate_instance()) CloudFormation recently added support for conditionals in templates. I'm not a super fan of it, but we could presumably have those in HOT as well. > k. Need to provide most simple method to compose stack from pre-made HOT > snippets and avoid dynamic generation when its not really needed > > l. Need to have some standard format to represent view of the world that is > what was designed in dashboard. In Murano it is JSON that describes object > graph with object property values and references to another objects. This is > somewhat similar to simple HOT template but with hierarchical structure > > I think at this point it becomes obvious that above describes object-oriented > Turing-complete language. And the most important property of that language > is that *everything* is declared in a way useful to be accessed from > dashboard/app catalog. You not often see properties/references/constraints > declared in dynamic language. I've answered several times already why this > language cannot be JavaScript or Lua. And this is just on the surface. There > are many things under the hood that makes MuranoPL stand out from all > embeddable languages I know and thats for a good reason. > > And I didn't even mentioned ALM aspects, orchestration, and many many other > things that go far beyond what can be expressed by HOT-style DSL Well orchestration should be Heat's job. What I can say is that I'm not convinced. The only use-case for a DSL would be if you have to upload user-written code, but what you mentioned is a Web interface, where the user doesn't use the DSL, and the cloud provider is the developer. There is no reason in this case to have a secure environment for the code. Cheers, -- Thomas _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev