On 19/03/14 17:13, Stan Lagun wrote:
On Wed, Mar 19, 2014 at 9:18 PM, Zane Bitter <zbit...@redhat.com
<mailto:zbit...@redhat.com>> wrote:
On 19/03/14 05:00, Stan Lagun wrote:
Steven,
Agree with your opinion on HOT expansion. I see that inclusion of
imperative workflows and ALM would require major Heat redesign and
probably would be impossible without loosing compatibility with
previous
HOT syntax. It would blur Heat mission, confuse current users
and rise a
lot of questions what should and what should not be in Heat.
Thats why
we chose to built a system on top of Heat rather then expending HOT.
+1, I agree (as we have discussed before) that it would be a mistake
to shoehorn workflow stuff into Heat. I do think we should implement
the hooks I mentioned at the start of this thread to allow tighter
integration between Heat and a workflow engine (i.e. Mistral).
So building a system on top of Heat is good. Building it on top of
Mistral as well would also be good, and that was part of the
feedback from the TC.
To me, building on top means building on top of the languages (which
users will have to invest a lot of work in learning) as well, rather
than having a completely different language and only using the
underlying implementation(s).
1. Murano application developers (publishers) are using Heat directly.
It is not the case that Murano executes some HOT templates under the
hood but HOT templates are essential part of application definition.
Developers still write HOT templates in HOT syntax
2. Please get me right here. It is not that we wanted to develop another
language just for fun or to stand out from community. It is not that we
wrote DSL and then started to think how we gonna use it and prove to
others that is needed. We completely understand all concerns regarding
new language. The only reason is that we had very concrete list of
problems and use cases that we wanted to address in Murano. We did
investigate on using HOT, Mistral, JavaScript, Lua and BPEL and we found
overwhelming obstacles with each of those approaches. It don't pretend
that having own DSL is good. Just that not having it is much worse. I'm
also not a HOT expert as you are and thus can be (partially) wrong about
HOT and not aware of some of its power features. If so as a technical
guys we would quickly come to consensus.
Now I would like to clarify why have we chosen imperative
approach with DSL.
You see a DSL as an alternative to HOT but it is not. DSL is
alternative
to Python-encoded resources in Heat (heat/engine/resources/*.py).
Imagine how Heat would look like if you let untrusted users to
upload
Python plugins to Heat engine and load them on the fly. Heat
resources
are written in Python which is imperative language. So that
MuranoPL for
the same reason.
We had this exact problem in Heat, and we decided to solve it
with... HOT:
http://lists.openstack.org/__pipermail/openstack-dev/2013-__April/007989.html
<http://lists.openstack.org/pipermail/openstack-dev/2013-April/007989.html>
If I may be so bold as to quote myself:
"...no cloud operator in the world - not even your friendly local IT
department - is going to let users upload Python code to run
in-memory in their orchestration engine along with all of the other
users' code.
"If only there were some sort of language for defining OpenStack
services that could be safely executed by users...
"Of course that's exactly what we're all about on this project :).
So my proposal is to allow users to define their own resource types
using a Heat template."
Excellent quotes! I can sign under each of them. No one would ever allow
to upload Python code so we haven't used Python (and it is not the only
reason not to use it)
There's a fine line between a domain-specific programming language (as
opposed to a domain-specific modelling language like HOT) and a
general-purpose programming language. The reason no-one would run
untrusted Python code is that Python is a programming language, not that
it is general-purpose.
"If only there were some sort of language for defining OpenStack
services that could be safely executed by users..." - we created such
language.
And we found we already had one, that we successfully re-used for the
purpose :)
It is exactly what you wanted to have then and now when it
exists you say it is useless. How this ironic.
I'm not saying it's useless; I'm saying that it's *too* powerful for the
job. It's clear that you guys have done amazing work and I don't want to
criticise it, especially after having become aware of it only very late,
but I also have serious reservations as to whether this is the right
solution for OpenStack.
It may be conceitedly to
assume that MuranoPL is 100% is safe for all kind of attacks exists. It
is not. But I know how to make it, I know it is doable because it was
designed to be such. MuranoPL code is completely isolated from
underlying operating system and its resources (sockets, files etc.). It
is not translated to Python or any other language and thus cannot be a
subject to code injection attacks. It is 100% interpreted and absolutely
everything about the interpreter can be customized and controlled. You
can meter everything, have ACL for every single aspect, you can limit
MuranoPL execution time, number of executed instructions. Almost
anything is possible. With Python even if you make it somehow 100%
secure it would be hard to convince people it is. But there is no such
problem with YAML DSL :)
This is all good news. I still worry that you have a very large
footprint to secure.
The only thing I cannot agree with (but I hope you prove me wrong) is
that HOT is a solution for the problem. Surely you can do a lot of
things in HOT. But there is even a bigger list of things that you can't.
Otherwise why not Heat resources are written in Python and not in HOT?
Because there have to be some base types that you can use as building
blocks. In the case of Heat, those base types are the set of things that
you can create by talking to OpenStack APIs authenticated with the
user's token. In the case of Mistral, I would expect it to be the set of
actions that you can take by talking to OpenStack APIs authenticated
with the user's token. And in the case of Murano, I would expect it to
be the union of those two.
Can you implement auto-scaling or load-balancer using HOT alone without
Python code?
We can and do implement the load-balancer using a Heat template :)
You could certainly implement autoscaling yourself, but you'd have to
spin up a Compute server to run it on as part of the template. Which is
sort of my point.
And what if application author (the guy that has absolutely
no control over the cloud where it going to be deployed. Even not aware
of its existence) needs his app to be autoscaled using Microsoft stack
technologies?
I'm not sure what this means, and I feel like we're getting off-topic.
What if cloud operator wants to have additional control
on scaling strategy so that it can be bound to his billing system?
The operator can install whatever plugins they want.
What
if he wants that auto-scaling would be based on input from his existing
Nagios infrastructure rather then Ceilometer?
This is supported already in autoscaling. Ceilometer just hits a URL for
an alarm, but you don't have to configure it this way. Anything can hit
the URL.
And this is a good example for our general approach - we provide a way
that works using built-in OpenStack services and a hook that allows you
to customise it with your own service, running on your own machine
(whether that be an actual machine or an OpenStack Compute server). What
we *don't* do is provide a way to upload your own code that we then
execute for you as some sort of secondary Compute service.
One can imagine any number
of use cases that just cannot be addressed by HOT without HOT becoming
another MuranoPL. So the only option you left with for such cases is
writing custom Python plugin. Now I don't want to reimplement in Python
the whole auto-scaling logic just because I need some customization. But
if I just patch existing Heat code then I would have to maintain my
custom Heat fork and have all sorts of problems when next version of
OpenStack arrives.There is a risk of introducing security breaches with
my new code that would be executed with root account permissions.
I certainly hope *nobody* is running as root.
The
changes you would make to Heat would affect all tenants and may break
existing resources. And making application depend on presence of some
custom plugin makes it not portable across different clouds.
My point here is that HOT is not sufficient for coding new resource
types that are not superposition of existing resources. How can you do
that with a language that cannot even do the simplest arithmetics
operations?
Everything is a combination of existing resources, because the set of
existing resources is the set of things which the operator provides
as-a-Service. The set of things that the operator provides as a service
plus the set of things that you can implement yourself on your own
server (virtual or not) covers the entire universe of things. What you
appear to be suggesting is that OpenStack must provide
*Everything*-as-a-Service by allowing users to write their own services
and have the operator execute them as-a-Service. This would be a
breathtakingly ambitious undertaking, and I don't mean that in a good way.
Which we did:
https://blueprints.launchpad.__net/heat/+spec/provider-__resource
<https://blueprints.launchpad.net/heat/+spec/provider-resource>
We want application authors to be able to express application
deployment
and maintenance logic of any complexity. This may involve
communication
with 3rd party REST services (APIs of applications being deployed,
external services like DNS server API, application licensing
server API,
billing systems, some hardware component APIs etc) and internal
OpenStack services like Trove, Sahara, Marconi and others including
those that are not incubated yet and those to come in the
future. You
cannot have such things in HOT and when you required to you need to
develop custom resource in Python. Independence on custom
plugins is
not good for Murano because they cannot be uploaded by end users and
thus he cannot write application definition that can be imported
to/run
on any cloud and need to convince cloud administrator to install his
Python plugin (something that is unimaginable in real life).
Shouldn't Mistral be able to do all of those same things too?
Yes it should. In ideal world. Assembly language can do everything
Python can do but you do not expect people to use assembly unless they
have to. The same is true for Mistral - although it can possibly
improved to do all above but you need to be real genius to write
something like auto-scaling in Mistral DSL.
I'm still confused by this autoscaling example. We can all agree that
clearly you would be insane to write your own autoscaling implementation
(whatever this means) in any language when your cloud provider already
has a highly-customisable one like Heat. But if you were to write your
own the way to do it would be in a programming language of your choice
running on a server you control. I still don't see how this has anything
to do with Mistral.
You listed above a list of features that, I submit, are required for
both Mistral and Murano. You are proposing to implement them twice,
rather than once.
We are working closely with Mistral team and maybe once we would be able
to lay all MuranoPL tasks on Mistral shoulders and consume Mistral DSL
instead of our own, but this is not going to happen soon.
When the TC said "Murano is slightly too far up the stack at this point
to meet the "measured progression of openstack as a whole" requirement",
IMO one of the major things they meant was that you're inventing your
own workflow thing, leading to duplication of effort between this and
Workflow as a Service. (And Mistral folks are in turn doing the same
thing by not using the same workflow library, taskflow, as the rest of
OpenStack.) Candidly, I'm surprised that this is not the #1 thing on
your priority list because IMO it's the #1 thing that will delay getting
the project incubated.
Mistral DSL is
good for another use cases and its developers may not like the idea of
having Turing-complete language in Mistral.
They sound very wise ;)
Talking to existing OpenStack services doesn't seem hard - you could
write plugins for those, or save time (and save the user learning
another language) by using python-openstackclient and its syntax.
For everything else you have a small number of generic operations -
e.g. post to a Marconi queue (ReST API calls to untrusted services
are problematic from a security perspective) - and allow the user to
handle them in their own code in a language of their choice running
either on their own machine or on a sandboxed, metered Compute
server, rather than in a custom Turing-complete DSL running
unmetered on the Murano server.
This is usually true. Usually, not alway. I can easily see how workflow
may need to talk to some cloud operator proprietary services that are
not accessible from VMs for security reasons. Workflow may need to
I said for everything *else*. For things the operator specifically wants
you to have access to (e.g. some proprietary service they run), they
would have the option of installing a native command plugin to handle
it. For everything *else* you drop a message in a Marconi queue and let
the user handle it.
acquire some licenses before instantiating VMs. It may require to talk
Where is the license server running? Can you stick a process on it to
listen for Marconi messages and generate license requests to the actual
server?
to cloud operator's billing system or quotas management system or
identity management system. There are many things that cannot be done
from VMs or need to be done before VM is spawned.
Calling external REST APIs is secure as long as you control the process
(can limit amount of traffic, request rate, timeouts, ACLs etc). This is
possible with MuranoPL.
And once again - everything that is done on server-side can be metered
and constrained.
It's only secure in a very narrow sense. You're effectively allowing
people to use your servers to make any request they like, and that's
going to be highly prone to confused-deputy-type problems. The only
completely safe URLs to fetch are ones that are in the Keystone catalog,
passing the token obtained from the user. (We break this rule in Heat
BTW, and we should stop. It's hard once you have users though, and even
harder when we can't depend on Marconi yet.)
cheers,
Zane.
Because DSL is a way to write custom resources (in Heats
terminology) it
has to be Turing-complete and have all the characteristics of
general-purpose language. It also has to have domain-specific
features
because we cannot expect that DSL users would be as skilled as Heat
developers and could write such resources without knowledge on
hosting
engine architecture and internals.
HOT DSL is declarative because all the imperative stuff is hardcoded
into Heat engine. Thus all is left for HOT is to define "state
of the
world" - desired outcome. That is analogous to Object Model in
Murano
(see [1]). It is Object Model that can be compared to HOT, not
DSL. As
you can see it not more complex than HOT. Object Model is what
end-user
produces in Murano. And he event don't need to write it cause it
can be
composed in UI.
Now because DSL provides not only a way to write sandboxed
isolated code
but also a lot of declarations (classes, properties, parameters,
inheritance and contracts) that are mostly not present in Python we
don't need Parameters or Output sections in Object Model because
all of
this can be inferred from resource (classes) DSL declaration.
Another
consequence is that most of the things that can be written wrong
in HOT
can be verified on client side by validating classes' contracts
without
trying to deploy the stack and then go through error log debugging.
This is being worked on:
https://blueprints.launchpad.__net/heat/+spec/param-__constraints
<https://blueprints.launchpad.net/heat/+spec/param-constraints>
I agree that contracts can become part of HOT. Note that MuranoPL
contracts are orders of magnitude more powerful than parameter
constraints. This don't necessary mean that HOT needs that powerful
capabilities. I'll be happy to collaborate on this in another thread/place
Because all resources' attributes types their constraints are
known in
advance (note that resource attribute may be a reference to another
resource with constraints on that reference like "I want any
(regular,
Galera etc) MySQL implementation") UI knows how to correctly
compose the
environment and can point out your mistakes at design time. This is
similar to how statically typed languages like C++/Java can do a
lot of
validation at compile time rather then in runtime as in Python.
Personally I would love to see many of this features in HOT. What is
your vision on this? What of the mentioned above can be
contributed to
Heat? We definitely would like to integrate more with HOT and
eliminate
all duplications between projects. I think that Murano and Heat are
complimentary products that can effectively coexist. Murano provides
access to all HOT features and relies on Heat for most of its
activities. I believe that we need to find an optimal way to
integrate
Heat, Murano, Mistral, Solum, Heater, TOSCA, do some integration
between
ex-Thermal and Murano Dashboard, be united regarding Glance
usage for
metadata and so on.
+1
To me that implies that Murano should be a relatively thin wrapper
that ties together HOT and Mistral's DSL.
We are okay with throwing MuranoPL out if the issues
it solves would be addressed by HOT.
If you have a vision on how HOT can address the same domain MuranoPL
does or any plans for such features in upcoming Heat releases I
would
ask you to share it.
[1]
https://wiki.openstack.org/__wiki/Murano/DSL/Blueprint#__Object_model
<https://wiki.openstack.org/wiki/Murano/DSL/Blueprint#Object_model>
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
