Re: [openstack-dev] [Murano] Heat resource isolation withing single stack
On 22/02/14 03:37, Stan Lagun wrote: Hi Everyone, While looking through Heat templates generation code in Murano I've realized it has a major design flaw: there is no isolation between Heat resources generated by different apps. Every app manifest can access and modify its environment stack in any way. For example it can delete instances and other resources belonging to other applications. This may be not so bad for Murano 0.4 but it becomes critical for AppCatalog (0.5) as there is no trust relations between applications and it may be unacceptable that untrusted application can gain complete write access over the whole stack. There is also a problem of name collisions - resources generated by different applications may have the same names. This is especially probable between resources generated by different instances of the same app. This also affects Parameters/Output of Heat templates as each application instance must generate unique names for them (and do not forget them later as they are needed to read output results). I think we need at least to know how we going to solve it before 0.5 Here is possible directions i can think of: 1. Use nested Heat stacks. I'm not sure it solves naming collisions and that nested stacks can have their own Output 2. Control all stack template modifications and track which resource was created by which app. Give applications read-only access to resources they don't own 3. Auto-generate resource names. Auto-add prefixes/suffixes to resource/output etc names indicating owning app instance ID and remove them upon read access from workflow so that generated names would be invisible to workflow. That would also mean all VMs would have generated names Hope to see better ideas and suggestions in this thread I would like the Murano project to evaluate the upcoming heat software-config resources to represent app configuration. Some patches are still in review [1][2] and I expect more changes before feature freeze, but now would be a good time for Murano to get involved to validate that software-config meets your needs. The current implementation of the SoftwareDeployment resource actually has the same issue mentioned above; any authenticated user can currently create a deployment which results in software being deployed to any server in that tenant/project. There may be times where this is useful, but there will soon be a default policy enforced which ensures that the deployment resource and the server resource belong to the same stack. Inferring from the above, it sounds like Murano is transforming an app definition into a single flat heat template. Using a combination of resource providers and software config should give you a significantly cleaner model, and you may even find that some of the problems you're solving in Murano are now solved in the Heat layer. [1] https://review.openstack.org/#/q/status:open+topic:bp/hot-software-config-rest,n,z [2] https://review.openstack.org/#/q/status:open+topic:bp/hot-software-config,n,z ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Heat resource isolation withing single stack
I would like the Murano project to evaluate the upcoming heat software-config resources to represent app configuration. Yes, we are closely watching progress being made in Heat in that area and evaluating how all new great features can be used in Murano Inferring from the above, it sounds like Murano is transforming an app definition into a single flat heat template. It is just not that simple. Murano by itself neither generates any templates nor deploys applications. Murano is just App Catalog. Application definitions that Catalog consists of are more then just a declarations. They also contains imperative executable parts written in Murano DSL language. Murano itself can be viewed as a runtime for application manifests. Murano engine provides carefully designed APIs for DSL code to access Heat and other OpenStack services. Those APIs are more restrictive and safe than native OpenStack APIs (provided by various python-clients) and DSL cannot access OpenStack services by any means other than those APIs. Heat API is one of them. With this API DSL can access Heat stack and make some changes in it. It is up to Murano users (software publishers) who fill AppCatalog to decide how and if to use Heat. They are free to use Heat just for infrastructure-level management and do software config by some external tools or use new software-config in Heat or even directly talk to Nova, Trove etc. But no matter how application decides to implement it, Murano must guarantee that DSL code of one app cannot damage or compromise OpenStack resources that belongs to another app. So even if new Heat feature makes life easier we still need to guarantee safe behavior of DSL when using old Heat template syntax or older Heat version that doesn't have that feature. On Sun, Feb 23, 2014 at 11:50 PM, Steve Baker sba...@redhat.com wrote: On 22/02/14 03:37, Stan Lagun wrote: Hi Everyone, While looking through Heat templates generation code in Murano I've realized it has a major design flaw: there is no isolation between Heat resources generated by different apps. Every app manifest can access and modify its environment stack in any way. For example it can delete instances and other resources belonging to other applications. This may be not so bad for Murano 0.4 but it becomes critical for AppCatalog (0.5) as there is no trust relations between applications and it may be unacceptable that untrusted application can gain complete write access over the whole stack. There is also a problem of name collisions - resources generated by different applications may have the same names. This is especially probable between resources generated by different instances of the same app. This also affects Parameters/Output of Heat templates as each application instance must generate unique names for them (and do not forget them later as they are needed to read output results). I think we need at least to know how we going to solve it before 0.5 Here is possible directions i can think of: 1. Use nested Heat stacks. I'm not sure it solves naming collisions and that nested stacks can have their own Output 2. Control all stack template modifications and track which resource was created by which app. Give applications read-only access to resources they don't own 3. Auto-generate resource names. Auto-add prefixes/suffixes to resource/output etc names indicating owning app instance ID and remove them upon read access from workflow so that generated names would be invisible to workflow. That would also mean all VMs would have generated names Hope to see better ideas and suggestions in this thread I would like the Murano project to evaluate the upcoming heat software-config resources to represent app configuration. Some patches are still in review [1][2] and I expect more changes before feature freeze, but now would be a good time for Murano to get involved to validate that software-config meets your needs. The current implementation of the SoftwareDeployment resource actually has the same issue mentioned above; any authenticated user can currently create a deployment which results in software being deployed to any server in that tenant/project. There may be times where this is useful, but there will soon be a default policy enforced which ensures that the deployment resource and the server resource belong to the same stack. Inferring from the above, it sounds like Murano is transforming an app definition into a single flat heat template. Using a combination of resource providers and software config should give you a significantly cleaner model, and you may even find that some of the problems you're solving in Murano are now solved in the Heat layer. [1] https://review.openstack.org/#/q/status:open+topic:bp/hot-software-config-rest,n,z [2] https://review.openstack.org/#/q/status:open+topic:bp/hot-software-config,n,z
[openstack-dev] [Murano] Heat resource isolation withing single stack
Hi Everyone, While looking through Heat templates generation code in Murano I've realized it has a major design flaw: there is no isolation between Heat resources generated by different apps. Every app manifest can access and modify its environment stack in any way. For example it can delete instances and other resources belonging to other applications. This may be not so bad for Murano 0.4 but it becomes critical for AppCatalog (0.5) as there is no trust relations between applications and it may be unacceptable that untrusted application can gain complete write access over the whole stack. There is also a problem of name collisions - resources generated by different applications may have the same names. This is especially probable between resources generated by different instances of the same app. This also affects Parameters/Output of Heat templates as each application instance must generate unique names for them (and do not forget them later as they are needed to read output results). I think we need at least to know how we going to solve it before 0.5 Here is possible directions i can think of: 1. Use nested Heat stacks. I'm not sure it solves naming collisions and that nested stacks can have their own Output 2. Control all stack template modifications and track which resource was created by which app. Give applications read-only access to resources they don't own 3. Auto-generate resource names. Auto-add prefixes/suffixes to resource/output etc names indicating owning app instance ID and remove them upon read access from workflow so that generated names would be invisible to workflow. That would also mean all VMs would have generated names Hope to see better ideas and suggestions in this thread -- Sincerely yours Stanislav (Stan) Lagun Senior Developer Mirantis 35b/3, Vorontsovskaya St. Moscow, Russia Skype: stanlagun www.mirantis.com sla...@mirantis.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Heat resource isolation withing single stack
On Fri, Feb 21, 2014 at 06:37:27PM +0400, Stan Lagun wrote: Hi Everyone, While looking through Heat templates generation code in Murano I've realized it has a major design flaw: there is no isolation between Heat resources generated by different apps. Can you define the requirement for isolation in more detail? Are you referring simply to namespace isolation, or do you need auth level isolation, e.g something enforced via keystone? Every app manifest can access and modify its environment stack in any way. For example it can delete instances and other resources belonging to other applications. This may be not so bad for Murano 0.4 but it becomes critical for AppCatalog (0.5) as there is no trust relations between applications and it may be unacceptable that untrusted application can gain complete write access over the whole stack. All requests to Heat are scoped by tenant/project, so unless you enforce resource-level access policy (which we sort-of started looking at with OS::Heat::AccessPolicy), this is expected behavior. There is also a problem of name collisions - resources generated by different applications may have the same names. This is especially probable between resources generated by different instances of the same app. This also affects Parameters/Output of Heat templates as each application instance must generate unique names for them (and do not forget them later as they are needed to read output results). A heirarchy of nested stacks, with each application defined as a separate stack seems the obvious solution here. I think we need at least to know how we going to solve it before 0.5 Here is possible directions i can think of: 1. Use nested Heat stacks. I'm not sure it solves naming collisions and that nested stacks can have their own Output I think it does, and yes all stacks can have their own outputs, including nested stacks. Of particular interest to you may be the provider resource interface to nested stacks, which will allow you to define (via a series of nested stack templates) custom resource types defining each of your applications. See this old blog post, which will give you the providers/environments 101, and contains links to most of the related heat docs: http://hardysteven.blogspot.co.uk/2013/10/heat-providersenvironments-101-ive.html 2. Control all stack template modifications and track which resource was created by which app. Give applications read-only access to resources they don't own I think we need more info on the use-case here, but perhaps you can either use the AccessPolicy resource, or we can work on defining an enhanced version which meets your requirements. 3. Auto-generate resource names. Auto-add prefixes/suffixes to resource/output etc names indicating owning app instance ID and remove them upon read access from workflow so that generated names would be invisible to workflow. That would also mean all VMs would have generated names Heat already does this internally, we create unique names for all your instances, unless you explicitly provide a name via the OS::Nova::Server name property. It might help if you could provide a really simplified example of the problem you are facing, or links to the real templates which we could review and make suggestions? Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Murano] Heat resource isolation withing single stack
Steve, thank you for very valuable suggestions. Your block post is really
great - I've read about environments in Heat documentation but didn't
really understood them until now.
Usage of nested stacks may or may not solve my problem depending on what is
possible to do within those stacks.
Let me explain with simple example.
As you probably know Murano uses Heat for all infrastructure-related
operations. This means if some application from Catalog needs VM instance
or any other type of OpenStack resource it creates it by inserting a
snippet into user's Heat stack template and executes UPDATE STACK command.
Now suppose there is WordPress application published in App Catalog.
WordPress app manifest says that it requires installation of MySql. There
is also another application in AppCatalog called GaleraMySql that is known
to be compatible with MySql. In Murano Dashboard user creates new
environment (this corresponds to Heat stack and is not related to what is
called environment in Heat)
and puts WordPress and GaleraMySql on it. Then he connects them so that
GaleraMySql instance would be used in WordPress for MySql requirement.
WordPress and GaleraMySql were developed by different vendors that are not
aware of each others presence. But because of unfortunate combination of
circumstances both vendors chose to merge exactly the same snippet into
user's stack:
Resources: {
myHost: {
Type: AWS::EC2::Instance,
Properties: {
InstanceType: large,
ImageId: someImage
}
}
}
Then instead of 2 different VMs there would be only one. Things would be
even worse if there was already resource myHost in user's stack.
It is more than a name-collision problem as incorrectly written application
manifest can cause any imaginable harm to the stack.
The obvious solution would be to give each app dedicated nested stack and
restrict it to that nested stack only. This would be a best solution. All I
need is to have the same level of control on nested stack I have on outer
stack - get stack template, modify and update them, access output
attributes. Is it possible to retrieve nested stack template, modify it and
populate it back to Heat?
Another option would be create separate top-level stacks for each app. But
in Murano applications themselves composed of smaller parts and in practice
this would lead to creation of dozen stacks with most of them containing
single resource. And then we would have to implement transaction update
between several stacks, coordinated deletion etc. This would also be bad
from a user's point of view at he doesn't expect to find long list of
stacks he has no idea where they came from.
My other options were on how nested stacks can be emulated on top of single
stack by controlling which app created which resource and dynamically
adjust resource names back and forth (myHost in example above) to some
unique values in a way that is opaque to application
On Fri, Feb 21, 2014 at 8:20 PM, Steven Hardy sha...@redhat.com wrote:
On Fri, Feb 21, 2014 at 06:37:27PM +0400, Stan Lagun wrote:
Hi Everyone,
While looking through Heat templates generation code in Murano I've
realized it has a major design flaw: there is no isolation between Heat
resources generated by different apps.
Can you define the requirement for isolation in more detail? Are you
referring simply to namespace isolation, or do you need auth level
isolation, e.g something enforced via keystone?
Every app manifest can access and modify its environment stack in any
way.
For example it can delete instances and other resources belonging to
other
applications. This may be not so bad for Murano 0.4 but it becomes
critical
for AppCatalog (0.5) as there is no trust relations between applications
and it may be unacceptable that untrusted application can gain complete
write access over the whole stack.
All requests to Heat are scoped by tenant/project, so unless you enforce
resource-level access policy (which we sort-of started looking at with
OS::Heat::AccessPolicy), this is expected behavior.
There is also a problem of name collisions - resources generated by
different applications may have the same names. This is especially
probable
between resources generated by different instances of the same app. This
also affects Parameters/Output of Heat templates as each application
instance must generate unique names for them (and do not forget them
later
as they are needed to read output results).
A heirarchy of nested stacks, with each application defined as a separate
stack seems the obvious solution here.
I think we need at least to know how we going to solve it before 0.5
Here is possible directions i can think of:
1. Use nested Heat stacks. I'm not sure it solves naming collisions and
that nested stacks can have their own Output
I think it does, and yes all stacks can have their own outputs,
